Audio Content Classification Using Python-based Predictive Modeling
Expertise Delivered
Industry
Our client,
Core Nova specializes in delivering fully customized SaaS solutions. They also offer customer service and marketing lead generation support across various business verticals such as banking, insurance, and automobile through its 500+ agents.
Disclaimer: The name ‘Core Nova’ is a placeholder, as there is an NDA signed between both parties.
Project Scope
Core Nova, a SaaS solutions provider, works with clients from multiple industries. Their most significant client base is political parties. The company’s 500+ telemarketing agents are appointed to make outbound calls to potential political donors and voters to get them interested in funding their client’s political campaigns.
Making sales pitches on the phone to an answering machine is an unnecessary waste of an agent’s time. If the company could avoid such bootless errands, they could save a lot of money and time in business operations.
Core Nova was looking to overcome this challenge by building a predictive model that could quickly identify whether the agent was speaking to a human or non-human, i.e., an answering machine, fax machine, dead audio, scanner machine, etc.
If the call is connected to a non-human, it should disconnect immediately and move on to the next number, so the agent does not waste their time. On the other hand, if a human picks up the call, it is routed to the agent for further conversation.
Challenge
Core Nova’s existing model for detecting audio inputs was built using Asterisk, an open-source framework for building communications applications.
However, this existing model had an accuracy of 60% after a timeframe of 3 seconds. The client was looking to develop a model that could determine the probability of whether it was a human or a machine on the other side of the phone within one second, with an accuracy of >90%.
Moreover, there was another critical challenge of overlapped audio patterns.
In multiple audio inputs, the audio characteristics were similar in the first 500 milliseconds, when the model was tested in a LIVE environment. This made it difficult for the model to determine whether the audio input was human or non-human.
In clustering, the algorithm grouped the audios into 2 clusters based on the audio characteristics. The hypothesis is that all the Answering Machine (AM) audios should fall in one cluster, and that of Human Answered (HA) should fall in the other cluster.
However, in the live scenario, it was seen that 73% AM and 27% HA (1st pie chart) fell in one cluster, which means their audio characteristics were similar. It quashed our hypothesis and proved that there is audio characteristic overlap in HA and AM audios.
The audio characteristics were not correctly labeled, contributing to a drop in the model’s accuracy. The audio patterns needed to be distinct for the model to achieve 90% accuracy.
Why Maruti Techlabs
Core Nova evaluated a series of vendors before signing the partnership with Maruti Techlabs.
We helped Core Nova get started by taking a top-down approach with an AI Readiness Audit.
This involved really validating the idea, through qualitative and quantitative analysis of their datasets, identifying the best-fit approach to model development, and putting together an implementation roadmap.
All this before writing a single line of code, and investing heavily into the idea.
What impressed them about us, was our track record of working with complex data structures and building predictive analytics/machine learning models.
This, together with our case studies and the AI Readiness Audit, made it clear to Core Nova that we had walked the walk, and they could trust us to see this through.
Solution
Core Nova was looking to build an effective model to identify the source on the other side of the call. They wanted to save valuable time and company resources by only routing human-answered calls directly to the appropriate agent.
After finalizing the partnership with us, we proceeded to conduct a feasibility study on the data provided by the client.
1. AI Readiness Audit:
After thoroughly understanding the use case and requirement, the first thing we worked on was conducting an audit. This spanned four weeks, wherein our team defined the scope of the solution and conducted a detailed analysis of the platform's current state.
During the audit phase, our AI experts analyzed if there are patterns that can identify whether the audio is of a Human or an Answering Machine. The goal here was to classify the audio input into HA or Non-HA within the first 1 sec of the audio samples provided by Core Nova.
Wave plot: The wave plot represents the frequency at different sample rates for the first 27ms audios.
We worked with a sample of the audio files to determine the feasibility of the desired outcome. After creating the audio training files acquired from the client's database, we filtered, organized, and labeled the dataset to make it search-friendly. Our AI experts studied, matched the sample data, and defined approaches to build the predictive model.
2. Model Development:
The team of data scientists at Maruti Techlabs developed a Python-based predictive model that could predict the audio characteristic as HA or Non-HA with a high success rate, within the first 500 milliseconds of the audio input.
The model enhanced backend operations by integrating with the client’s existing tech stack, increasing efficiency, and strengthening the existing systems.
Although the challenge of overlapping audio patterns could not be eradicated, the implementation of our solution made the system faster and more accurate than the client’s existing solution.
The team went about the development of the Answering Machine Detection Solution in the following steps -
- Dataset Creation - The dataset is composed of audio recordings of 2 categories - Non-Human and Human.
- Dataset Preprocessing - The audio recordings are in the form of complete call recordings. These recordings need to be sliced into smaller periods, and the data extracted from these files will be used to feed the model.
Zero-crossing rate: The zero-crossing rate refers to how a signal changes from positive to zero to negative or from negative to zero to positive.
- Model Development & Training - In this step, we developed the model and trained it on the training data.
- Model Assessment - This involved testing the model against performance benchmarks.
The team initially built a model for 300ms, 500ms, 700ms, and 900ms, out of which we selected the 500ms model. To verify the new audios and correctly label the mislabelled audios, we constantly communicated with the client.
A final model was created for the live environment through careful correction and further testing.
Validate your AI vision, before you invest any resources. Opt for our AI Readiness Audit
Communication & Collaboration
The Maruti Techlabs team communicated with the client regularly, ensuring no gaps in expectations via weekly sync-up calls using Zoom.
For daily communication, we used Slack and email. The timelines and roadmaps for the project were maintained in a Google spreadsheet and JIRA.
We deployed the following team for the development phase of the project.
- AI Architect - to manage the entire project from start to finish across the business units, DataOps, MLOps, and the extended engineering team.
- ML Engineers - responsible for building, deploying, and scaling models in production-ready environments, along with ongoing feedback from the data to ensure their model is continuously improving.
- Data Scientists and Engineers - focus on data integration, modeling, optimization, and quality. They oversee and handpick the suitable datasets and algorithms to build the model with the team of ML Engineers.
Technology Stack
Our Development Process
We follow Agile, Lean, & DevOps best practices to create a superior prototype that brings your users’ ideas to fruition through collaboration & rapid execution. Our top priority is quick reaction time & accessibility.
We really want to be your extended team, so apart from the regular meetings, you can be sure that each of our team members is one phone call, email, or message away.
Our data analysis process is oriented towards drawing maximum value out of every decision made in every department of your company. We strive to deliver analytics, reports, BI, and predictions of increasing accuracy to solve your problems, sometimes even before they crop up.
AI Readiness Audit
Finding the right AI partner is no easy task.
Building an AI product that delivers on value, is even more challenging.
We help you get started with a slightly different approach. Before we get into the trenches and kickstart development, we take a top-down approach with an AI Readiness Audit.
This involves really validating the idea, through qualitative and quantitative analysis of your datasets, identifying the best fit approach to model development, and putting together an implementation roadmap.
All this before writing a single line of code, and investing heavily into the idea.
Achieve more with less.
