Customer Identification and Fraud Detection Using Automatic Speech Recognition

It’s standard practice for banks to conduct their customer due diligence (KYC) process before offering services. The KYC process primarily includes document verification, such as utility bills, address verification, and biometric verification. Hence, it's easy for agents to learn the specifics of a call for an inquiry or request.

However, it's challenging for agents to determine the identity of the person conversing on the other side of the call. Due to this predicament, the customer authentication process consumed agents' time without offering a foolproof system to identify fraudsters against genuine customers.

Fraudsters today have numerous tactics to learn a customer's identity and confidential information. The client wanted to introduce automatic voice recognition to offer safer and quicker means of conducting bank-related transactions.

The key goals were to overcome challenges such as decreasing the turnaround time (i.e., 45-60 seconds/customer) for authenticating customers on calls, offering effortless voice authentication instead of PIN authentication, and mitigating fraud risks from compromised customer information through social engineering and phishing.

Passive voice biometrics leverages voice recognition technology to identify individuals based on their unique vocal characteristics when speaking passively or naturally. To achieve this, our team designed a web portal that would seek consent from customers to create a voice pass, enroll the voice pass on the IVR, and register the customer once their voiceprint is generated.

We designed a two-part solution covering a voiceprints management portal and a fraud detection application. Here's a brief overview of the implemented solution -

    1. Voice Biometric Enrollment

Human ears cannot perceive subtle traits that help recognize the human voice. But biometric solutions powered with artificial intelligence can do this effortlessly. To design this solution, we deployed a voice recognition platform that compares a caller's voice with a passive voiceprint stored in the database.

A passive voiceprint is a mathematical impression of a customer's voice created during the initial interaction with the platform. Once a customer's real-time voice finds a match with the passive voicepass in the database, the platform apprises the agent that the caller has been verified.

We implemented the voice recognition solution using Phonexia (a voice biometric and speech recognition software). Our foremost task with new speakers was to add their voice samples to the database to identify them and eventually leverage them to train the voice-assisted self-service model.

Designing the voice identification and enrollment process involved the following steps:

1. The Agent receives a call from a particular number.
2. The system checks if this is an already existing contact or a new caller.
3. The system examines audio for an existing contact and compares it with the registered voice biometric.
4. If the voice matches the registered passive voice biometric, the user is authenticated to access bank services.
5. If the system doesn't authenticate the caller, the system guides the agent to ask security questions.
6. Users are authenticated if they answer the security questions correctly; otherwise, the system marks the caller as a fraud.
7. New speakers with unregistered passive voice print are asked for consent to enroll.
8. Agents observe the customer ID and consent to enroll on the ITQAN CRM.
9. Collect a 20-second voice sample using Phonexia. 
10. The collected voice print is passed to the Admin to enroll the voice sample in the speaker's tab. 
11. Admin assigns a Speaker ID and a Voiceprint ID at enrollment time.
12. The speaker tab includes customer ID (CID), registered phone number, first and last name, voiceprint ID, and voiceprint status (Active-Inactive-Locked).
13. Admin can sort or search for a particular voiceprint using the CID, voiceprint status, date of creation, and date last used.
14. The speaker's tab doesn't reflect confidential bank details such as account numbers, answers to authentication questions, addresses, etc.

    2. Fraud Detection

Fraudsters have a bag full of tricks that can deceive even the most experienced customer care representatives. For most agents, it’s difficult to perceive that they’re being victims of fraud.

The voice portal would learn verbal cues from the caller’s voice, compare it with the unique traits in the voice sample saved from passive voice biometrics, and alert the agent about the suspicious demeanor of the call.

Any suspicious callers would be added to a distinct repository. These voiceprints would eventually be used to educate the voice recognition model on how to spot and flag similar future interactions.