Optimizing Workflows: Medigap Life Achieves 88% Reduction in Execution Time
Expertise Delivered
Industry
Our client,
Medigap Life, is a prominent online insurance aggregator based in Florida, USA. Established in 2016, Medigap Life emerged as a key player in the insurance space, particularly within the medicare domain.
Medigap Life’s business operates on a two-fold business model. They help top insurance providers reach an untapped market of low-income households, thus making health insurance an accessible tool for all. Simultaneously, they help policy buyers make informed decisions by enabling online policy comparisons and digital policy purchases.
Presently, Medigap Life proudly ranks among the foremost digital insurance platforms in the United States.
Challenge
The client, Medigap Life, partners with a multitude of insurance providers and maintains strong connections with a sizable audience of insurance buyers.
One of the biggest challenges that the client’s team was grappling with was managing a large volume of customer data through their traditional CRM workflows. Given the size of their clientele, modernizing their workflows for seamless operations was the need of the hour.
Their existing CRM application, vTiger, had closely interconnected workflows. Due to excessive interdependencies, the system struggled to handle multiple requests simultaneously, resulting in a notable decline in performance.
The CRM lacked flexibility and configurability. For example, it couldn't handle cases where the client wanted to process only 5000 out of 20000 records. It was also not able to handle the execution of the workflows at predefined time intervals.
Additionally, the system's record processing followed a strict sequential pattern, which made it incapable of handling multiple requests simultaneously. Even basic functions, such as sending out daily batch messages (marketing SMSes to 25,000 users daily), required an extensive 7 to 8 hours.
This resulted in their agents having to wait for a longer duration to get a reply and engage the users further. As the workload grew, it further caused a lot of strain, leading to a considerable decline in performance and overall efficiency.
Why Maruti Techlabs?
Medigap Life was actively seeking a skilled and capable team to help them define, expand, and execute their CRM and workflow optimization initiative seamlessly. They came across our blogs on the subject and reached out to us.
The client’s tool suffered scalability issues and inefficiencies in speed, performance, and functionalities.
We conducted a collaborative workshop with the client's team to understand their requirements thoroughly and outline a clear project scope. The workshop proved highly valuable as it exemplified our structured approach to project management and also highlighted our commitment to truly understand business goals before getting started.
This approach allowed us to lay a strong foundation for the successful execution of the project and instilled confidence in the client about our capabilities.
“Maruti Techlabs has demonstrated an exceptional project management system. Their deployment is meticulously organized, which helps them consistently meet deadlines. Their accuracy in terms of understanding the work scope is also commendable.”
- CTO, Sean Chapman.
Solution
After carefully considering all the challenges, Medigap Life realized the importance of modernizing its CRM workflow. They wanted an experienced technical team that could upgrade their CRM workflow to streamline processes, automate repetitive tasks, enable faster response times, and execute this migration without disrupting their business processes. In addition, they were particular about the scalability and flexibility of the new CRM system.
When they reached out to Maruti Techlabs with these requirements, we insisted on conducting a comprehensive workshop to meticulously document their requirements, ensuring that every intricate detail was considered. Upon completing the workshop, we began the development process, launched the beta, executed QA tests, and successfully deployed the product.
Here is a detailed overview of the execution of this project -
1. Discovery Workshop
After a comprehensive analysis of Medigap Life's requirements, we engaged in close collaboration with the client's team to gain a deeper understanding of their existing system.
The client wanted to improve and optimize the current workflows within their existing CRM system (vTiger CRM). vTiger is a free, open-source CRM application that the client was already using to manage their interactions with customers, track sales, and monitor their marketing activities. Our team of technical experts examined the intricacies of each module and understood the processes within their workflow.
This immersive approach allowed us to identify their pain points and grasp the scope of improvement more effectively, providing a clear roadmap for the CRM modernization and workflow optimization initiative.
By the end of the workshop, we outlined the detailed scope of work as follows -
- Set up an application to get notifications from the CRM system (when a lead is created/updated) to execute a particular workflow through the workflow management and automation tool (Airflow DAG).
- Create the following tasks in the workflow management system:
- Start task - Set a trigger to start the workflow
- Condition task - Set conditions to direct the workflow.
- WebService task - Set up a call to web service or API. It helps Interact with external services, retrieve data, or send data to external systems.
- Set values task - Assign or modify values within the workflow.
- External provider task - Set up interactions with external providers or services.
- External workflow task - Call an external workflow based on the condition met.
- Delay Task: Introduce a pause or wait period within a workflow to control the timing of task execution and to introduce time gaps as needed.
- Sample workflow: Build 5 sample workflows using the above tasks in Airflow.
- Miscellaneous - Monitoring, tracking, and storing workflow logs.
- Create a Twilio provider for SMS.
- Design API to Invoke Airflow.
Considering the complexity of the overall process, we categorized the scope of work into manageable sprints with minimal ambiguity. This approach was pivotal for enabling development teams to start promptly and transition seamlessly into the implementation phase. The detailed scope of work and blueprint facilitated the agile adoption of a sprint based delivery model, ensuring that the project progresses efficiently and delivers tangible results steadily.
2. Workflow Migration Planning
We identified that the existing implementation contained over a hundred interdependent workflows. The client wanted to migrate all the workflows to the new system.
However, considering their high performance and scalability requirements, we strategically narrowed our focus to the most critical workflows. We sorted and prioritized the essential workflows which were centered primarily around customer acquisition and retention campaigns, along with data sync from call center platform to vTiger CRM.
3. Migrating to Apache Airflow
Adhering to the client’s need for a distributed, configurable, and scalable tool, we migrated the selected workflows to Apache Airflow. Apache Airflow enables horizontal scaling of workflows to accommodate an increasing workload.
In addition to this, Airflow provides monitoring capabilities, parallel task execution, and time-based and event-based scheduling. This would solve most of the existing pain points within the existing CRM tool.
However, to fully harness the power of Airflow, we leveraged Amazon Elastic Kubernetes Service (EKS) along with Amazon Elastic Compute Cloud (EC2), which provided the scalable compute infrastructure needed to run the workflows in Airflow with lightning speed. We further assembled an EKS cluster - a group of powerful computing units ready to execute Airflow tasks at scale.
Amazon Simple Storage Service (S3) and Amazon Relational Database Service (RDS) were further utilized to store and safeguard sensitive customer data. The workflows could now seamlessly access and process data stored in S3 buckets and RDS (with data interceptor service), ensuring a smooth and uninterrupted flow of information.
Finally, we employed Amazon CloudWatch within the Airflow environment to ensure comprehensive monitoring and logging capabilities. With CloudWatch's watchful eye, Medigap Life could closely monitor the performance of their workflows, swiftly identify any potential issues, and maintain the smooth operation of their data processing pipeline.
- Steps we took to migrate to Apache Airflow -
- Apache Airflow installation
- Identify and define DAGs for each workflow, representing the sequence of tasks and their dependencies
- Execute task migration
- Handle dependencies
- Set up scheduling parameters for each DAG
- Handle data dependencies
- Testing and deployment
- Monitoring and optimization
4. Set up notifications from vTiger
We enabled real-time notifications from the vTiger CRM based on customer interactions and triggered specific workflow execution on meeting pre-established conditions.
This strategic integration aimed to elevate automation and responsiveness in data workflows, effectively addressing the client's concerns regarding slow performance.
Steps we took for setting up notifications from vTiger -
- Configure Webhooks in vTiger to trigger event-based notifications
- Select the notification channel
- Create and configure a notification receiver
- Integrate with Airflow
- Trigger Airflow DAG execution
- Error handling and logging
- Test the setup
- Monitor and maintain
5. Create a Twilio Provider for SMS
Another pressing problem with the existing implementation was the slow execution of batch SMS messaging. To address this, we implemented a Twilio provider to send SMS messages. Twilio is a reliable SMS service provider that offers scalability, customization, and ease of integration.
Steps we took for creating a Twilio provider for sending SMS messages -
- Sign up for a Twilio account
- Install Twilio Python Library
- Import Twilio Library
- Initialize Twilio Client
- Send SMS
6. Design API to Invoke Airflow
To cater to the client’s requirement for dynamic execution of workflows, we designed an API (Application Programming Interface) to trigger and manage workflows, monitor their progress, and retrieve information related to workflow execution, like checking workflow status and retrieving logs or results. The API enabled them to trigger workflows effortlessly, define endpoints, pause or delay workflows, and run workflows at predefined time intervals.
Convert traditional siloed structures into streamlined workflows.
“Most of our communications are handled through weekly and semi-weekly phone calls, along with Slack communications.”
- CTO, Sean Chapman.
Communication and Collaboration
The team at Maruti Techlabs maintained consistent and effective communication with the client, ensuring a clear understanding of expectations and project progress. During a two-week workshop, we collaborated closely with the client's core team.
Our technical architect worked with Medigap Life's CTO to define the product roadmap. In response to their needs, Maruti Techlabs assembled a dedicated team, including:
- Backend Engineers
- Frontend Engineers
- DevOps Engineers
- QA Engineers
- Technical Architect
We held weekly calls for updates and discussions to ensure the development progress remained on track. The teams efficiently communicated using JIRA, Slack workspace, and email, creating a cohesive and streamlined approach for project success.
Technology Stack
“Maruti Techlabs developed one of our products from scratch and executed a flawless migration of our existing workflow to a state-of-the-art CRM system. Their proactive approach to user-centric design, including in-depth research and layout finalization, deserves special commendation. This partnership has been exceptionally rewarding, and we eagerly anticipate continuing our collaboration with them.”
- CTO, Sean Chapman.
Our Development Process
We follow Agile, Lean, & DevOps best practices to create a superior prototype that brings your users’ ideas to fruition through collaboration & rapid execution. Our top priority is quick reaction time & accessibility.
We really want to be your extended team, so apart from the regular meetings, you can be sure that each of our team members is one phone call, email, or message away.
