Lavender fields at sunset
Case Study
The better the question The better the answer The better the world works
Case Study

Using AI to improve a bank’s agent effectiveness

With help from EY, a commercial bank leveraged machine learning and natural language processing to improve sales workforce effectiveness.


How can AI bring structure to unstructured data to improve sales performance?

A scalable, AI-based approach unearths behaviors that drive successful outcomes and form the basis for ongoing performance management.

While digital is increasingly the channel that customers choose to interact with businesses, phone-based outreach is still the most commonly used engagement and sales tool. Yet, many companies lack a formulaic means of analyzing why some agents are more effective than others.

Performance analysis typically relies on supervisors listening to sampled calls, coupled with using surface-level metrics from small data sets. However, the feedback developed using this approach can be highly subjective and often generic. Being manual, the process is slow and difficult to scale, and does not provide a comprehensive view of agent performance. The inability to track metrics on an ongoing basis and across a full population of agent interactions makes it impossible to understand an agent’s performance trajectory. What’s more, it makes peer comparisons and delivering feedback difficult, which, in turn, can lead to resentment, lack of job satisfaction and, ultimately, staff churn.

To address these issues, the client needed a system that would provide greater visibility at the scale of agents’ activities (outreach, conversations, follow-up, etc.) and uncover behaviors to drive successful outcomes. The insights obtained from this analysis could, in turn, be used to develop a performance measurement and management framework with the ability to provide targeted coaching and quantify agent improvement over time.

To achieve this, the EY teams curated a robust, fact-based comparative data set. They leveraged sophisticated AI approaches to analyze conversation behaviors associated with overall progression and help identify behaviors demonstrated by top-performing agents. These insights were used to track individual team member’s performance and provide highly specific, personalized feedback. The insights were packaged into an interactive dashboard that agents and team leaders could access and use for ongoing performance review and actions.

The team needed to overcome the significant challenges of working with huge volumes of unstructured data, which is inherently noisy and often poor quality. Making the task even more complex was the difficulty of linking data from structured sources, such as customer relationship management platforms, agent hierarchy files, and performance or outcome data, to unstructured call data.

By operationalizing these insights powered by AI and machine learning, the client would be able to:

  • Gain a better understanding of what top performers were doing and apply these learnings to improve training, coaching and overall performance trajectory.
  • Strengthen the links between performance and incentives, and thus improve retention.
  • Improve follow-up efficiency with a 360-degree view of customer outreach.

A data-driven approach coupled with forensic attention to detail

Combining natural language processing-derived insights from agent conversations with sales progression metrics was key to success.

EY teams started by creating a “book of record” around the entirety of an agent’s interaction with a client. They captured what happened before and after across various channels – from what emails were sent and calls made, to cases opened and sales recorded.

In tandem, they used natural language processing (NLP) techniques to extract nuanced qualitative and unstructured insights from agents‘ conversations, such as topics discussed, talk track utilized and call outcome. This made it possible to gauge how agents responded to certain customer situations; whether they were invested in understanding the client’s specific situation; how they were talking about product features and functionality; and whether they were carrying out robust follow-up activity. Leveraging NLP methods overcame the data quality issues that typically drive down the performance of keyword-based approaches. These often yield unsatisfactory results due to issues including mis-transcription and contextually similar word variants.

“By addressing the quality issues inherent in conversations recorded by legacy systems and over-reliance on keywords, we were able to identify very subtle signals with a very high level of accuracy,” says Sameer Gupta, EY North America Financial Services Organization Advanced Analytics Leader.

Together with structured metrics, these foundational insights were tied to overall progression and outcome, for example, a sale recorded.

Using a variety of techniques to improve links between unstructured and structured data sets, EY teams were able to tie the foundational NLP insights to overall progression, thereby identifying gaps and opportunities by progression stage. In addition to agent-level insights, the approach helped understand and collate behaviors and methods correlated to impactful conversations and successful outcomes. The next step was to distill these insights into an interactive review dashboard. Developed with input from the client’s operations team and the end users to encourage adoption, it provides customized views for agents and team leaders that can be refreshed daily or when required.

By addressing the quality issues inherent in conversations recorded by legacy systems and over-reliance on keywords, we were able to identify very subtle signals with a very high level of accuracy.
Sameer Gupta
EY North America Financial Services Organization Advanced Analytics Leader

Actionable insights to drive individual performance

High-performance techniques leveraged for unstructured data insights generation have applications beyond sales.

Unlocking insights embedded in the unstructured data of agent conversations generated impressive results. The client identified an uplift in opportunities of upward of 50%.

By packaging these insights into an interactive dashboard that can be used by different stakeholders, EY professionals provided the client with a far more sophisticated foundation for performance measurement and management:

  • The dashboard provides a much broader and deeper view of agent activity, pinpointing weaknesses at the call-level that might be preventing optimal performance.
  • With full visibility of a range of successful techniques and performance drivers, team leaders now can provide highly targeted, personalized coaching, and track performance improvement over time.
  • By taking subjective assessment and feedback out of the mix, data-driven performance measurement helps eliminates bias.
  • With robust data at the heart of performance measurement, the incentive structure can be recalibrated and strengthened.
  • A data-driven, formulaic approach to performance analysis allows for faster training and onboarding processes.
The bigger picture

The project has helped the client to understand and track key drivers of effectiveness in a phone-based account development environment. It has also helped to provide the ability to offer targeted coaching. This allows agents to take a more individualized approach and, ultimately, improve performance through greater product engagement and upselling opportunities.

In addition, a better view of the leakage at each progression stage makes it easier to prioritize account follow-up. It also provides an opportunity to potentially automate part of the workflow. This visibility improves upstream analytics and supports more cost-effective investment in CRM, in refining prospect targeting, or related insights. Customers benefit too, as agents are empowered to have richer conversations and meet their needs with more tailored solutions.

“Looking beyond the salesforce and performance measurement, elements of this approach can also be used to derive and scale insights across a range of business functions, including servicing, collections and tech support,” says Gupta. “In short, it can be operationalized in any specialized environment with phone-based conversations at its core.”

Contact us

Like what you’ve seen? Get in touch to learn more.