A Fortune 50 Telecommunications provider needed help in measuring how a major implementation would improve its operations. The communications provider was implementing a new Interactive Voice Response (IVR) system with the goal of enhancing customer service in its call centers through improving routing accuracy. The client had released the new system to a subset of customers and required an analytics solution to investigate how the new system was performing compared to its predecessor.

Successfully executing a transition as large as the implementation of a new IVR presents several major challenges. Without a well thought out and thoroughly vetted set of requirements, the new solution may fail to satisfy all stakeholders, which would defeat the reason for the change. Without clear communication, business rules could be translated incorrectly, negating the value of the requirements gathering effort. Finally, without accurate Information Insight, key performance indicators would be analyzed incorrectly, resulting in incorrect decision-making and adjustments to the IVR.

Recognizing the need to avoid these pitfalls, Kenway worked to carefully plan how to best monitor and evaluate the trial of the new solution early in the project lifecycle. Through a combined effort and bi-directional communication between the client and Kenway, key performance indicators and metrics were defined as requirements were being documented.

Through iterative development, an application was delivered that clearly visualized the deltas in the performance of the two systems. The application also allowed the client to drill down into certain metrics to understand why one system was performing better than the other. For example, views into three key performance indicators that were previously difficult to measure were readily accessible via the new application:

1. Repeat Caller Rate – This metric reports the number of times a customer makes a call to the client and then calls back within a predefined window of time.
2. Account Collection Metrics – This indicator displays the collection rate for both the old solution and the new one, enabling a simple side-by-side comparison.
3. Call Duration – A metric that measures the amount of time a caller spends in each segment of the IVR during a call. This number is a useful statistic for analyzing customer experience.

The final application Kenway created analyzes vast amounts of data with numerous intersecting touch points. Because Kenway and the client were in lockstep on requirements, the end product delivered was directly in line with the client’s needs. The final result was a continued, cyclical approach to update the logic behind the IVR, evaluate the impact of those changes in the application using data driven analysis, and continue to improve the application accordingly.