Telecom Giant Solves Checkout Promo Code Error
A no-contract mobile phone provider in the US, with 21 million subscribers and a network of more than 90,000 retail locations. Their analytics engineering team is responsible for ensuring the promotion redemption process runs smoothly by catching issues and escalating them for resolution.
Decreased the error rate by 6x within a week of its detection, saving over $500K in annual sales from potential abandoned purchases. But with survey data alone, they lacked the contextual information they needed to validate, escalate and prioritize the issue and then locate, reproduce and resolve it.
Their analytics team knew they needed to act fast when customers started reporting, via a Qualtrics survey, that they were unable to use promo codes they had received.
This was during peak season and each day the issue went unresolved it impacted NPS scores, decreased customer satisfaction and jeopardized potential sales. But with survey data alone, they lacked the contextual information they needed to validate, escalate and prioritize the issue and then locate, reproduce and resolve it.
Was this a user error (i.e. they were simply putting in the wrong code)? Or a technical error (i.e the promo code system was generating invalid codes or there was a faulty API validation)? This lack of information risked increasing the time spent diagnosing, reproducing and resolving the issue.
It wasn’t clear how many promo codes and product checkouts were affected, how many customers were experiencing the same struggle or its effect on revenue. This hindered their ability to route the case to the right team and provide them objective reasons to prioritize it.
There was no simple way to track user flow and measure the effectiveness of any fix they would implement. Without the ability to monitor the fix, their VoC analysts couldn’t be confident they had solved the root cause of this issue.
The analytics team leveraged the deep link integration between Glassbox and Qualtrics to answer three key questions quickly and accurately: why were users struggling, what was the impact and how effective was the resolution?
Understand the struggle
Using the deep link, they easily located and reproduced the error by replaying the sessions with customer feedback. By searching the replay activity tree for text in the customer message, such as “expired,” they were able to jump directly to the struggle. After watching a few replays, the VoC team reproduced the issue. Customers were receiving promotional codes from a specific campaign that were already expired. By sharing this critical information with the appropriate team the time to resolution was dramatically reduced.
Prioritize by impact
They then created an ad hoc funnel showing the journeys featuring this error. Knowing the average order amount, how many customers were affected and the increase in error rate allowed them to measure the true impact—over $10,000 in lost sales in one week alone. Armed with these measurements, they were able to assign a high priority to the case.
Monitoring the user flow showed an increase in customers successfully completing this step. Furthermore, to ensure they had visibility into a potential recurrence of this issue they implemented a threshold alert The alert triggers when the error rate goes above their 3% baseline, giving them the confidence that they will know in real-time if the issue resurfaced.
They decreased the error rate by 6x within a week of its detection, saving over $500K in annual sales from potential abandoned purchases. In the process they also saved engineering time, preserved NPS scores and ensured continued customer satisfaction.
And now with automated alerts, they're catching such issues before customers report them. Meaning their VoC team can switch from putting out fires to proactively improving the user experience.
reduction in error rate, saving over $500K in annual sales from potential abandoned purchases.