Agile product management needs better data. Here’s how to get it

We’re living in the golden age of Agile. Seven out of ten U.S. companies have adopted Agile in some form—an increase of nearly 90% since 2002—and there are over 70 product management solutions on the market to help manage Agile workflows. But with all the tools, frameworks and general support for Agile, product teams still consistently fail to meet customer expectations. In fact, less than half of product professionals (47%) are confident their product roadmap reflects the needs of users.

This shouldn’t be happening—not when Agile is more popular and well resourced than ever. So what gives? Agile may be a superior methodology to old-school waterfall, but if the data behind it is disjointed or insufficient, it can just as easily run you into a brick wall as it can your next stage of growth. And that’s precisely what’s happening on too many product teams—product data is examined in isolation of other critical elements of a holistic customer experience, including technical performance.

It also doesn’t help that product managers often have to redirect their focus from customers and are instead forced to navigate a complex web of stakeholders, internal politics and conflicting priorities—even when their organization insists that they’re “data-driven.” No wonder so many products and features miss the mark.

Innovative product teams are leveraging a new approach to data and analytics to overcome some of the biggest hurdles to Agile product management—and consistently delight customers. In this post, we’ll take a look at what they’re doing differently, and how you can replicate their approach to deliver value faster.

Why Agile isn’t enough on its own

There has never been more support for Agile teams, but product managers are still frequently hindered by people, practices and processes alike. The reason almost always boils down to lackluster data—disparate, disorganized, misaligned, obsolete, inaccurate, inaccessible or otherwise unusable. This is especially true at larger companies—according to IBM, over 80% of data coming through an enterprise organization isn’t actionable, due to being siloed or unstructured.

It gets worse. Organizations have a dwindling margin of error for guessing wrong and shipping features that don’t have real value. Especially since:

• Customer acquisition costs increased 222% from 2013 to 2022, putting a greater burden on retention efforts

• The instant gratification era has already supercharged demands from impatient customers—now throw in an economic downturn slashing their budgets and increasing pressure to extract as much value from vendors as possible

• Low-code and no-code technology has significantly weakened the barrier to entry for competitors, affording customers a wealth of alternatives to choose from the second a product fails to deliver.

And that’s the thing. Even an impeccable methodology like Agile will fail if there’s insufficient data behind it—which ultimately wastes time, drains resources and stalls time to value.

How lousy data impacts Agile product management

Insufficient data has a direct, tangible impact on Agile product management. Some of the biggest problems it can cause include

1. Insufficient customer feedback

Accessing meaningful customer feedback is often an uphill battle. Less than half (42%) of product teams have a system for capturing customer feedback, while 27% say their customer feedback collection process is non-existent. Even teams who do gather customer feedback are limited, as their primary methods—interviews and surveys—often fail to capture real, actionable and reliable insights into users’ experiences.

Customer interviews

Interviews aren’t just time consuming and difficult to scale, but also unreliable predictors of future behavior. Humans are notoriously bad at predicting the future—especially about themselves. However, it’s not just good ol’ fashioned psychology working against you. Customers also don’t have a nuanced enough understanding of your product to anticipate the “technical cost” implementing particular features—and how that may influence their behavior as well.

Customers will happily sound off their ideas and suggestions, but that doesn’t mean they’ll be equally content to stick around if new features cause even the slightest bit of friction—like a millisecond delay in loading time.

Surveys & satisfaction scores

Thirty percent of Agile practitioners rely on customer surveys to measure against business metrics and 25% use Net Promoter Scores (NPS). However, while surveys may be easier and faster to deploy than interviews, they’re equally susceptible to limited or inaccurate information—especially since feedback typically represents a small minority. Voice of the customer (VoC) data is sourced from just 4-7% of users on average (our own research has found this closer to 4%).

And that’s not all. If customers don’t find value in new features, product teams may be left scrambling to figure out why. Most users won’t grant you a tell-all interview or even a simple survey response before they churn—91% of unhappy customers will leave without complaining.

2. Roadmapping guided by stakeholder requests instead of value to customers

Agile software engineers operate with a “definition of done”—concrete requirements for a feature to be considered complete. Most product teams, however, lack a similarly objective process for quantifying high-value and low-value features based on predictable business impact—specifically, patterns in user behavior associated with conversion and retention. Instead, PMs often find themselves attempting to juggle stakeholder and customer requests:

• 41% of product managers cite prioritization as their top challenge

• Roughly one in five Agile practitioners still aren’t sure what’s being used to measure business value

• More Agile teams are measured by on-time delivery (47%) than business objectives achieved (44%)

Without business impact to ground roadmaps to big picture goals, product teams are more likely to get derailed by conflicting priorities and preferences, bottlenecks and stakeholder interference.

3. An obstructed view into the complete customer experience

Product, DevOps, marketing and CX teams all have a significant impact on how customers experience a product. But typically, each is working with their own data in separate analytics tools. While you’re entrenched in product data, DevOps is combing through error reports and marketing is off to the side with some digital analytics platform you’ve probably never laid eyes on.

In some ways, it’s perfectly understandable that different teams performing different functions would use their own purpose-built tools to collect different types of data. The issue is that product analytics, web analytics, technical performance and VoC data are all deeply entwined and relevant to one another—especially when it comes to assessing and refining customers’ overall experience.

Sure, any decent product analytics tool will help uncover trends in adoption, feature usage and in-app behavior over time. But the technical events enabling specific functionality are often locked up in an application performance monitoring (APM) solution that’s almost impossible to decipher without a computer science degree. Product teams also may lack sufficient access to web analytics or VoC data, making it difficult to fully measure, analyze and interpret different behavior, and guide new features accordingly.

Product, DevOps, Marketing and CX each know the what and how within their respective domain, but are missing all the puzzle pieces to fully understand the why.

How to leverage better data for Agile product management

Agile has come far from its early days as a short list of software engineering principles compiled on a weekend ski trip over 20 years ago. But it can’t do what it was supposed to do—consistently improve your product—using product data alone. At least not in today’s market, with today’s customer demands. Here’s what to do instead:

1. Replace data silos with an integrated, 360-degree view of the entire digital experience

Innovative product teams have addressed these challenges with a more integrative approach to analytics, leveraging better data to contextualize product engagement within a broader customer experience. Considering seemingly separate and external factors affords more depth and nuance—like how air turbulence causing a drink to spill and stain a flawless carry-on bag will affect your air travel experience (even if it’s not the airline’s fault).

Digital experience intelligence—which consolidates vast combinations of data sets spanning product analytics, web analytics, technical performance and VoC data, among others—enables product teams to quickly and accurately observe the impact of technical events and sources of friction on different outcomes, including conversion, adoption, abandonment, churn and renewal.

2. Reverse-engineer conversions, renewals and churns to identify associated user actions—and quickly act on real-time predictions

Product analytics tools already record user actions like clicks, taps, swipes and views. Digital experience intelligence pushes these insights further by automatically analyzing and determining patterns, trends or sequences within those actions, as well as cross-referencing them with technical events and sources of friction. This makes it easier to identify the cause of different types of behavior—as well as make more accurate predictions for the future.

For example, instead of flagging at-risk users by inactivity, digital experience intelligence can essentially reverse-engineer churn among different user segments to identify common actions across multiple channels preceding the cancellation. Let’s say 24% of active users with a “Pro” business account ignored three consecutive emails containing helpful product tips and downgraded to a free account at least two weeks before they canceled. Active users with a “Pro” account who have ignored two consecutive emails and visited their account settings page—indicating they’re considering a downgrade—are tracking to churn according to your own benchmarks.

Beyond improving predictions, you can also act on this data a lot sooner. Assessing which features had the biggest drop-off in usage among churned customers before the first two emails were ignored, and then checking technical performance data to see if any errors or glitches occurred during that time, gives DevOps the ability to quickly isolate and correct the problem. Marketing, meanwhile, can launch a re-engagement campaign before churn-risk “Pro” users actually go ahead and downgrade their account, taking another step closer to cancellation.

3. Give voice to the silent majority

Most users will never let you know when they’re frustrated—but that doesn’t mean they aren’t providing feedback. It’s just that they’re doing it with their actions instead of words.

Let’s say 800 users dropped out of the same upsell funnel, but only 25 complained about the checkout page not loading properly. With traditional VoC data focusing on direct feedback, the majority of affected users would be untraceable.

Digital experience intelligence, however, can form a precise timeline of user actions across the 25 sessions and automatically compare them with all other sessions that occurred over the same period. This is how you can uncover the remaining 775 users who encountered dead clicks and page reloads on the checkout page, indicating they were similarly affected.

You can even take it a step further by integrating CRM data to measure infinite variables against revenue won or lost. If 800 users encountered the same error on the checkout page, digital experience intelligence can calculate the average membership upgrade—an extra $15 per month—to determine that this single error cost $12,000 per month in revenue. This enables teams to identify the most important, highest value paths to optimize or improve, as well as the biggest issues to solve.

Supplementing interviews, surveys and scores with digital experience intelligence also helps bridge the divide between customer requests and real-time interactions with your product—especially since wants, needs, preferences and behaviors often evolve over time.

Back to you

Product teams are facing increasing pressure to deliver as much value in as little time as possible. If the benefits aren’t immediately clear, customers will bolt before you’ve even had a chance to address the problem. To deliver, it’s not enough to have an Agile process in place—you also have to drive it with integrated and actionable data. That’s precisely what future-thinking product teams are now doing to understand their customers sooner—and delight them faster.

There’s more where that came from

We wrote an entire playbook for product managers on leveraging better data to consistently delight customers—in one of the most cutthroat markets in years. Grab your free copy here.