When you run a product, it seems there are dozens of ways you could go. One high-caliber customer requests feature A, several smaller customers say that feature B is really what they need, and the company’s founder has his own idea of what the product needs.
How do you decide what to spend your time on and improve your product? Which features should you be working on?
You need focus on a specific, measurable direction.
I’d like to share with you how we did it for a client I’ve been working with using a “One Metric That Matters” approach.
- Client: big public organization launching an innovative product for job seekers (in France).
- User: unemployed job seeker.
- User need: finding a job! but there are not that many job offers for most of our target population.
- Product solution: recommendations based on company employment data and a prediction engine on which companies are looking for people like you (in France), without considering offers on job boards.
- User success criteria: applying to companies and getting jobs.
- Challenge: people won’t take action if the results we give them are perceived as irrelevant (companies outside their domain, that have been downsizing lately etc…). In that case, they won’t apply to companies and we would have failed them.
- The “One Metric That Matters”: we decided to focus on “result relevance” as our “North Star” for a few weeks. Every product decision would be considered in the light of the following question: “if we do this, will it positively impact result relevance?”. If yes, then we would make it a priority to do it. If not, it would be pushed to the backlog.
Measuring our “One Metric that Matters” and getting user feedback
To understand result relevance, we used Hotjar, a nifty online tool that lets you ask contextual surveys and polls on top of other cool features (session recording, heatmaps…).
You can learn A LOT about your users if you are strategic about getting user feedback. In particular, don’t just rely on a passive feedback button. Don’t rely also on after-the-fact surveys. The best feedback you can get is highly context-dependent. You need to ask the right question at the right time!
In my experience, lots of companies don’t use all the opportunities to get good user feedback. As a result, they rely on long out-of-context surveys which means:
- little feedback: they receive few answers because a user won’t remember what he did with your product if you ask for it out-of-context
- biased feedback: as a user experience with your product isn’t fresh, he will cover what he doesn’t remember with explanations that seem right for him at the moment.
Today you have the tools to get contextual feedback (Hotjar, Qualaroo, Web Engage, and a whole lot more) so use them. Contextual feedback is a good way to understand what drives user behavior in your app.
In our case, the right place to ask for it was the search result page and the right time was a few dozens of seconds after he arrived there (we did experiment with timing).
We also added an optional field inviting users to tell us more about the relevance. The freeform text turned out to be a goldmine of user feedback about how they perceived the result relevance. We learnt each and every objection in the minds of our users as to why our results were not relevant for them. As a consequence, the product roadmap incorporated all the solutions to users objections.
Why having One Metric That Matters?
It forces you, in the midst of new project launch and many competing priorities, to focus on making progress on the One Thing that is more important than all others. It doesn’t mean you should drop every other worthwhile activity you’re doing, but it means you need to have a laser focus on having a specific number you want to see going up. And when you have that laser focus, it’s hard for that number not to go up!
For more about the One Metric That Matters, I recommend you read The Lean Analytics book. There’s a whole chapter about it.
Results of our pig-headed discipline on the One Metric that Matters
By analyzing every objection our users had about result relevance, we were able to drive result relevance from 58% to 78%, a jump of 35% in a few weeks. It involved a better mapping between the user’s occupation and a company’s industry, and more focus into explaining our secret sauce to users. We redesigned the User Experience to communicate about this (more on that in a future post).
Challenging the One Metric that Matters and keeping the end in mind
As useful as the OMTM is, it’s essential to stay grounded in user-centered focus. At the end of the day, “result relevance” means nothing if users don’t take action and apply to companies.
But it’s true of any metric focus. For example, it’s very easy to focus on landing page conversion at the expense of full product conversion (from visitor to paid customer). You need to make sure your tactical “One Metric that Matters” focus doesn’t negatively impact your end-to-end metrics.
The end game for this project is doing everything to get job seekers to apply to companies and get jobs. So the next stage of growth involves improving another metric, now that “result relevance” has been significantly improved.
Using the One Metric that Matters in your project
What the “One Metric that Matters” should be specifically depends on what stage your product is, and so what the priority should be.
- If you’ve just built the product, focus on rough engagement metrics (“can our customers use the product?”)
- If your product has value but most users never get round to it, focus on activation metrics (“can our customers have a successful first experience with our product?”)
[I’m digressing, but most companies looking for “growth hacking” are actually mistaking what metric they should focus on.]
What’s the goal for your product right now?
Define a single “One Metric that Matters” that moves the needle for your most pressing goal and focus on it for a few weeks.
One of the worst things for startups and small teams is lack of focus. By using this tactic, you’ll sure have a lot of focus on one thing. Try it.