Defining and Tracking Success: Crafting Meaningful Metrics
Continuing from our deep dive into five years of Product Management experience, this article shifts focus to a crucial aspect of both professional and personal growth: crafting meaningful metrics. Just as I’ve previously explored the intricacies of managing to-do lists, here we’ll navigate definitions and measurements of what truly counts.
We’ll cover the metrics and measurements topic in a way that makes sense regardless of the project — personal or work, physical or digital. Based on my experience, there’s no difference in the method and thinking process to get you there.
Yes, but why metrics?
Ignoring metrics is like living in a Dreamland, where everything is possible at any time. No reasoning exists or is needed. Everything that stands out is a happy accident of chance.
Where to start defining the metrics?
I find it very helpful to start with two questions so that you don’t overcomplicate things.
1. “Why people/I bother doing this thing?”
If you’re talking about a digital service you can ask “Why do people bother signing up?”; Consider you’re still in Dreamland and answer freely but relate to the service, aka. “Because the service is A-WE-SO-ME!” is not a correct answer!
If this is a personal goal, e.g. fasting then the question is “Why am I fasting?”
In both cases, we’re talking about something fundamental and “easy” to answer, right? Well… actually… if you try to stay honest and keep digging these answers are not easy.
Are you fasting to become leaner, be more likable, or feel good about yourself?
Are people signing up to get the value you’re offering or just for quick free grabs of a trial?
The simplicity of the question doesn’t guarantee a simple answer. But don’t be discouraged! You can stay top-level or dig deeper as much as you feel comfortable! This question works either way 👍!
Armed with your answer, proceed to question number 2
2. “What people/I need to do to [insert answer from question 1]?”
Answering this question will give you the sequence of intermediate steps to achieve what you answered in the first question. Now you’re sprinting away from Dreamland!
If the answer in the example of digital service signing up was “to access feature X”, then the second question becomes “What do people need to do to access feature X?”. I’m sure you’re connecting the dots already in your mind with a real app example. I’m almost certain that in your mind you picture clicks on specific buttons and views of specific app pages; these are your intermediate steps.
In the personal case example of fasting and the safe answer “to get leaner”, the question becomes “What do I need to do to get leaner?”. Besides the obvious answer of “I need to fast” — which is circular — you might answer “Stick to the schedule of fasting” and “Check your weight”; again these are your intermediate steps.
Don’t underestimate the question, and don’t overcomplicate your answers!! There’s a gazillion stuff you can imagine as intermediate steps but keep it simple!
Setting up the metrics
Now that you have the intermediate steps, it’s time to “translate” them into metrics. Don’t rush into making a one-to-one match of a step to a metric. Also, widen your approach and way of thinking on choosing the right metric; will share solid examples a bit later.
The characteristics of a good metric are:
- Measurable, well duh… yet still… Measurable means that you should avoid adding too much stuff in the mix, e.g. “awesomeness factor of 4” which is a combination of X, Y, and Z.
- Impactful, you can connect any measurement with the relevant step
- Actionable, where you can clearly see the measurement of the metric and understand what needs to be done to improve it (or pat your back on a good measurement)!
- Readable, which means that other people can easily understand what it represents and what that metric’s part is in the bigger picture.
- Relevant, meaning both that a) it is closely related to what you measure and b) it will not be dropped or skipped at some point.
In the example of the digital app and accessing feature X, measuring the click of a button in one of the steps is a good metric. Thinking wider, you can also add a metric of support tickets you get on people looking for feature X; the more support tickets you get it means that people don’t find it on their own in the app!
In the personal goal of fasting and sticking to the schedule, measuring the days of fasting is a good metric. Thinking wider, you can also measure the days you couldn’t focus due to fasting; if the days you were out of focus are many, then you probably need to adjust something in your diet as it is affecting many more aspects of your life than your stomach and your goal of becoming leaner!
What’s NOT a good metric
This is the most important part of this post! We are usually eager to jump into action that we haste some parts and decisions! Measuring the right stuff is important, so please keep on reading!
A bad metric can lead you to bad decisions AND bad results! A bad metric can make a Dreamland look like the real world, still, you ignore vital stuff and you might end up worse than you started!
I promise that building a habit of defining a good metric sticks with you but it needs practice at first and that takes time, aka. DON’T SKIP THIS SECTION!
So a bad metric is:
- Partial, covering a small part of the story behind the measurement, where measurement going UP means one thing but going DOWN might mean nothing. In the example of the digital app and feature X, a bad metric could be the time to click a button when on a page; What do short or long times mean? Is it safe to interpret it exclusively?
- Vanity-based, measuring something unrelated to the bigger picture and the overall goal. A vanity metric is usually self-contained but it doesn’t directly affect another aspect of the goal or target set, e.g. measuring the social media presence of your personal or service brand without binding that presence with measurable reach-outs or visits to your website; where people can follow through the overarching journey.
- Telling the wrong story, where the interpretation of the story is simply off. While the measurement itself can work wonders, you tend to “translate” that measurement in a very wrong way. Two examples in mind come from aviation, one is on defining the durability of returning aircraft from a WW2 front (Survivorship bias), and the other is on designing the pilot’s seat based on the average.
- Focusing on validation, trying to prove a preset point of view as “the truth”. This metric is a combination of all of the above. Such a metric is very dangerous to conclude from and it falls heavily into the Dreamland world! A good question to check against such a metric is “What value/measurement of this metric will prove that we were wrong?”. Focusing on the invalidation of an assumption or “your truth” is way more productive and action-oriented than sticking with a preset truth.
If you’ve reached this far into the article, I’m happy 😊! You just re-read the most important DOs and DON’Ts on setting up metrics! In the following two sections we’ll cover the pace of measurements — how often should you measure — and the end-game of measurements — setting goals and KPIs on your metrics.
How often should you measure?
The TL;DR is: “As often as it makes sane sense!”.
Going too granular is a waste of your time, especially for metrics that don’t change that often (e.g. there’s no benefit in measuring your weight daily!). On the other hand, going too sparse you might lose some important middle fluctuations that could help you take action (e.g. measuring your weight once a year is not that helpful to edit your metabolic habits: eat and exercise).
Based on my own experience — and regardless of the metric at hand — measuring weekly is ideal yet pretty hard to follow, while monthly is a more realistic goal.
What to do next with these metrics? Setting performance goals
So far we’ve seen how to build the right set of metrics and how often to measure them. The next step — or end-game — is to turn these metrics and measurements into a plan!
After monitoring your measurements for a while, you will have an idea of the rate these measurements change. In the meantime, you will have some questions raised and surely some solid answers by simply checking these measurements. In the fasting example, measuring how your weight changes, you will be able to quantify your effort to actual weight loss and your feelings during these periods.
Having a better grasp of your metrics and measurements, you can build some goals to work towards in the future. These goals in the business world are what we call KPIs or Key Performance Indicators. Put simply, these are goals or targets for these metrics in time. E.g. “Fasting will get me leaner and thinner by 10 pounds until April”, this is a goal you set based on your actions and measurements in place.
A key advice here is to set a goal AFTER you’ve monitored your metrics for a while! Kicking things off with a goal is arbitrary and most likely Dreamland territory! In some cases setting an arbitrary goal is unavoidable, so please keep in mind that this arbitrary goal is a best guess at that time which you should adjust with more measurements coming in.
Wrapping it all up 🎁
In this post, we covered a very high-level, simplistic approach to metrics, measurements, and goals (KPIs). In this journey, we started from the end goal of a project (fasting, or signing up for a digital service) and explored the What, How, and finally Why for metrics.
As we explored all these aspects, I tried to stay consistent and tell a single story, the story that can lead you to your project goals.
Closing this post with a bit of advice: stay close to your project’s story when creating your metrics, measurement cadence, and goals. If you build your metrics and measurements as a part of your story it will be much easier for you — and everyone involved — to resonate and empathize with these metrics, they will be a part of your project’s story too! Here’s a reminder of the power of storytelling.
One more thing, when you create your metrics avoid half-metrics, focus on vanity, or tell the wrong story!
This post is intentionally simple, but it is created taking into account more sophisticated and complex work on metrics and KPIs. Will be covering the more complex parts of funnels, creating dashboards, deep-dive on specific metrics, and metrics to measure the impact of product/behavioral changes in a future post.
In the meantime, I would appreciate any questions or feedback 🙏
