Why not all metrics are equally useful

The data often looks convincing: attractive graphs, positive trends, well-designed reports. But does this really mean that things are going well?

Today we will talk about the concept that distinguishes between metrics that have practical value (first-class data) and those that are convenient to show but difficult to interpret. They look like measurable progress, but they do not provide real answers. In other words, secondary metrics are like plastic fruit on a table: they look impressive, but you can’t eat them.

This article is about why most familiar metrics may not actually have any analytical value. And about how to replace superficial reporting with effective analytics.

Vanity Metrics vs Actionable Metrics

Vanity metrics are superficial indicators that are often used in reports and dashboards but rarely lead to concrete actions or changes. They can be informative, but on their own, they have no practical value for business.

Actionable metrics are indicators that directly influence decision-making, product changes, marketing, or process optimization. They:

Common mistakes when working with data

1. Using metrics only for reporting
Many organizations create regular reports that have no analytical value. For example, a weekly report with total traffic, without a breakdown by channel and without correlation with conversions.

2. KPIs that are not tied to strategy
Sometimes companies measure what is easy to track rather than what has strategic value. For example, the marketing department may focus on increasing video views, even though the strategic goal is to generate leads.

3. Data collection without further use
Technically, an analytical system can collect dozens of events and attributes, but none of them will be used for decision-making. This creates analytical noise and makes it difficult to focus on truly important indicators.

4. Look at the data separately
What is measured — clicks, transactions, visits — are not isolated events; they interact in a shared environment. And this is the key to understanding why vanity metrics are often misleading.

It makes no sense to look at bounce rate separately from traffic quality. It makes no sense to calculate open rate without considering clicks and sales. Data must be analyzed as a single chain—from the first contact to the final result.

Beautiful metrics can be misleading

Let’s consider a situation that HubSpot encountered at one point. For a long time, the company focused on the total number of MQLs (Marketing Qualified Leads) as the main metric for marketing success.

The growth in MQLs looked like progress: more leads, more traffic, more activity. But it later became clear that demo requests had three times higher conversion rates than, for example, e-book downloads. In other words, growth in MQLs did not translate into growth in sales.

The team optimized marketing campaigns for vanity metrics and lost real analytical value.

The situation is similar with mobile applications. The Sovereignty of the Apps study analyzed over 213,000 applications. The result: 70% of users drop out within the first week after installing the application. Even in the top 100, the retention rate is around 45%. The number of installations looks impressive in reports, but this is only the beginning: if there are no retention rates, these are secondary metrics.

Blind spots in analytics

Equally dangerous are metrics that are not tracked at all because the team is unaware of their importance. Examples:

Time-to-first-value (TTFV) is the time between a user’s first interaction with a product and the moment when they experience real value. This metric is critical for SaaS, but is often overlooked.

Customer Effort Score (CES) – an indicator of how easy it is for a customer to perform a key action. Poor integration, a complex UI, or bugs increase effort, leading to customer loss.

Segmented Retention – overall retention may appear normal, but if you don’t look at the segments (paid traffic, organic, referrals), important patterns will remain invisible. This is a classic case of Simpson’s paradox: in general, the data indicates one trend, but after dividing it into segments, the opposite becomes apparent.

Context is everything: when vanity ceases to be vanity

It should be added that there are no bad or good metrics per se. The key question is: who uses them, at what stage, and for what purpose.

For SMM specialists, engagement is an actionable metric that shows content effectiveness. CTR in advertising is not a goal, but it is useful in A/B testing to determine what works best.

What gets measured gets improved. KPIs are not just numbers, but a way of thinking and a focus for action.

How to work with metrics effectively

1. Start with the goal. What exactly needs to be changed/improved? This determines the choice of metrics.
2. Focus on indicators that are responsible for behavior, not just activity.
3. A number without context is worthless. Why did it increase/decrease? What happened in the external environment?
4. Regularly review the relevance of metrics. What was important at the start may be a hindrance at the scaling stage.

Yes, not all data is equally important, and not every metric is truly useful. When looking at numbers, it is important to ask the right questions and understand the answers. This is where the power of quality business analytics lies: seeing where the true value of data lies and turning it into effective action.

This is exactly what we do at IWIS: we help companies structure analytics so that it works for the business, rather than hiding weaknesses behind a beautiful facade.