Here is the uncomfortable version up front: your KPIs probably do not measure exactly what you think they measure.

That is not a reason to give up on measurement. It is the reason to take measurement more seriously.

Done well, DMAIC is not a ritual for polishing dashboards. It is a disciplined way to examine the gap between what a metric captures and what the system is actually doing.

The uncomfortable truth about measurement

Every KPI is a proxy. It is a partial view of a system that is always richer, messier, and more conditional than the metric suggests.

If you do not have a model of your measurement error sources — one that makes sense for the domain you are operating in — then your KPI is not ground truth. It is an estimate built on assumptions.

A velocity metric in software is not productivity. A conversion rate is not customer value. A yield number is not system health. Each one may still be useful, but only if you stay honest about what it leaves out.

DMAIC is a learning framework before it is an optimization framework

DMAIC — Define, Measure, Analyze, Improve, Control — is often taught like a process-improvement checklist. That framing is incomplete.

A more useful framing is this: DMAIC is a learning framework disguised as an improvement framework.

The Define phase is not only about selecting metrics. It is about asking whether those metrics deserve the interpretation you want to give them. It is where you surface assumptions, missing variables, blind spots, and mismatches between the proxy and the real objective.

If you rush that step, you can still improve a number. You just lose confidence that the number means what you think it means.

Where most teams should spend more time

Early in a project, a large share of the work should go toward understanding what your metrics miss.

Not just refining the KPI. Not just adding more dashboards. Understanding the parts of reality the metric does not capture.

Useful Define-phase questions include:

• What would ideal measurement look like if perfect information were available?
• What is the gap between that ideal and what we can actually observe?
• What assumptions are we making about what this metric represents?
• What error sources does our measurement model ignore?
• What would have to be true for this KPI to mean what we say it means?

If those questions are hard to answer, that is usually a sign the system needs more modeling before it needs more optimization.

The gap is often the signal

The strongest measurement cultures tend to treat KPIs as hypotheses, not as verdicts.

They ask:

• if this metric moves, what do we think is happening?
• what else could explain the same movement?
• what would falsify our interpretation?

They compare leading and lagging indicators. They pay attention when those indicators diverge. They study the shape of the gap between measurement and reality because that gap often reveals how the system actually behaves.

Example: software delivery

Take deployment frequency. It is a useful software metric, and in the right context it can tell you something important.

But what does it actually measure? It measures how often code ships. It does not directly measure:

• whether the shipped changes solve real user problems
• whether the team is burning out to maintain that pace
• whether deployments are safe or risky
• whether throughput is creating value or just volume

The Define-phase work is not simply "pick a better metric." It is understanding what deployment frequency misses, how it interacts with other signals, and how easily people can game it when incentives narrow around the number.

Example: manufacturing yield

Now take yield. Nearly every manufacturing organization tracks it, and for good reason. But it is still only a partial view.

At the line level, yield is the ratio of good units to total units. Straightforward on paper. Less straightforward in practice.

Yield can hide:

• defects that escape detection and only show up at the customer
• rework that consumes time and material without showing up as scrap
• downstream costs created by one department optimizing its own metric at the expense of the overall system

That is how local KPI success can still produce global system failure. One team improves its number. Another inherits the hidden cost.

More useful questions become:

• Where did the defect actually originate?
• What upstream choices are pushing cost downstream?
• What is the total cost of quality, including rework, warranty exposure, and customer trust?
• What would a system-level yield measure require that we do not currently capture?

What the Define phase is really for

George Box gave us the line that all models are wrong, but some are useful. In practice, the useful ones are usually the ones we have interrogated hardest.

The Define phase is where that interrogation belongs. It is where teams get honest about:

• what the metric captures
• what it misses
• which assumptions are doing hidden work
• which error sources remain unmodeled
• how local optimization may be distorting the wider system

That is a more durable use of DMAIC than treating Define as a quick setup step before the "real" improvement work begins.

The point

The goal is not to find a perfect KPI. The goal is to build a more honest relationship with imperfect measurement.

If you spend the Define phase selecting metrics without interrogating them, you can still produce clean-looking dashboards. What you may not produce is understanding.

The best measurement frameworks do not remove uncertainty. They make it more visible and more manageable. DMAIC done well is a practice in epistemic humility, not a promise of perfect metrics.

Next in this series: when departments optimize for different metrics, the result is often not a better system but coordinated dysfunction.