A dashboard can look healthy while the outcome it represents gets worse. | Image credit: Tanía Diaz / Ordinal
Main takeaways
- A better KPI does not always mean better performance.
- What you measure can change how people work.
- Targets can encourage teams to optimise the number.
- Context helps explain what sits behind the metric.
- Good measurement should support decisions, not replace them.
A customer service team is measured on how quickly it closes tickets. Within a few months, average resolution time falls. The dashboard looks better. But customers start coming back with the same problems because cases are being closed before they are fully resolved.
The team has improved the number it was asked to improve. Whether customer service has improved is a different question.
This is one of the difficulties with Key Performance Indicators (KPIs). A KPI usually reduces a complex area of performance to a single measure, such as customer satisfaction, productivity, quality, sales performance or operational efficiency. The measure makes that outcome easier to see. It can also begin to influence the work it was designed to measure.
When the Measure Becomes the Target
This problem is often described through Goodhart’s Law. Economist Charles Goodhart originally made the observation in relation to monetary policy. Anthropologist Marilyn Strathern later captured its broader application to performance measurement: “When a measure becomes a target, it ceases to be a good measure.”
That does not mean every target eventually becomes useless. It describes what can happen when people have a strong reason to improve a proxy rather than the outcome it was intended to represent.
A sales team measured heavily on the number of leads generated can produce more leads without generating more valuable opportunities. An operations team pushed towards maximum utilisation can keep everyone busy while leaving little capacity to respond when demand changes.
Neither requires anyone to manipulate the system deliberately. People respond to what the organisation signals as important.
Research into performance management systems has documented the consequences. A review by Monica Franco-Santos and David Otley found that directive performance systems can produce gaming, information manipulation, selective attention and an illusion of control. Their analysis also shows that the way a performance system is designed influences the unintended consequences it can create.
The KPI is therefore doing more than recording performance. It can change the conditions under which people make decisions about their work.
Better Numbers Can Hide a Different Problem
The risk becomes more difficult to see when a small number of measures are used to represent a complex operation.
Research into target-setting in the English NHS provides a useful example. Gwyn Bevan and Christopher Hood examined a system in which healthcare organisations were assessed against centrally determined targets. They found evidence of reported performance improvements alongside problems with measurement and gaming.
A target can therefore produce improvement in the measured result while creating behaviour elsewhere that the number does not show.
Experimental research into organisational league tables found another effect. When managers were given information about changes in their ranking, they became more willing to take risks, particularly when doing so offered the possibility of moving their unit to the top of the table. The researchers described this as an unintended dysfunctional consequence of the performance measurement system.
For a CEO or COO, this changes what a green dashboard can tell you. If ticket resolution time has improved, what happened to repeat contacts? If utilisation is at 95%, how quickly can the operation respond to an unexpected order? If sales activity is up, did conversion or revenue move with it?
Adding another KPI for every possible consequence is unlikely to solve the problem. Eventually the dashboard becomes another source of complexity. The more useful question is whether the measure still tells you what you think it does.
Look at the Behaviour Behind the Number
KPIs are useful because leaders cannot observe every part of an organisation directly. They need signals that show where performance is changing and where attention may be required. But a signal is not the outcome itself.
When a KPI begins to influence incentives, resources or performance reviews, it is worth looking at how people respond to it. What behaviour does the measure reward? What work receives less attention because it is harder to count? Can someone improve the number without improving the result the business cares about?
These questions become increasingly relevant as organisations automate reporting and performance monitoring. A dashboard can update continuously and still measure the wrong thing.
A useful KPI helps someone understand what is happening well enough to make a better decision. When improving the number starts to take priority over improving the work, the measure may no longer be doing the job it was designed to do.
References
- Strathern, M. (1997). ‘Improving ratings’: audit in the British University system. European Review, 5(3), 305–321. https://doi.org/10.1002/(SICI)1234-981X(199707)5:3%3C305::AID-EURO184%3E3.0.CO;2-4
- Franco-Santos, M., & Otley, D. (2018). Reviewing and theorizing the unintended consequences of performance management systems. International Journal of Management Reviews, 20(3), 696–730. https://doi.org/10.1111/ijmr.12183
- Bevan, G., & Hood, C. (2006). What’s measured is what matters: targets and gaming in the English public health care system. Public Administration, 84(3), 517–538. https://doi.org/10.1111/j.1467-9299.2006.00600.x
- Keasey, K., Moon, P., & Duxbury, D. (2000). Performance measurement and the use of league tables: some experimental evidence of dysfunctional consequences. Accounting and Business Research, 30(4), 275–286. https://doi.org/10.1080/00014788.2000.9728945