The measurement of institutional reform progress is the mechanism through which reform either deepens or degrades. Measuring the wrong things produces the wrong reform.
The Measurement Problem
Measuring institutional reform progress is harder than measuring institutional outputs for the same reason that measuring institutional quality is harder than measuring institutional activity: what matters most is typically less measurable than what is most easily quantified. The reform that is supposed to produce better outcomes for the institution's population is more straightforwardly measured in terms of institutional activities — the number of procedures changed, the training hours delivered, the policies revised — than in terms of the population outcomes that those activities are supposed to produce. The measurement system that tracks activities rather than outcomes gives the institution the wrong incentives: to perform the activities that the measurement system rewards rather than to produce the outcomes that the reform was designed to achieve.
The outcome measurement that would give the institution the right incentives is harder to design, takes longer to produce data, and is more vulnerable to the confounding factors that complicate the attribution of outcomes to institutional actions. These difficulties are real but not insurmountable. The measurement investment that produces credible outcome data is the measurement investment that makes reform accountability possible — and without accountability to outcomes, the reform that is measured in activities will produce the activities it is measured by rather than the outcomes it was designed to produce.
Measuring reform progress by activities rather than outcomes is the accountability architecture of the institution that wants to demonstrate compliance rather than achieve change. The measurement system that tracks what the institution does rather than what happens to the people it serves will produce exactly what it measures: institutions that do things differently without necessarily producing better outcomes for the people whose lives the things are supposed to affect.
Discussion