The challenge of collecting, measuring, and understanding indicators

There are many potential pitfalls in moving from a situation where there has been very little province-wide information available – and virtually no data comparing residential service providers with each other – to one where key indicators are used to “evaluate the effectiveness of programs and services across service providers and sectors”. These include (1) finding an appropriate measure or indicator, (2) ensuring that the information collected is accurate, (3) minimizing the response burden and data collection costs, (4) interpreting the results in their appropriate context, and (5) ensuring that services are not inappropriately incentivised to maximize good rankings on indicators at the cost of other important unmeasured dimensions. The challenge of dealing with these issues is particularly complex in a sector where the use of psychometrically validated measures varies significantly, where there has been very limited public reporting of data, and where there isn’t enough information available to establish contextualized baselines for setting performance targets.