One of the tasks included in the Panel’s terms of reference was the identification of “key indicators that would make it possible to evaluate the effectiveness of programs and services across service providers and sectors”. Many of our recommendations hinge on the Ministry’s ability to gather such information. The importance of accessing and analysing the right indicators is not only a concern for the Ministry, but has been raised in all previous reviews. The framework and indicators presented in this chapter were developed on the basis of a review of indicators that are currently reported by the Ministry or by some residential service providers or are being reported in selected jurisdictions across Canada and internationally.

The challenge of collecting, measuring, and understanding indicators

There are many potential pitfalls in moving from a situation where there has been very little province-wide information available – and virtually no data comparing residential service providers with each other – to one where key indicators are used to “evaluate the effectiveness of programs and services across service providers and sectors”. These include (1) finding an appropriate measure or indicator, (2) ensuring that the information collected is accurate, (3) minimizing the response burden and data collection costs, (4) interpreting the results in their appropriate context, and (5) ensuring that services are not inappropriately incentivised to maximize good rankings on indicators at the cost of other important unmeasured dimensions. The challenge of dealing with these issues is particularly complex in a sector where the use of psychometrically validated measures varies significantly, where there has been very limited public reporting of data, and where there isn’t enough information available to establish contextualized baselines for setting performance targets.