Principles for selecting indicators

Given the challenges inherent in developing meaningful and useful indicators, the Panel has approached its review and recommendations on the basis of several principles:

  1. Indicators clearly linked to objectives: Indicators should be developed and selected on the basis of an outcomes framework that clearly articulates the short and long-term objectives of residential care.

    Reporting on what can be easily measured runs the risk that less important – or possibly even misleading – easily measured indicators end up incentivising service priorities in directions that do not reflect the objectives and values that should be driving services. Indicators should be selected on the basis of a framework, or logic model, that clearly articulates the link between the indicator and the desired outcomes.

  2. Incrementalism: Developing a top down set of indicators that also requires the creation of new information systems runs the risks of escalating costs and implementation resistance.

    1. Where possible integrate data across existing databases.

    2. Develop indicators that can be generated using the range of different clinical tools used by local service providers. Imposing a single tool risks undermining its effective clinical use.

    3. Make full use of available data before requiring the collection of new data.

    4. Use sample surveys and research studies to address complex questions rather than attempt to collect extensive information for every young person in residential care. With methodologically sound sampling procedures, small sample exploration provides for more meaningful and accurate information than whole population studies that are often fraught with corrupted data, poor management of data extraction processes and ethical issues related to research practices.

  3. Understanding before benchmarking: Extensive analysis and reporting should be completed to ensure that indicators are robust and truly reflect the objectives they are intended to measure. Most indicators are indirect measures of the intended outcomes and objectives. Before using indicators as benchmarks or performance targets:

    1. conduct extensive trend and contextualized comparative analyses at the provincial level

    2. report publicly

    3. support use of indicators in service providers’ local planning processes