Existing Outcomes Frameworks and Reported Indicators

The Ministry, along with most jurisdictions across Canada and internationally, has been putting increasing emphasis on developing methods to track and report on outcomes for young people in residential care. The Ministry’s Strategic Plan for 2013-2018, Growing Together, articulates four overall goals: 1) Children and youth are resilient; 2) Children and youth have the skills and opportunities needed to shape their own lives; 3) Children and youth have a voice; 4) Children and youth experience high-quality, responsive services. While these goals map well to the overall goals that should guide a residential services delivery system, they need to be translated into a more specific set of objectives that reflect residential care processes.

Building on the Ministry’s Strategic Plan, the Youth Justice Outcomes Framework identifies four specific outcomes: 1) improved functioning and positive social behaviours, 2) increased skills and abilities, 3) increased youth engagement with supports and 4) decreased re-offending. Nine indicators have been selected to measure these outcomes, although other than the recidivism indicators, most are still under development and will require the introduction of new data collection instruments.

For the child welfare sector the Ministry is currently publicly reporting on five “performance indicators” in three key areas: 1) safety, 2) permanency and 3) well-being. Safety is measured on the basis of two indicators of recurrence of investigation. Permanency is tracked on the basis of two additional indicators: days of care, by placement type and the time it takes for a young person to be reunified, placed in a permanent alternative home or discharged from care; and well-being is measured for young people in long-term care who report on the quality of their relationship with their caregiver. Additional performance indicators are being developed in collaboration with the OACAS and with the Association of Native Child and Family Services Agencies of Ontario (ANCFSAO).

The three domains of safety, permanency and well-being are similar to domains reported on in a number of jurisdictions across Canada and internationally (e.g., Quebec, British Columbia, Alberta, California). Jurisdictions vary considerably with respect to public reporting on these indicators. The most longstanding source of reports is generated by the U.S. Department of Health and Human Services through their Child Welfare Outcomes Report to Congress. The 2010-2013 document reports on seven indicators, including comparative state-level data, focusing primarily on safety and permanency. A number of jurisdictions also report on well-being indicators. These focus most often on educational outcomes, such as grade-level (e.g. British Columbia) or math and reading scores (England) and in some instances health (e.g. immunization and dental exams, substance misuse - England).

Surprisingly, we were unable to find many examples of system-wide publicly reported data on well-being indicators in Ontario. Well-being indicators are more frequently reported in sub-populations followed through specific assessment initiatives. A growing number of residential care providers are tracking outcomes using a range of self-report measures. For instance, Ontario’s Looking After Children (ONLAC) assessment tool which is completed with many young people in long-term care includes a number of well-being indicators.

Most of the outcomes framework we reviewed focused on sector-specific systems level outcomes, specifically ones related to child welfare systems and youth justice. While many of these, especially permanency and educational outcomes, relate well to the types of indicators that could be tracked for young people in residential care, we found fewer examples of reported indicators that monitor quality of care and the everyday experience of young people. The American Association of Children’s Residential Centers has developed a promising framework based on four types of indicators: practice/process indicators, functional outcomes, perception of care and organizational indicators (American Association of Children’s Residential Centers, 2009). While the AACRC framework is a helpful conceptual model, it has not yet been implemented as a data reporting framework.