Implications for Recommendations

An effective province-wide cross-sectoral information and data management system is essential to ensure that Ontario is able to offer young people in residential care the quality of services that they need. Such a system should include (1) a clearinghouse which provides young people, their parents and professionals working with them with information about the full range of placement options that are available to them; (2) a data repository that can be used to monitor services, analyze service trends and track outcomes; (3) a research and evaluation strategy that supports innovation and implementation of evidence-based practices.

Existing Information and Data Collection Systems

Although there currently is very little province-wide information being reported about young people in residential care, the Ministry collects an enormous amount of data about these services. Crown Ward reviews, licensing reports and serious occurrence reports are rich under-utilized sources of data. The Ministry has already developed a province-wide Youth Justice information system (Youth Offender Tracking and Information System). The Ministry is also investing significant resources into the development of CPIN, the province-wide child welfare information management system. CPIN tracks many key indicators that could be used to monitor services, analyze trends and track outcomes. The placement module in CPIN can be expanded to include information about all residential service providers. The deployment of CPIN has been slower than expected and in several of our consultations concerns were raised about its implementation. It is also important to note that CPIN has limited capacity to allow for complex analyses to be conducted with data from pre-existing systems. To analyze long term trends, analysts will need to access legacy systems; OCANDS has the capacity to be used to tap into those legacy systems.

Service providers also collect a range of important information: the Assessment and Action Records used by child welfare agencies through the OnLAC initiative and the Partners in Care project conducted by OARTY include a rich amount of information about young people’s relationships, education and well-being that can be used to help understand service needs and outcomes. As discussed, these data collection efforts have serious limitations alongside their strengths. In reviewing these various tools we considered the merits of selecting a single assessment tool to be used across all service providers. While a common tool would have some advantages, such as simplifying referral procedures between service providers, and facilitating comparisons of services and outcomes between different service providers, we were concerned that imposing a single assessment tool could (1) undermine clinical approaches that integrate existing tools, (2) be poorly completed by staff who may fail to buy into a Ministry imposed tool, and (3) risk moving the Ministry’s role from oversight to administration. These tools nevertheless provide helpful information that include indicators of key dimensions that can be compared across different assessment platforms and analyzed at the aggregate level. MCYS does not intend to mandate the use of any specific assessment tool and the Panel found this to be appropriate.

User satisfaction ratings are one area where new data collection systems may be required. Other jurisdictions, including British Columbia, collect client satisfaction information following the provision of services to children and youth. In Ontario, informal reviews for certain residential services such as shelters are sometimes posted in online forums such as Google Reviews, but there has been no systematic collection of user satisfaction from young people who have experienced residential services. The exception is youth justice, where youth feedback is gathered through a youth experience survey completed with the probation officer following discharge from custody. A parent/guardian experience survey is also used.

Legal and Ethical Considerations

Opening up of access to information about services must be balanced against a range of privacy and safety considerations. Accessing and sharing information about residential service providers was identified as a major challenge in many of our consultations:

Some of these issues were examined through the Ministry’s recently completed review of the CFSA. Given the Ministry’s role in funding residential services and its licensing function, it will be important to ensure that the Ministry is able to access information needed by the proposed Quality Inspectorate to meet its function. A framework will also need to be developed to determine what types of information can be shared at what level of aggregation and with whom.

Data Infrastructure Needs

Given the wealth of existing data and the significant investments that are being made to develop common information in the child welfare sector, we do not think that a new residential services information system is the appropriate way to go. Initiatives like OCANDS demonstrate how programming and data analytics can be used to effectively combine information across different platforms. While some information collection systems require some enhancements – Serious Occurrence Reports are currently faxed into regional offices, Crown Ward Review data are provided to children’s aid societies in spreadsheets, CPIN has limited cross-sector information sharing capacity and there is no comprehensive strategy for using the data to inform policy – we have concluded that a focused investment in developing programming and data analysis capacity would be more effective and expedient than creating new data collection systems dedicated to residential care.

In addition to developing the analytic infrastructure for making better use of existing data, a commitment to a standard and comprehensive set of public reports is a powerful mechanism for ensuring that data are used and for creating incentives for improving data quality. The Ministry is currently reporting on several child welfare indicators and on important trends in Youth Justice. These initiatives could be significantly expanded by including a broader array of indicators. In Chapter 10, we review many of the indicators used in other jurisdictions and present a framework for selecting key indicators that should be reported on an annual basis. We also discuss some of the analytic issues that have to be addressed before reporting publicly on meaningful data comparing service providers.

One of the information and data challenges that will need to be addressed will be finding effective methods to track young people who experience placements across different sectors. The sector-specific information systems that are currently in place fail to provide a good mechanism for tracking and understanding the placement trajectories of these “cross-over” youth. Given that these are some of the most vulnerable young people in residential care, it is critical that enhancements made to existing information systems provide the Ministry with the ability to track and analyze the service trajectories and outcomes for these young people.

Along with enhancing the Ministry’s data analytic capacity, service providers and independent researchers also need to be supported to make better use of the data that they produce. Independent researchers should be encouraged to access non-identifying data to conduct more in depth analyses of these administrative datasets. In addition, research studies based on selected samples of young people in residential care are important tools to generate more nuanced and in-depth understanding of the experiences of young people and the outcomes of services.