The Outcomes Star is an approach used to measure change when working with vulnerable populations, particularly homeless people. The model was developed by Triangle Consulting, originally for St. Mungo’s Broadway, a UK charity dedicated to supporting homeless people. There are several versions of the Outcomes Star adapted to different groups, such as the elderly and people suffering from mental illness, yet we will focus on the Outcomes Star approach adapted for homeless populations.

How it Works

The Outcomes Star method is based on the idea that people experiencing change move through several stages while transitioning from dependence to independence. The Outcomes Star approach identifies ten outcome areas: motivation and taking responsibility, self-care and living skills, managing money and personal administration, social networks and relationships, drug and alcohol misuse, physical health, emotional and mental health, meaningful use of time, managing tenancy and accommodation and offending. Within each outcome area, the Outcomes Star approach measures change through a specific ladder of change. The Outcomes Star Chart, pictured below, provides a visual representation of this process.

Outcomes Star Chart

Ladder of Change

The Ladder of Change combines the theory behind the Stages of Change model (discussed in the Theories to Support the Work chapter) with the Outcomes Star. The first stage of the ladder is “stuck”, which means that when people begin a change process, it is initially very difficult. Many people find it challenging to accept that they have a problem that requires change, and/or they may not want to accept help from others. The second stage is “accepting help”, which is when people seek outside assistance, as they no longer want to tackle their issues alone. The third stage is “believing” which is when people begin to acknowledge that they will be able to change. When someone begins to believe in their ability to change, they often anticipate future benefits, and are much more welcoming of outside assistance. The fourth stage of change is “learning”, which is when people try different strategies, in order to change. This is a challenging stage, as most people are not usually successful on their first attempt to change, which can be a discouraging process. The last stage of change is “self-reliance” which is when individuals are able to maintain change without assistance from others. Many individuals begin the ladder at different stages, and do not necessarily progress linearly, as it may take several attempts to reach the self-reliance stage. 

Ladder of Change Model

As you complete the Ladder of Change for each outcome area, you are instructed to measure your progress on the Outcomes Star, by indicating which number you are at, for the different ladders of change. Once you have chosen a number for each outcome area, you link the different numbers together to create your own star, which represents your progress and room for further development.

The image below is a composite of several youth and their progress as viewed on the Outcomes Star. A profile of an individual youth would look similar.

Program Outcomes for Individual Youth 

(8 February 2013 - 9 September 2014, Vancouver)

Homeless Hub Thoughts:

Overall, both organizations do a great job collecting and using data and making sure that all programs are evaluated. While many funders ask for this level of evaluation, few provide the financial supports to achieve it. Covenant House’s private funding base allows them to dedicate the resources necessary to build evaluation in to their work across the board. 

Evaluation is certainly critical to any program operation. There are two ways to think about evaluation in a transitional housing program model: evaluation of the individual youth’s progress and success of the program activity as a whole. Both agencies have established a variety of measures, including individual surveys, focus groups, the use of Efforts to Outcomes and the two youth engagement tools: the Youth Engagement Scale and the Outcomes Star.

We like Vancouver’s method of setting up the parameters for data collection, creating an ongoing process for gathering information and then not reacting to the data for a set period of time. This is a good practice to ensure that you are noticing a trend rather than outliers. One of the keys to good data collection is patience.

It is also important, as Vancouver has emphasized, to not “chase numbers” but have a clearly defined logic model and understanding of what your key indicators are. This makes it easier for you to gather and use data without staff feeling that all they do is fill in forms and create numbers.

Some of the factors in your evaluation may be based upon the needs of your funders. If their primary goal is for you to house youth faster, then you will need to be able to compare how long it used to take to house youth and prove that your methods are making housing youth easier and faster. If funders are more concerned about the development of life skills, then your evaluation methods would most likely include a pre and post life skills assessment of the youth (maybe every few months depending upon length of stay), as well as tracking the types of sessions offered, the number of residents who participated in each one etc.

Listening to the youth and seeking their input was also discussed by both organizations. They also emphasized the need to listen to staff as they are the ones working in the program at the ground level.  It is important to include youth and staff input in your data and evaluation methods. When creating your key indicators both groups should be part of that process. Ideally, hard numbers will also be supplemented by qualitative surveys, interviews or focus groups to help understand and expand upon the data. While singular anecdotes are not evidence, gathering stories can help explain trends.

Youth evaluation may form part of your overall data collection but can also be used in other ways. For example, as noted above, pre and post life skills assessments may meet the needs of a funder, but they can also be used to measure and support the progress of an individual youth.

Using tools such as the Youth Engagement Scale or the Outcomes Star measure success and progress by a youth but they can also identify areas of concern for a program. If only a few youth are progressing in a certain item on a scale it could be a sign that program changes are needed to improve service delivery in a certain area.

These tools are also integrated into the service delivery; they are not just separate, standalone evaluation measurements. For example, the areas being assessed in Toronto’s Youth Engagement Scale are all embedded into the goals of the program. They make up part of each youth’s case plan and are discussed in the Case Management Team meetings.