In its April 2023 launch event, The Ending Youth Violence Lab (at the Behavioural Insights Team) unveiled its two inaugural research projects. We were delighted to be one of two charities selected to receive the Lab’s transformative support in evaluating our work. For us, the partnership marked a significant milestone in our journey to evidence the impact of our flagship programme.
About Khulisa
Through our flagship therapeutic programme ‘Face It,’ we support young people who have a history of adversity and trauma that has put them at risk of exclusion from school.
Informed by the latest developments in neuroscience, our group therapeutic programmes enable them to build their social and emotional skills. At a time when schools are struggling with limited skills and resources to support children with complex needs, our therapeutically trained facilitators step in to provide intermediate support for young people who are otherwise unable to access critical mental health interventions.
Delivered experientially using a mixture of group activities and 1-to-1 reflection sessions, ‘Face It’ aims to build young people’s emotional self-regulation, coping skills, resilience, and overall social and emotional wellbeing through creative techniques like storytelling, art, debating & role-play. In doing so, our programmes not only help young people improve their wellbeing, they close the social and emotional skills gap by helping the most disadvantaged young people develop the skills required for success and dealing with adversity.
Driving Impact: Khulisa’s Journey from Theory to Causality
Our commitment to evaluation and impact measurement is deeply ingrained in everything we do. Our Theory of Change, developed in 2018, is informed by a robust evidence base which shows that building young people’s basic foundational social and emotional skills is critical to not only mitigating the negative behavioural effects of trauma but also to the promotion of lifelong learning & success. Over the last few years, we have been committed to testing and proving this premise.
In 2019 our programme was one of ten high-potential social and emotional skills programmes funded by NESTA. The fund was designed in recognition of the fact that social and emotional skills are employability skills associated with financial stability and desistance from crime in later life. Over the course of 12 months, we worked with Nesta and the University of Sussex to test our programme methodology while contributing to the growing evidence base which links social and emotional skills and wellbeing with improved life outcomes. This evaluation showed that after just 6 weeks of working with us, young people had:
- Improved wellbeing: 68% of our participants report improved wellbeing – with a paired t-test indicating the reported improvement among our participants is statistically significant.
- Greater resilience: 62% of our participants report an increase in resilience.
- Better coping skills: 72% of our participants reported using the coping skills they learned on our programmes.
- Improved emotional self-regulation: 55% of participants reported an increase in their use of positive regulation strategies.
Findings also specifically highlighted the power of the social element of Face It. Young people often reported an improved sense of belonging, as well as learning how to relate to others, empathy, tolerance and communication – all attributes associated with successful life outcomes.
To build on the learnings from this evaluation, in 2021 we launched a multi-year quasi-experimental design study with ImpactEd, evaluation experts in the education sector. The final study found statistically significant improvements across all our core outcomes. In an associated report, we also observed promising results in relation to school attendance. At a time when national attendance levels fell by 6.74% due to the pandemic, attendance for Face It participants dropped by just 0.7%. This was a significant finding as our young people are often most at risk of disengagement from school.
Having established that our Theory of Change targets the right outcomes and promising results from our first evaluations, we recognised the need to go one step further and establish causality. This would allow us to not only demonstrate the impact of our programmes but to understand the mechanisms behind that impact.
Now, our journey takes us to an exciting new phase of collaboration with the Lab. In addition to conducting a pilot randomised control trial (RCT) focused on testing feasibility, we will be working together on a process evaluation, which will help us understand how the process of change happens. We have begun delivering Face It in two London schools for this project. In each of these schools, researchers from the Lab have been randomising young people and collecting their outcome data. We will be delivering to 3 more schools next year.
We’ve already learnt so much about how to best evaluate Face It in the context of a randomised trial – what challenges we’re likely to encounter, and how best to overcome them. Our partnership with the Lab has also driven us to enhance our internal evaluation processes. To support the exploration of the connection between our programme and improved outcomes, we have set up processes to standardise our programme so that we can assess programme fidelity (this is something that can be difficult to capture when delivering an emergent therapeutic programme) and to better track participant dosage (i.e. how many sessions a young person attends and what was delivered in those sessions). These changes will help us better understand a) the extent to which our programmes align with our intended model and b) the influence of participation levels on outcomes.
At Khulisa, our journey is a testament to our commitment to driving impact. From theory development to causality, we have consistently sought ways to validate our work and understand the ‘why’ and ‘how’ behind our impact so that we can influence policy and practice. As a small charity, we recognise that we can only help so many young people directly. However, in building a credible body of evidence on effective practices, we hope to improve the lives of thousands more young people and the professionals who work with them. We also want to learn how to improve our programmes and our impact.
We look forward to publication of the Lab’s findings next year, and its recommendations supporting us to improve and refine our approach to evaluating impact. Before then, we will be delivering Face It in 3 more London schools, and hope to publish another blog sharing more about the experience of delivering Face It in these schools. Our partnership with the Ending Youth Violence Lab is a testament to making a tangible difference in the lives of young people, all anchored in the pursuit of evidence and lasting impact.
To find out more about this project, check out our website.
