Five years of What Works
Monday saw the publication of a 5-year update on the UK’s What Works Network. The event was hosted at the Institute for Government and introduced by Oliver Dowden, Minister for Implementation at the Cabinet Office.
There’s a lot to be proud of. We’ve gone from having one such centre in 2010 (NICE, formed in 1999 to better inform medics about which treatments work) to having ten What Works centres today. The centres cover schooling, crime, local growth, early intervention, better aging and wellbeing, plus cross-cutting centres in Scotland and Wales.
The newest arrival covers youth social work, its necessity explained on Monday by Isabelle Trowler, Chief Social Worker for Children and Families. Nationally she has seen the 152 local authorities take 152 different approaches to youth social work – it may well be that there were several different ways that are highly effective, but which? It’s not just £7bn in expenditure, but 36,000 social workers, 70,000 foster carers, and around 400,000 children at risk.
Alex Murray, super-cop and founder of the Society for Evidence Based Policing, gave fascinating insight into the practical reality of bringing more evidence-based approaches to his profession. He shared the example of an randomised control trial (RCT) showing how redirecting first time offenders away from the criminal justice system led to significantly less subsequent offending (a result suggested in longitudinal data, but not proven until the RCT).
At the same time, it became clear that the trial and later implementation was at high risk due to the reactions of victims and officers. Both groups often have clear expectations about what should happen after an offender is caught. Alex talked about how they found they needed to develop a script for officers to understand the logic and results of the trial, not least so they could explain to victims why the police were handling the case in that way.
Kevan Collins, head of the Education Endowment Foundation (EEF), brought his characteristic blend of relaxed humility and awesome content. A million children in the UK have now taken part in EEF-funded RCTs, and they can make a real claim to have fundamentally transformed the evidence base for education.
I was struck by the details he shared about the large RCT they published last year on free breakfasts in schools (n>30,000!) The overall effect size is an advance equivalent to about 2 months of extra school. But what I didn’t know is that the effect is not limited to children from low income families, but instead leads to advances in performance across all groups. The kids who are no longer hungry can focus more, but they also take less of the teachers’ time in crowd control, and allow the entire class to do better with less disruption.
There was even an award from the Campbell Collaboration in recognition of the What Works contribution from policy.
Sense about Science, and Permanent Secretaries give evidence about evidence
This week also saw the publication of a new report by Sense about Science, assessing the use of evidence in government publications and announcements, and a session by the Public Accounts Committee on the use of evidence.
The SaS report is really neat, and its methods well worth stealing by other countries and public bodies. Essentially they take a sample of publications and rate them on whether they back their claims with evidence (note: they don’t take a position on whether the policies are right or wrong, just whether the department set out their evidential workings). They stop short of ranking the departments, but a scan of the summary results gives a pretty clear impression of which departments are doing a good job (eg. Transport, Health, and Business, Energy & Industrial Strategy) and others which have, well, room for improvement. It’s a nice balance between encouraging and gentle scolding – you might even call it nudging for good.
I haven’t seen the PAC session yet, but I’m sure it echoed the SaS report: some great examples, but also room for improvement. It will be interesting – and important – to see how the arrival of UKRI on the landscape, as well as the Treasury-led follow-through from the recent Barber review will move further forward the generation and use of evidence, including plugging some of the key gaps. The publication by Departments of the ‘areas of research interest’ – ten are now out – is also part of the story.
A European gathering on behavioural and experimental methods
Finally, we had a gathering this week of nine European countries, together with OECD and EU Commission representatives on the use of BI and experimental methods to share our respective experiences.
Some, like the UK, now have results, such as Germany and the Netherlands. Others are just getting going, like France, Austria, Portugal, and Iceland. We really are getting to the point where we can make the (bad) joke that ‘most governments are using social norms (and BI) in policy’.
A good week for evidence-based policy!
