Fresh from its election victory in December, the UK government has put civil service reform high on the agenda. Writing in the Telegraph last week, Rachel Wolf, one of the authors of the Conservative Party election manifesto, added ‘superforecasting’ to her wishlist for civil servant training. We agree that this is badly needed. So what is superforecasting and how can it help government?
The term ‘superforecasting’ entered the public discourse in 2015 when Professor Philip Tetlock and Dan Gardner published their book ‘Superforecasting: The Art and Science of Prediction’, which showed that the ability to predict future events is a skill that can be learned and developed over time. The key is to ask people not just to make predictions, but also to think about how certain they are about them, and then to ‘keep score’ based on how well their predictions match reality.
Over the last 18 months we’ve conducted surveys with over 500 UK policymakers and found that over two thirds of respondents showed signs of overconfident judgement (the same problem affects BIT and our readers):
The first wave of government superforecasters
Last year we asked 81 officials across the British government to make predictions about important, near-future questions. In line with Professor Tetlock’s work, we asked participants to quantify their beliefs about the likelihood of successful delivery and outcomes (not just ‘X will happen’ or ‘X is very likely’ but ‘X is 82% likely to happen’).
Here is how they did:
Across everyone who gamely took part – and we are indebted to each of them – some made much more accurate predictions than others:
And here were the top performers, our first wave of government ‘superforecasters’:
- Charlotte Hesketh, HR Coordinator at BIT
- Anonymous, Senior Whitehall Civil Servant
- Charlotte Riley, a Civil Servant in the Cabinet Office
- Michael Kaemingk, Senior Advisor at BIT
- Nicholas Pole, Futures Lead at the College of Policing
We’ve asked five additional questions about events that will or won’t have happened by the middle of this year, so we look forward to updating the rankings again soon.
Our 2020 vision
Thinking clearly about the future is something all humans struggle to do, and we have been impressed by the energy with which our participants so far have thrown themselves into the challenge of doing it better. In government, the costs of imperfect foresight are particularly high. Our hope is that a culture of superforecasting can help departments avoid being overtaken by predictable crises.
But to get there, we need your help. More specifically, we need your questions. We want to start generating forecasts that can be a concrete input into government planning and decision-making, and this means asking about the issues which departments and agencies really care about. For example, these might be questions about when that mission-critical IT upgrade will actually be working, how many people will sign-up to a flagship initiative, or what staff turnover is going to be in a stretched team over the next sixth months.
If you work in government and are interested in working together to help forecast the policy future, please get in touch at forecasting@bi.team.
