How many of us have our heating programmed to automatically come on when we get home? In which case, how often do we come home late, to find the heating has been on unnecessarily? Or perhaps we find ourselves watching TV, suddenly a little hot, realising we could have turned the thermostat down half an hour ago? Or maybe, like many, we’ve simply failed to get to grips with programming our boiler controls at all?
Conventional heating controls can be difficult to use, and we often fail to turn them off or down as much as we could – such as overnight when comfortable temperatures are lower. This leads to wasted energy and money, and considering a 1-degree reduction can save around 10% on your heating bill, this wastage is far from trivial. In a world where every tonne of carbon matters for the planet, and every wasted penny matters for consumers, helping people to use their heating more efficiently seems like an obvious win.
Trialling consumer advice
With this in mind, in 2014 we worked with the Department of Energy and Climate Change and Newcastle City Council to test the impact of providing advice to consumers on efficiently using their heating. With around 1500 homes we trialled written advice in the form of a leaflet, and face-to-face advice from boiler engineers, against a control group receiving nothing. What did we find? Zero impact on energy consumption. Though disappointing, it was a critical piece of evidence that led to substantial funds being redirected to more effective policies (when it comes to evidence-based policy, null results are often as useful as positive ones). It also resonates with the wider behavioural literature showing that where forgetfulness, ingrained habits, hassle and effort are involved (which is very much the case with our use of heating controls), giving people information is rarely enough. It might increase awareness, or even raise intentions to act, but that’s a long way from changing actual behaviour.
Nest – making energy savings easy
One of our mantras is ‘if you want to encourage a behaviour, make it easy’, and one of the best ways of making something easy is to automate it. Enter stage left: the Nest Learning Thermostat. The Nest uses sensors and machine-learning to understand the thermal properties of your building and your occupancy habits, and tweaks the heating accordingly. It turns heating off or down when the home is empty, and makes corrections for weather since we don’t feel as cold inside when it’s warmer outside. If you want to use the Nest manually, it’s easier and more intuitive than most traditional controls, and even nudges you towards more efficient settings by encouraging you to collect ‘green leaves’. But if you’re not interested in interacting with the device at all, it aims to do a pretty good job on its own.
But does it work?
Until now there has been an absence of robust evidence on smart heating controls in the UK, Nest or otherwise. Marketing claims exist from some manufacturers, but these tend to be estimated based on modelled simulations, not real-world evidence. We started working with Nest in 2015, acting as independent evaluators to fill this evidence gap.
Since then we’ve undertaken four studies using data from Npower customers. First, a 4,500-home propensity-score-matched study, comparing 2250 homes that had chosen to buy a Nest, to a matched sample with various other heating controls. We found evidence of around 5.8% savings in household gas consumption – extremely promising, but the matched-sample methodology didn’t quite meet our high standards of academic rigour (though it was undoubtedly the most rigorous study of its kind in the UK to date, and garnered encouraging policy interest). More critically, this study left two key questions unanswered: Firstly, energy savings compared to what? We’d demonstrated savings compared to an unknown mix of other heating controls, but it is far more useful for policy-makers, consumers and engineers to quantify the savings relative to a specific comparison technology. And secondly, can the Nest achieve these savings without making people cold? Unlike insulation products which improve the thermal envelope of the building, the Nest saves energy by reducing the heat put into it – in theory only when users won’t notice, but this needed to be proven.
In 2016/17 we therefore ran a small-scale randomised controlled trial, this time drawing on higher quality smart meter data. We had a specific comparison group – the ‘modern suite’ of controls used by 49% of UK homes (a programmable timer, room thermostat, and radiator valves). Here we find evidence of savings of around 6-7% of the heating system’s gas use, or 4.5-5% of total household gas consumption. This is achieved with no loss of user comfort (and possibly a modest improvement).
Weather-corrected gas consumption starts to dip after installation of the Nest Learning Thermostats
Finally, we ran an independent evaluation of data collected by Nest from 20k devices, analysing the impact of their Seasonal Savings programme – an opt-in feature which seeks additional savings in the winter by making incremental reductions of a fraction of a degree when users are least likely to mind (such as at night), unless the user intervenes. Here, we find additional savings of 4.5% of heating-system gas use (or 3.3% of total household gas use) among the 85% of eligible Nest users choosing to adopt the feature.
So what’s the significance of all this? Well, for one – the Nest seems to deliver on its promise, and the magnitude of savings are substantial. We think this provides a strong case to develop policy measures that support smart heating controls where savings are similarly demonstrated through rigorous evidence. Compared to similar evidence of retrofits like loft, cavity-wall and solid-wall insulation, the payback period of the Nest is shorter: it costs £279 installed, and we estimate it saves £45-65 per year (for medium-large homes respectively, and assuming Seasonal Savings is used). Moreover, recall that this is compared to a ‘modern suite’ of heating controls. Though the evidence doesn’t exist, it’s reasonable to assume savings may be greater where more rudimentary controls are being replaced (for example, 23% of homes have no thermostat at all).
Empirical data versus modelled estimates
There is also a more fundamental narrative to this work about evidence-based policy making. The field of energy efficiency is dominated by engineers (no qualms about that, I am one), and this is reflected in the widespread use of models, or simulations, to predict energy consumption. These models are used, for example, to estimate the efficiency of your home based on all its known physical characteristics, to give it an A-to-G rating on your Energy Performance Certificate (EPC). In this manner they’re a substitute for real-world data, and a useful one: obtaining real-world measurements of every home, and measuring the impact of every imaginable modification to that home, would be impossible.
Problems arise, however, when models and simulations are no longer a substitute for real data, but become so central to policy that they exclude real-world data even when it is available and more accurate. This has tended to be the case, with technologies like boilers and insulation recognised within policy (EPCs, the Green Deal, ECO, and many others), not for their real-world performance, but for their modelled performance, and the two can be quite different. This reliance on simulations is even more problematic for Nest and many other disruptive, smart technologies because the UK’s official energy-efficiency model cant actually compute the savings they generate: the model assumes human behaviour is fixed, and that the building needs a set amount of heat, whereas Nest works precisely because humans are complex and flawed, and some of the heat delivered to a building is wasted.
We believe the solution is for energy efficiency policy to put greater onus on real-world, empirical evidence: evidence of energy savings in-situ, with people and their messy behaviour taken into account. The good news is that the UK government is starting to do precisely this: the recently announced Boiler Plus policy is a case in point, with technologies (including smart heating controls) mandated on the basis of empirical evidence like ours, not just their modelled performance. This argument, for empiricism over modelled assumptions, is one we’ve been making for years – generally in the world of policy driven by economic assumptions about behaviour, and in this case, engineering assumptions about behaviour. The key message is the same, however – people’s behaviour matters a great deal, and can’t be put aside with convenient assumptions. We think policy should reflect this fact in every way it can.