In partnership with BEIS, we recently published guidance to support energy suppliers in accurately analysing the impact of smart meters on household energy consumption.
What are smart meters?
Unlike traditional meters, which simply register a running total of energy consumed, smart meters record half-hourly consumption and price data and can feed the data back to consumers (e.g. via an in-home display) and energy suppliers. This has important benefits for consumers and the wider energy system by reducing overall energy consumption. Smart meters also enable better management of our energy supply, reducing our reliance on fossil fuels – but that’s for another blog post.
How do smart meters reduce energy consumption?
BEIS expects smart meters to deliver energy savings through four levers.
- Direct feedback on energy use: the smart meter’s in-home display, for example, gives people real-time information about their energy consumption, nudging them to be mindful.
- Indirect feedback on energy use: smart meters enable automated reading and accurate bills instead of infrequent manual meter reads and estimated bills, so customers with smart meters are made aware of unexpected increases in their usage more quickly than those with traditional meters.
- Advice and guidance: engineers provide tailored energy-saving during the smart meter and in-home display installation.
- Motivational campaigns: Smart Energy GB, the independent body responsible for consumer engagement on smart metering, has an objective to motivate consumers to save energy using their smart meter.
BEIS estimates that the smart meter roll-out will reduce household electricity and gas consumption by 3% and 2.2% respectively. On average, households spend £1,400 per year on energy. Seemingly small savings of 2.2-3% add up to £30-£40 per year per household. By 2034, the Government estimates that totalled across Great Britain’s 28 million households, those savings will be worth £4.7 billion (with a further £1.5 billion expected across non-domestic energy consumers).
BEIS’s expectation of savings is based on early smart meter trials in Great Britain, evidence from other countries, and suppliers’ own analyses of impacts throughout the rollout, but it is important to continuously validate the magnitude of these savings.
Measuring savings from smart meters is not easy, however. How do we model smart metered households’ counterfactual consumption? How much energy would these households have consumed had their supplier not installed a smart meter?
Comparing smart metered customers to a ‘control’ group
Ideally, we would want to test the impact of smart meters using a randomised control trial (RCT). A sample of consumers would be randomly sorted to create two groups which have, on average, identical characteristics that affect how much energy they consume. Those in one group would receive a smart meter, those in the other would not. We could then evaluate the impact of the smart meter on consumption by measuring the difference in energy usage.
In reality, smart meter allocation has been influenced by customer attitudes, eligibility criteria, and suppliers’ marketing and targeting strategies.
Customers who are more attentive and/or savvier about making energy savings may be more likely to request a smart meter. In comparing them to customers without smart meters, we need to be careful to avoid misinterpreting their energy-savviness as an effect of the smart meters, when it might actually be a cause of having a smart meter. The logic could go the other way, too – people using more energy and paying higher bills may be the ones who are more likely to want smart meters. Either way, simply comparing average consumption between customers with and without smart meters is unwise.
Pre-post analysis
Another possibility might be to compare consumption before and after installing a smart meter – a ‘pre-post’ analysis. This, too, could be misleading, as external events other than the smart meter might affect consumption through time. Seasonal effects from year to year can cause increases or decreases in energy consumption that we might misinterpret as an effect of the smart meter, and it’s challenging to ‘control for’ changes in weather. Moreover, household energy consumption has been steadily decreasing over the past decade – simple pre-post analyses would not account for this trend. Perhaps most importantly, events such as the COVID-19 outbreak – which suddenly increased working from home and thus domestic energy consumption – make ‘pre-post’ analyses even more challenging.
What does BIT suggest?
In the guidance we produced for energy suppliers on how to analyse the impact of smart meters, we drew on two classic ideas from econometrics: 1) matching and 2) difference in differences. In theory, you only need one of the two, but we think combining them provides extra rigour.
- Matching: the idea is to compare smart-metered customers with traditional-metered customers in the same region who have similar levels of previous consumption. Past consumption is very predictive of future consumption in energy analysis, which is why BIT is a fan of matching for this purpose. By matching within regions, we ensure that the comparison of the customers’ consumption is not confounded by major differences in weather between regions.
- Difference in differences: in addition, instead of calculating the difference between the two groups’ post-installation consumption per year, we recommend calculating the difference between the groups’ change in consumption per year. This ‘difference in differences’ approach is a further assurance that differences in baseline consumption do not confound our comparison, and that we account for UK households’ steady decrease in energy use.
We do dream of working with a supplier which allocates smart meters randomly to customers (for example, where there is a waitlist). Assuming that exit from the waitlist were truly random, this ‘natural experiment’ would give us even higher confidence in the validity of comparisons between groups.
However, in the absence of this dream project, we believe our recommended strategy is a robust alternative to an RCT. We are also excited about suppliers using this approach for analyses of other products, services, or interventions where customers’ energy consumption is an outcome of interest. Please read our full guidance here and let us know what you think at hello@bi.team.
