One of the most cost-effective ways of scaling the use of behavioural insights (BI) around the world is by building BI capacity. BIT does this by running projects in close collaboration with partner organisations.
These partnerships are not only useful for spreading BI – they are essential for delivering successful projects. While BIT provides BI and impact evaluation expertise, local champions make sure that all relevant stakeholders are involved, provide valuable insight on cultural and contextual nuances, and are ultimately in charge of implementation.
One of BIT’s early capacity-building partnerships in Indonesia, funded by the Global Innovation Fund (GIF), has been in collaboration with BPJS Ketenagakerjaan (BPJSTK), the Indonesian social security agency for the labour force. Although BPJSTK hadn’t run any BI projects when BIT first approached them in 2017, the team didn’t need much convincing — several members of staff were already familiar with behavioural economics and were keen to apply these ideas to public service delivery.
Our first joint project followed BIT’s TESTS methodology:
The first step was to identify a specific project objective aligned with BPJSTK’s priorities. We needed a specific, changeable behaviour that a BI project could make a meaningful contribution towards.
We decided to focus on encouraging on-time payments by companies and reducing their contribution arrears. These are well-defined and easily measurable behaviours and, while it is clear what the ‘right’ thing to do is, many companies fail to engage in them: more than 50 percent of companies registered with BPJSTK had been in arrears at least once during the previous year. This put employees and their families at risk, as BPJSTK’s programmes only offer protection if all contributions are paid.
To understand the context and behavioural barriers to on-time payment and paying off arrears, we conducted field research with BPJSTK frontline staff and staff in companies in charge of human resources and finance. Our objective was to understand the perspective of the company staff that make the decision to pay (or not to pay) employees’ contributions. While the BPJSTK team had visited branch offices in the past, they had never done so with the objective of understanding a specific challenge for the end user, i.e. the individual or institution whose behaviour they wanted to change.
By the end, we had identified two behavioural barriers: many companies were either not aware of the consequences of non-payment and prioritised other payments, or didn’t even know whether and how they should pay.
Promising BI interventions are often low-cost and easy to implement within existing systems. Our project was no exception: we designed an email to remind companies to pay their contribution on time and repay potential contribution arrears.
We know that reminders can be impactful, but that details matter – not every message has the same effect. We therefore tested four different behavioural messages: 1) encouraging employers to follow the social norm of paying contributions; 2) encouraging employers to make a plan to pay that month; 3) highlighting the risk of prosecution if employers failed to pay; and 4) highlighting the risk that employees faced if employers failed to pay. Messages 3 and 4 addressed the first barrier identified during fieldwork. Every email also outlined in detail how contributions had to be paid, thereby addressing the second barrier.
Designing an intervention can be tricky, especially when translating previously tested messages into a different language and context. For BIT, working together with and getting candid feedback from local partners is therefore crucial.
For example, for the social norm message, BIT originally suggested ‘You are in the minority of Indonesian companies that do not pay their contributions on time’, pointing out that someone is in the minority often strengthens the norm message. However, ‘minority’ translates to ‘minoritas’ in Bahasa Indonesia, which is often used to refer to minority groups and can have a negative connotation. BPJSTK therefore suggested ‘Be part of the majority of companies that pay contributions on time’, which ended up in the final email version.
To rigorously evaluate the impact of our emails on the outcomes of interest (on-time payment and arrears repayment), we designed a randomised controlled trial: on 7 September 2018, BPJSTK sent close to 100,000 companies either no email or one of the four email variations as a reminder to pay their contributions by the 15 September deadline.
While the BPJSTK team found the attention to detail enforced by BIT during the trial design both mind-boggling and tiring, the results compensated for the efforts: we were able to show rigorously that highlighting the risk of prosecution led to a 2.5 percentage point increase (10% in relative terms) in the number of companies to making on-time payments and almost doubled the repayment in August arrears by the end of September (see Figure 1 and the full report for more detail).
Figure 1: Impact on likelihood of a payment by the deadline
BPJSTK’s research team is now working with other departments to decide how to scale up this intervention: while the ‘Risk of Prosecution’ email was the most effective, we unfortunately did not find any sustained impact on payment behaviour after the first month. However, resending the same message every month might not be effective either. The research team is therefore considering to test whether supplementing the one-off ‘Risk of Prosecution’ email with simple reminders would deliver long-term impact.
Being able to rigorously demonstrate the positive contribution of their work towards BPJSTK’s organisational objectives has greatly helped the research team gather support from the rest of the organisation for future projects.
The collaboration between BIT and the BPJSTK won’t stop here – we have already started our next project to tackle an even more challenging problem: discouraging companies from underreporting the number of employees and salaries to avoid paying higher contributions. This project will be a further step in BPJSTK’s journey to adapt an experimental mindset to all their work.
BIT will soon be publishing a guide on how to run simple BI projects using our TESTS methodology, so watch this space if you want to find out more.