Skip to content
Menu
  • Blog
  • 13th Oct 2022

Reaching nonprofit program participants – developing more effective tools by learning what doesn’t work

Nonprofits do critical work. They help people access essential services, protect ecosystems, and more. But in order to fulfill these missions, many have to make the most of limited resources. That means pursuing strategies that are effective—and avoiding those that aren’t.

How will organizations know what works? BIT has helped many nonprofits use behavioural insights approaches to answer this question, from understanding what drives people to give to increasing program uptake.

In 2020, we partnered with United Way Halton and Hamilton (UWHH) on a collaborative Ontario Trillium Foundation grant to understand how behavioural insights might improve financial empowerment programming. UWHH supports an essential network of community organizations and is playing a leading role in exploring applied behavioral insights in the Canadian social sector.

UWHH’s lead community partner, Oak Park Neighbourhood Centre (OPNC), is based in the Halton region of Ontario and offers many vital programs and services, including tax filing clinics for low-income households. Although the service is free, and helps people who are eligible for refunds get about $5,000 on average, it can be difficult to max out participation.

Our first trial in 2020 tested a behaviourally-informed email encouraging people to sign up. The email performed well. Building on these results, we wondered if applying the same behavioural principles to direct mail (ie letters) might help reach an even broader set of potential beneficiaries.

We found that the direct mail letters did not have an effect—an important and useful insight.

What we did

Earlier this year, our team designed a randomized controlled trial to see if an intervention letter could increase participation in OPNC’s tax filing clinic. The letter used active-choice framing, asking people to make a decision about what they want to do while drawing out the implications of not taking action. We also tested an envelope teaser message.

While we expected the letter to have an effect, we also knew that program uptake would be generally low relative to the number of letters sent. Therefore, we planned to conduct a trial with a large sample of 15,000 people. Although running a test with fewer recipients may have been simpler and cheaper, it would have risked us overlooking a real, potentially meaningful effect. 

We leveraged Canada Post to help identify households in low-income neighborhoods using public data. Half of those households were selected at random to receive the letter. The other half became our control group. Among the households that were sent the letter, a further half got a special envelope with a teaser message.

Overall, 17 more people in the treatment group filed their taxes compared to those in the control group. However, these results were so small that we couldn’t rule out that they might be due to chance.

 

The letter used enhanced active choice framing.  Half the people in the treatment group received an envelope teaser. 

Key takeaways for nonprofits

  • Direct mail wasn’t an effective channel for service uptake in this scenario. While direct mail serves fundraising and donor communications well, other targeted outreach (e.g., volunteer canvassing in low-income neighbourhoods, billboard or bus stop ads, etc.) may be more effective for increasing program participation among people with lower incomes. 
  • Scientific rigour is critical. Applying strong research methods to practical challenges is integral to the behavioural insights approach. Because we ensured the sample size was sufficient and randomly assigned treatment and control groups, we were able to give confident results and recommendations to OPNC and UWHH. Let’s say we had assumed that sending a letter would have a significant impact, and instead, decided to launch an A/B test comparing two different versions of a letter. In that situation, we wouldn’t know whether our null results were due to both letters encouraging uptake, both letters having no effect, or both letters discouraging uptake. More testing would have been necessary. Having a control group (i.e., “no letter”) to first assess the efficacy of a strategy is imperative.
  • Collaborating with partners like Canada Post is easy and accessible. They generated a new contact list of 15,000 Halton residents for OPNC and UWHH and randomized the letters and teaser envelopes, allowing us to evaluate the intervention clearly. It may be a good opportunity for small- and medium-sized nonprofits to test other direct mail initiatives—like encouraging volunteering—with them. While Canada Post adheres to privacy law, organizations should consider potential unintended consequences of sending mail to people who aren’t expecting it, such as recipients feeling spammed or distrustful.

Want to learn more?

Many nonprofits strive to increase program participation, but reaching new people can be challenging. By applying the behavioural insights approach, organizations can learn with greater certainty which outreach tools are worth investing in, and ultimately make a bigger positive change in their communities. 

To learn more about our work with OPNC and UWHH and how behavioural insights can support nonprofit missions, please contact Sasha Tregebov at sasha.tregebov@bi.team, Michelle Knoll at michelle@opnc.ca or Vivien Underdown at VUnderdown@uwhh.ca.

Authors