Search

We look back on our proof of concept campaign and ahead to our pilot campaign


In our first blog post, we explained why we think family planning is a cause worth prioritizing. Enabling couples to access modern contraceptives may be one of the most cost-effective ways to avert maternal deaths, in addition to providing a host of other benefits.


Often, beliefs and attitudes are the largest obstacle to couples’ accessing modern contraceptives. FEM seeks to promote more informed beliefs and attitudes through educational radio campaigns. We operate in Northern Nigeria, where radio is affordable and widely trusted. We estimate that by 2025, FEM will have helped avert approximately 600 maternal deaths, at a cost of $3,600 per death averted.


In this post, we look back on our first half year, FEM’s proof of concept phase, and ahead to our plans for the remainder of 2021, FEM’s pilot phase (Figure 1). For those interested, we close with some opportunities to support us.


Our proof of concept (Sep. 2020-Feb 2021)

Approach and overall results

The overarching aim of the proof of concept phase was to learn, “is it feasible for us to implement our model?” We believe we passed the feasibility test.


In early February, we conducted a one-week campaign that:

  • Broadcast ads 140 times on two radio stations

  • Reached up to 2-3 million listeners

  • Was remembered by about four out of five radio listeners, according to a small survey we conducted

To successfully implement the proof of concept, we set three key objectives for ourselves: (1) select a promising location to launch, (2) develop implementing partnerships and build a network of supportive stakeholders, and (3) execute a short campaign. We describe how we approached each objective and our results in the section that follows.


Figure 1: FEM's proof of concept and medium term plans


Results by objective

(1) Select a promising location to launch

We selected Kano, Nigeria as our launch point after a three-month, multi-stage analytical process (Figure 2). It was important for us to rigorously evaluate our geographic options. Much as the impact of your charitable dollar can go further in low-income countries, the cost-effectiveness of family planning interventions can vary dramatically across regions and even within countries.

We used cost-effectiveness analysis, weighted factor modeling, crucial considerations, and expert opinion to narrow down a range of potential locations. Ultimately, our country selection came down to Ghana, Nigeria, and Rwanda. We selected Nigeria largely due to its more affordable radio airtime rates, which significantly improved the projected cost-effectiveness of our work. Within Nigeria, Kano stood out across multiple prioritization methods. Key decision factors were Kano’s high rates of maternal mortality, relatively consistent supply of contraceptives, supportive Ministry of Health, and low airtime rates (~$5 for a one-minute ad on a station reaching about 2 million listeners).


Figure 2: Location prioritization methods - country and state

(2) Develop implementing partnerships and build a network of supportive stakeholders

We secured support for FEM from key stakeholders at the federal and state level, and kickstarted implementing partnerships with several organizations. We prioritized establishing partnerships because they are necessary to operate in some cases, and can improve our effectiveness in others.


We formed productive implementing partnerships with:

  • One local implementing partner, iDevPro-Africa, which is based in Kano. iDevPro-Africa has supported our local stakeholder engagement, prototyping, and monitoring and evaluation. The organization is led by one of the foremost experts in family planning in Nigeria, Dr. Mansur Muhammad Tukur, who shares FEM’s vision and values.

  • Two radio stations. Freedom Radio, which targets a more urban demographic, and Arewa FM, which targets a more rural one, played our ads during our short campaign. We were able to secure discounted rates from them due to their commitments to corporate social responsibility.

  • Three radio production firms. We initially assessed firms through referrals and a request for proposals. Three firms then developed prototypes of ads. We then tested the prototypes with target listeners and selected one firm to produce the final versions for the short campaign.

We also worked to build a broad network of supportive stakeholders for our model. We held more than 40 one-on-one meetings with stakeholders and partnered with the Government of Kano State to hold two stakeholder workshops.


We’ve been pleasantly surprised by the positive reception we’ve received from the Nigeria Ministry of Health, Kano State Ministry of Health, Kano State Primary Healthcare Management Board, local NGOs, and religious and community leaders. As an illustration of this support, 43 stakeholders in Kano signed on to a communique saying they “welcome and support” our work, and our co-founders were formally invited to meet both the Emir of Kano (Alhaji Aminu Ado Bayero, Picture 1) and the Emir of Bichi (Alhaji Nasiru Ado Bayero), who expressed their appreciation for FEM’s work.


Picture 1: At the palace with the Emir of Kano









FEM co-founder, Ken (second from left), and iDevPro-Africa country director, Mansur (right) with with the Emir of Kano, Alhaji Aminu Ado Bayero


(3) Conduct a short campaign

In February, we placed two ads on two radio stations ten times a day for one week. We wanted to trial a short campaign relatively quickly, as we thought it would help us anticipate issues to address in future, longer campaigns.


Figure 3: Components involved in researching, producing, and broadcasting the short campaign

The week-long campaign we conducted was the product of a months-long process of:

  • Audience research. We reviewed surveys and reports, and spoke to dozens of stakeholders to understand what barriers to contraception to target in Kano. Our audience research helped us to avoid missteps. For example, one of our scripts referenced a husband’s “energy” leading to more babies. Several stakeholders indicated that this innuendo would offend some listeners or distract attention from our message. We replaced it.

  • Script writing and prototyping. Based on our audience research and other successful campaigns, we worked with our partners to develop seven scripts for ads. We created prototypes of three of the ads and gathered feedback from target audience members through focus groups and interviews. We also gathered feedback from stakeholders. We then selected two ads to be professionally produced.

  • Production and pre-testing. We conducted another round of focus groups (Picture 2) and interviews with target audience members to get their feedback on the professionally produced versions of the ads.

After the one-week campaign, we conducted a small-scale survey of radio listeners, which indicated that our ads were heard, remembered, and understood. About four out of five of those surveyed recalled hearing our ads. Half of those who remembered our ads were able to convey their core messages about family planning to the surveyor. We consider this to be a strong result after just a week of ads. Equally importantly, we received no negative feedback from surveyors or other stakeholders during or after the short campaign.


Picture 2 & 3: Focus group discussions conducted with target audience members

Reflections

What we did well

Small-scale experimenting. We try to conduct low-risk tests before making major commitments. For example, we had three production firms produce prototypes, and tested them with stakeholders and target audience members before selecting one for our proof of concept. Our research indicated that, at least in this case, the prototypes that most resonated with listeners were those produced by the firm with the most experience on health campaigns - as opposed to firms who extensive experience working in Kano.

Timeboxing. Timeboxing is the practice of setting a maximum duration to implement an activity. We’ve found it helpful to timebox activities where there is no clear endpoint or level to reach. For example, we gave ourselves six weeks to select a country to launch in. Timeboxing forced us to focus on only the most critical factors in the decision. It also meant that we could encounter implementation challenges earlier and reverse our decision earlier, if we had to (we didn’t).

Adapting our operating model. We strive to be a nimble organization that adapts as it learns and gathers evidence. For example, initially, we had planned to hire a Research and Communications Manager to provide on-the-ground support for our pilot in Kano. As we came to better understand the capabilities of iDevPro-Africa, we found that there were many advantages to working with them instead of hiring someone. We’ve benefited from having an entire organization that can provide far more support and expertise than a single individual could.


What we could have done better

Outsourcing legal registration in Nigeria earlier. Initially, we thought we could reduce costs by managing the process of legally registering FEM in Nigeria ourselves. We underestimated the time required to understand the process and complete the necessary requirements. In December, we hired a Nigerian law firm to manage the process for us, which has been well worth the cost in terms of management time saved.

Ensuring quality assurance on documents. We pressure test ideas and strategic and operational plans with several experts, including a FEM Advisory Board. However, until recently, we didn’t have a review system in place to catch proofreading errors. The quality of documents we shared was, in some cases, undermined due to typos and other such defects. We now share important documents with a proofreader for quality assurance.


Our pilot phase (Mar. 2021 - Dec. 2021)

Having answered to our satisfaction the question, “is it feasible for us to implement our model?” we now seek to learn, “how can we implement our model as cost-effectively as possible?” To answer this question, we’ve set out three key objectives to achieve over the course of a nine-month pilot phase:

  • Further calibrate our intervention approach and content to the local context. We’ll update our approach (e.g., number of ads per day, length of ads, type of shows) and content (e.g., barriers addressed) through desk research, barrier analysis surveying, human centered design, and other quantitative and qualitative research methods.

  • Conduct a pilot campaign. We’ll integrate findings from our research into a 2-3 month long campaign consisting of ads and hour-long radio shows.

  • Assess the effectiveness of the pilot. We’ll use a number of methods to assess the pilot’s effectiveness, including a potentially innovative one: creating a control group by disrupting our programming in one location with a short-range signal transmitter.

By the end of the year, we expect to have far more evidence of FEM’s impact (or lack thereof) in Kano. This evidence will inform our decision on rolling out a long-term campaign in Kano and conducting a pilot in other Nigerian states.


How you can help

Supporters can help FEM by:

  • Providing technical advice. We’re interested in speaking with more academics and other experts on family planning, social and behavior change interventions, and qualitative research.

  • Volunteering for short- or long-term engagement. We’ve identified numerous opportunities for volunteers from anywhere in the world to support us, especially in desk research and website design. You can express interest in opportunities here and read more about them here.

  • Contributing funding. We have room for more funding. For $700, individuals may sponsor a week’s worth of advertisement airtime in Kano. Interested donors may contribute on our donation page.

Overall, we’re encouraged by the progress we made through the proof of concept phase and excited about our prospects for the pilot phase. We look forward to hearing from you! Please feel free to reach out to us directly with your feedback, questions, or ideas.

Cross-posted on the Effective Altruism Forum


Updated: May 3

Working towards better health, education, and economic outcomes through more informed family planning and birth spacing


TL;DR Greater adoption of contraceptives has a lot of positive benefits. Studies indicate that radio campaigns can be highly cost-effective in increasing adoption. Our new organization will implement radio campaigns where they can have the greatest impact, starting in Nigeria. If this sounds promising to you, there are several ways you can help us move from proof of concept to full rollout.


By Kenneth Scheffler and Anna Christina Thorsheim


In September, we founded Family Empowerment Media (FEM), a non-profit organization that will help people plan their families. We will provide clear, compelling, and accurate information on modern contraceptives through the radio.

We believe a well-run family planning charity focused on social and behavior change (SBC) campaigns on the radio could achieve outcomes in the realm of the most effective global health charities. Increasing access to and understanding of family planning services has been shown to have substantial positive effects on health, as well as education, income, and several other dimensions. SBC campaigns on family planning have generally been cost-effective, and a recent randomized control trial indicates that high-intensity radio-based SBC campaigns have the potential to be especially cost-effective

This post describes (I) the challenge we aim to solve, (II) our approach, (III), our value add, and (IV) our progress and plans. We close with a quick note on (V) how you can help FEM navigate its first year.

I. The challenge: Modern contraception’s vast benefits are not fully realized due to information gaps


In countries with developing health systems, pregnancy can be a major health risk for women. According to the Guttmacher Institute, just under 300,000 women and girls in low and middle-income countries die of pregnancy-related complications each year. Pregnancy is the most common source of mortality for adolescent girls. Pregnancy is also a source of non-fatal but serious morbidity. Conditions arising from pregnancy and childbirth include obstetric fistula - which 50,000-100,000 women experience each year - postpartum anemia, and postnatal depression.


Helping women avoid unintended pregnancies is a cost-effective means of reducing maternity-related health burdens. Almost a quarter of women in low and middle-income countries want to avoid pregnancy but are not using modern contraceptives. This “unmet need” for modern contraceptives results in 85 million unintended pregnancies per year. If all women with unmet need were provided access to and used modern contraceptives 70,000 maternal deaths per year would be averted. Guttmacher estimates, “every dollar spent on contraceptive services beyond the current level would reduce the cost of pregnancy-related and newborn care by three dollars,” while helping couples realize their family planning intentions. The Copenhagen Consensus estimates that a dollar spent on access to modern contraception leads to 120 dollars of social, economic, and environmental benefits.

In addition to reducing maternity-related health issues, helping women avoid unintended pregnancies can have positive effects on education, income generation, and children’s welfare (Figure 1). For example, a study in Indonesia found that providing access to family planning was three times more powerful than improving school quality in keeping girls in school an extra year. Research in Colombia found that girls with access to family planning clinics were 7% more likely to participate in the formal workforce as adults. More long term, a Brookings Institution study examined the effect of providing access to family planning programs to American women in the 1960s and early 1970s. It found that access led to improved college completion rates of children and higher family incomes decades later. These spillover effects are hard to measure, leading to the possibility of them being neglected in traditional cost-effectiveness analyses.


Figure 1: Benefits of improved family planning


Use of modern contraceptives has increased in recent years, but so has the unmet need (Figure 2). For example, in Nigeria (our target location), the portion of women using contraceptives rose from 11.2% to 14.2% from 2012-2019. Meanwhile, the unmet need also rose from 22.4% to 23.7%. An increasing share of Nigerian women prefer to avoid or delay pregnancy, and uptake of modern contraceptives needs to grow faster to close this intention- behavior gap.


Figure 2: Growth in unmet need for contraceptives


Minimal or incorrect information about modern contraceptives is often a big barrier to greater use. Perhaps contrary to expectations, lack of access to contraceptive commodities is generally not a primary reason for low adoption. In Nigeria, just 5% of married women said physical access was a reason they did not use contraceptives. In contrast, 10% cited lack of knowledge on methods or sources of contraceptives; 20% cited health concerns. The latter is often connected with the former, as some women overestimate the side effects of contraceptives due to lack of knowledge. For instance, a study in two states in Nigeria found almost one in three women believed that contraceptives can make a woman permanently infertile. Unfortunately, there are countries and regions with major contraceptive stockout issues. However, by strategically picking areas where commodities are available, NGOs in this space can close information gaps and significantly increase contraception use.


II. Our approach: Radio-based campaigns to cost-effectively close information gaps


The model FEM will employ, radio-based SBC campaigns is uniquely poised to close information gaps related to modern contraceptives.

In general, family planning SBC interventions are typically effective in reducing unintended pregnancies and their associated maternal health burdens. SBC interventions seek to understand and facilitate change in behaviors and the social norms and environmental determinants that drive them. In family planning, SBC interventions use multiple channels and take diverse forms, including mass, digital and social media; interpersonal communication; and community mobilization and participation. A review drawing upon 130 studies of SBC interventions found that on average SBC increased use of modern contraceptives 30%-60% (i.e., median odds ratios achieved were 1.3-1.6). However, the cost of these interventions varied widely, with mass media interventions by far being the most efficient in reaching people. In addition, impact achieved varied by geographical location.

Radio-based SBC campaigns are especially well suited for West Africa - where the rates of fertility and unmet need are the highest in the world - including in Nigeria, where we plan to operate. Radio is both the most consumed and trusted form of media in West Africa. In Nigeria, there are more than 200 radio stations reaching a large portion of the 196 million people in the country. These stations allow for targeted geographical outreach, and at the same time they can reach up to millions of unique listeners. About three-quarters of Nigerians rely on the radio weekly for news. Moreover, radio communications are affordable. Based on previous rates obtained from production companies, we estimate that in parts of Nigeria we can reach a potential audience size of 1 million women with a one-minute advertisement for $4-8. Radio’s high reach and low cost suggest the massive potential to close contraceptive related information gaps at scale.

A recently conducted randomized controlled trial on a radio-based SBC campaign in Burkina Faso bears out this potential. From 2016-2018, Development Media International (DMI), a pioneer in conducting mass media SBC interventions, worked with Innovations for Poverty Action to test its campaign in Burkina Faso. DMI broadcasted ten family planning ads per day and three longer interactive programs per week on local radio stations. The results were dramatic. In treatment areas, use of modern contraceptives increased by 5 percentage points (pp), with belief that contraceptives make women infertile dropping 9 pp and belief they cause disease dropping 8 pp.

FEM plans to adapt this proven model to the Nigerian context. In Nigeria, the total fertility level (the average number of children a woman will have in our lifetime) is 5.4. As noted earlier, almost one quarter of sexually active women in Nigeria have an unmet need for family planning. Meanwhile, the Government of Nigeria has demonstrated a strong commitment to family planning. Modern contraceptives are freely available at public clinics and stockouts are relatively infrequent in many states. And as mentioned, radio airtime rates are reasonably priced. Due to the high (and relatively homogeneous) population in the north of Nigeria, we expect to be able to scale up our programs comparatively easily as we progress. In many (though not all) respects, conditions are ideal to execute a radio-based SBC campaign in this environment.



III. Our value-add


FEM plans to extend radio- SBC campaigns to fill in gaps in coverage by existing actors. Radio-SBC campaigns are neglected relative to their potential for impact. DMI, the Johns Hopkins Center for Communications Programs, and BBC Media Action are three notable organizations that implement high quality radio-SBC campaigns. Yet, in many locations with high unmet need for contraceptives and low stockouts, there are little to no efforts to provide family planning information over the radio. After extensive research and helpful conversations with these organisations, we selected as FEM’s launch point a state in Northern Nigeria (Kano) where radio-SBC campaigns are underutilized.


FEM is also different from other family planning organizations in our narrow focus. Many organisations expand into multiple aspects of improving reproductive health, like distribution, advocacy and demand generation. We aim to be hyper focused on providing effective family planning radio campaigns at scale, which we believe is a particularly evidence based, and cost effective intervention within the cause area. In doing so, our plan is to remain a lean organisation with a small effectiveness minded staff.


FEM may also be distinct in that we use common tools from the effective altruism community to make decisions. For example, we often weigh major decisions through a cluster thinking approach that leverages, among other mechanisms, cost-effectiveness analysis, weighted factor models, and expert opinions. We also think closely about counterfactuals before making decisions.

Finally, we believe that family planning could be an underexplored and neglected cause area in the EA community. We believe that it is worthwhile to have EAs exploring this cause area and its potential. We aim to do rigorous evaluations of all our programming and therefore contribute to the literature on the effectiveness of radio SBC family planning programs.



IV. Our progress and plans


FEM was conceived during the Charity Entrepreneurship Incubation Program in July and August and officially launched in September. Since then, we developed our strategy and work plan, selected Kano state in Nigeria as our launch point, selected a media production partner, and built a relationship with a local implementation support partner. We have also interacted with a number of key governmental and NGO and civil society stakeholders within the state who are aware and supportive of our plans.


In the next six months we will conduct small scale operational experiments to test the feasibility of working in our target location and with chosen partners. The first experiment is a one week proof of concept which we aim to have on air within two months. If the proof of concept is successful, and we are able to raise sufficient funding, we will next conduct a longer pilot of our model. While the proof of concept is a feasibility test, we will conduct M&E on our pilot in order to measure early predictors of longer term behaviour change. Depending on results we may pivot our work to a new location or shut down completely. In any case, we will be transparent in our decision-making process and share what we learn.


V. How you can help


There are various ways supporters can help FEM in this make-or-break year. High priority needs include:

  • Technical advice. We seek expert academics and practitioners on family planning, SBC interventions, qualitative research to inform script writing, radio, monitoring and evaluation, and Nigeria to test out our ideas and gather feedback.

  • Volunteering. We’ve identified numerous opportunities for volunteers from anywhere in the world to support us, even if they can dedicate just a few hours of time as advisors.

  • Funding. We are currently seeking to close a $40,000 funding gap to complete baseline data collection and a pilot study. Individual donors may contribute to FEM on our website or on Facebook Fundraisers (we are participating in Giving Tuesday 2020, on December 1st). We also plan to engage more risk-tolerant institutional donors.

We think FEM has a chance to make an important contribution to the family planning space, and be a high impact and efficient organization. We look forward to reviewing any feedback you may have here - and please feel free to reach out to us directly.


Kenneth Scheffler and Anna Christina Thorsheim incubated Family Empowerment Media during the Charity Entrepreneurship (CE) Incubation Program in July and August 2020. CE has provided FEM a $75,000 seed grant. Ken is a former Dalberg and Deloitte consultant, and program manager at USAID. Anna Christina co-founded Fornix, a mental health startup, and led the Effective Altruism chapter of the Norwegian University of Science and Technology.


-----

1. “Need” is inferred based on a whether a woman “wishes to have a child (or another child) soon or ever,” (Guttmacher Institute, “Unmet Need for Contraception in Developing Countries,” Jun. 2016)

2. Based on FEM’s analysis of data provided through the Federal Ministry of Health’s “Family Planning Dashboard,” as of 15 Nov. 2020