by Todd C. Helmus and Elizabeth Bodine-Baron
Government efforts to counter the propaganda and radicalization that lead to violent extremism are becoming more common around the world, but there's little research on whether such programs work. Funded by the Global Engagement Center at the U.S. Department of State, RAND conducted three randomized controlled trials—the gold standard in evaluation design—of what are called countering violent extremism (CVE) interventions, using radio and social media in Nigeria, Indonesia, and the Philippines. RAND also conducted a qualitative assessment of a program designed to train civil society members in the Philippines to directly counter violent extremism.
The results were mixed, but one conclusion was inescapable: countering violent extremism is not an easy task, and programmers should not always assume their content will be successful.
Nigeria
In communities in northern Nigeria affected by violent extremism, RAND recruited 2,064 participants via SMS/text message and assigned them randomly to listen to a CVE-themed radio talk show called Ina Mafita or to a control program (professional soccer matches) each week over two months. To monitor compliance, we texted participants a weekly quiz and correct answers earned a small financial incentive. Baseline and monthly surveys also were also delivered via SMS.
Ina Mafita had a positive effect on listeners' beliefs about the importance of being a role model and a positive but not significant effect on the belief in the value of locally run initiatives to reintegrate at-risk youth. Results were more pronounced for those in the treatment group who complied well and for those who reported liking the show's story line.
Indonesia
In Indonesia, RAND recruited more than 900 Indonesian youth via Facebook and randomly assigned them to systematically observe CVE-themed Facebook posts or control content (posts derived from advertising and public service campaigns) for two months. The campaign's goals were to increase support for diverse, inclusive communities and freedom of speech and to decrease support for the use of violence in addressing grievances. Surveys were conducted via an online portal.
The survey found a positive effect on attitudes toward promoting inclusivity online. But the result stemmed from an unusual and sudden drop in such attitudes among the control group. Also, the effect was no longer significant in analyses that controlled for baseline differences among study participants. Moreover, there were negative treatment effects on respondents' attitudes toward living in separated communities, or what is known as a boomerang effect—the unintended consequence of an attempt to persuade that results in the adoption of an opposing position instead.
Social media still offers a promising platform for CVE messaging, particularly when targeted at young people who use social media as an important source of news and information. However, creators will need to continue to tease out which messages are most effective.
Philippines
In the Philippines study, RAND recruited 1,200 participants in Mindanao and randomly assigned them to listen to Saranggola, a 48-episode CVE radio show that seeks to promote women's empowerment and support of and participation in good governance—or to a control condition (locally aired basketball matches). Afterward we surveyed participants about the program's intended themes.
Ultimately, a series of challenges undermined confidence in the study results. Gender differences between groups revealed selection bias. A series of earthquakes that shook Mindanao in the study's initial weeks affected several data integrity measures. And although analyses of attitudinal and knowledge outcomes were positive, the effects were the same in both the treatment and control groups.
In the Philippines we also conducted in-depth interviews with participants in a “Peace Promotion Fellowship” program that helped local representatives of civil society design and run their own CVE programs. The report offers recommendations to improve future programming.
The Potential and Challenge of Rigorous Impact Evaluations
These studies suggest both the potential and the challenge of rigorous impact testing of CVE interventions. In terms of potential, the Nigeria and Indonesia studies successfully recruited a large sample of participants using non-traditional methods of data collection—text message and Facebook advertisements—and retained a significant share of the participants throughout the study. Both also showed that small financial incentives could motivate participants to listen to or view the CVE content routinely, thereby ensuring maximum exposure to the treatment.
Some of the challenges, such as the earthquakes in Mindanao, were outside researcher control; but some were not. The Indonesia study suggests that content creators could use pretesting to improve the likelihood of a positive effect on the target audience. Future impact evaluations of online content could also use alternative study designs to mimic more realistic exposure to campaign content. The Philippines study shows that greater oversight of data collection may have helped address the selection bias problem and that making the study logistically simpler—and less ambitious—might have helped. In particular, it may have been advisable to perform more qualitative testing as was conducted with the Peace Promotion Fellowship program.
Where CVE Intervention Research Goes from Here
The demonstrated effectiveness of Ina Mafita suggests that policymakers should continue to support radio programming that is designed to promote peace and address root causes of violence. But because overall, results of CVE interventions were mixed, it is critical to conduct more research to tease out which programs are most effective.
To start, researchers should supplement rigorous impact evaluations with qualitative process evaluations that take advantage of focus groups and other audience insights. The programs themselves should further follow best practices for program development and draw on formative research, pretesting of content, and other practices routinely advised by communication experts.
Todd C. Helmus is a senior behavioral scientist and Elizabeth Bodine Baron is an information scientist specializing in complex networks and systems at the nonprofit, nonpartisan RAND Corporation.
This research was funded by the U.S. State Department's Global Engagement Center, which Congress has charged with leading and coordinating government efforts to counter propaganda and disinformation from foreign countries and international terrorist organizations. The research informs GEC activities, including communications work with international partners such as the 82-member Global Coalition to Defeat ISIS.
Commentary gives RAND researchers a platform to convey insights based on their professional expertise and often on their peer-reviewed research and analysis.
No comments:
Post a Comment