8 December 2021

What Afghanistan Teaches Us About Evidence-Based Policy

Corinne Graff, Ph.D.

Even as the debate over the lessons learned by the U.S. government in Afghanistan continues, several clear conclusions have emerged. One is that U.S. agencies repeatedly underestimated the time and resources needed to support a nation wracked by decades of war, while they failed to follow a consistent plan for civilian recovery efforts. U.S. personnel also lacked the training needed to be successful in the field, and monitoring and evaluation efforts did not receive the policy attention required to enable course corrections and learning.

Local Afghan police in Kakeran, Afghanistan, Feb. 18, 2013. While some militias the U.S. helped create to protect areas from the Taliban are operating as hoped, others have a reputation for abuse and banditry. (Bryan Denton/The New York Times)

While these gaps certainly all had a role in undermining efforts to stabilize Afghanistan over the past two decades, one finding stands out not only because it has been repeatedly identified as an obstacle to U.S. policy in Afghanistan and elsewhere, but because it would be relatively easy — and inexpensive — to fix: the U.S. government’s poor understanding of the conflict environments in which it operates.

Power Structures, Patronage and Violence

The nature of the problem is described in a recent report by the Special Inspector General for Afghanistan (SIGAR), a government watchdog established in 2008 to audit and investigate U.S. programs. It finds that government agencies lacked accurate data on the operating environment in Afghanistan, including the terrain and climate. Buildings were constructed in colder regions without due consideration to the potential for heavy snow loads on roofs, or without consulting topographic maps. To some extent, these types of information gaps are understandable, given the dangerous conditions on the ground, particularly as violence increased over the past decade. Policymakers lacked access to critical sources of information, no doubt an unavoidable situation and part of the so-called fog of war.

But as the report makes clear, the biggest blind spots in Afghanistan were not failures of historical or cultural knowledge or intelligence collection, but of conflict analysis: U.S. policymakers failed to take into consideration the political and economic dynamics driving these conflicts, and to anticipate the impact U.S. interventions would have on those dynamics. “The United States seems to have misunderstood the dynamics of political power in Afghanistan, particularly the role of patronage networks, which were born of several decades of armed conflict and had become entrenched in the country’s political economy,” the SIGAR report noted. This type of analytical gap is not specific to U.S. engagement in Afghanistan, but affects U.S. aid to countries that experience conflict around the world.

As a result of such analytical blinders, U.S. projects too often inadvertently provided support to — or withdrew resources from — the wrong power-brokers, upsetting local power dynamics and fueling further conflict. In Afghanistan, state-owned enterprises were privatized without sufficient attention to the power and patronage networks that hold sway over Afghanistan, fueling corruption and community grievances. Foreign advisers drafted laws that conflicted with local traditions and lacked broad acceptance, incentivizing entire communities to turn to the Taliban for more familiar solutions. U.S. security cooperation and supply chains enriched and fueled violent competition between elites, further alienating the citizens it was meant to help protect.

While it should be expected that the underlying elite power structures and backroom deals through which traditional societies are informally governed are opaque, there is a growing body of research that zeroes in on these conflict dynamics and is assessing their policy implications. Political settlement analysis — not to be confused with the formal “peace settlements” that end wars — is a well-established field of study. It examines how rival elites inside and outside governments exercise political power through transactional political deals among themselves. In countries where formal government institutions and the social contract between elites and citizens are weak or nonexistent, systems of elite bargaining drive decision-making — and organized violence. Political settlements can be more or less exclusive and involve more or less violence and corruption driven by rent-seeking elites. The task for policymakers seeking to resolve conflicts is to gradually transform these arrangements so they become more inclusive, representative of minority groups, accountable and, gradually, more stable.

Political settlement analysis prompts policymakers to look beyond the motives of political leaders and performance of formal institutions, and to seek to understand the system of political power within which elites operate and policy is made. It starts with questions such as:
Who are the key national and sub-national elites and what role do formal and informal institutions play in shaping their actions and decisions?

From where do they derive their power, and how are they wielding this power over different groups?

What is the role and impact of external policy and programs on elite bargaining processes, and what can donors do to make the political settlement more inclusive and stable?

This type of analysis could help the United States achieve better results in the many conflict-affected countries where it continues to engage. So far, however, this type of innovative research has not been incorporated into U.S. policy and doctrine.

Upgrading the U.S. Toolbox

Despite the urgings of individual advisers who have been strong advocates for it, this type of analysis has had limited impact on U.S. policy. One problem is that there are of course many barriers to ensuring that good evidence informs public policy — a challenge that is not specific to conflict prevention and stabilization policy. Yet clearly there are also important analytical gaps withing the U.S. government when it comes to engaging in fragile environments. Recent U.S. stabilization and conflict prevention doctrine reflects the need to focus on the “political roots” of conflict and on “improving governance” to stabilize communities, but doesn’t address the power structures and networks that are at the center of conflicts, as SIGAR found in Afghanistan. For the most part U.S. doctrine continues to ignore the central role of political settlements in driving conflict and constraining policy and reform.

This is perhaps not surprising since U.S. government agencies are noticeably absent from the publicly disclosed institutional funders that support cutting-edge research on conflict and on the impact of relevant policy interventions, including at MIT, the London School of Economics and Oxford University. This is not to say that advances have not been made across the U.S. government, but existing analytical tools are overdue for an upgrade. In some quarters of the U.S. government, technical analysis is being done to better understand these political-economy dynamics, but it is not focused on conflict and unclear how much impact this analysis is having on aid programs, let alone U.S. policy.

By contrast, other donors are leveraging the latest findings from this research. Political settlements analysis is heavily funded by the UK government, as is research on the related concept of political marketplaces. Based on this line of work, other donors have developed entire policy frameworks that shed new light on elite decisions and bargains in these contexts — as well as the breakdowns that lead to devastating violence and commensurate policy responses. This work includes policy recommendations for supporting pathways out of conflict, taking into account the interests and positions of elites. Although each conflict setting is different, research on political settlements tends to emphasize the following policy priorities: participatory conflict analysis and needs assessment; community-based accountability mechanisms and programs; conflict-sensitive humanitarian and development aid; forging coalitions for security sector reform; and developing accountable state capacity for taxation and budgeting.

While there are no easy solutions when it comes to understanding and navigating complex political and economic dynamics in distant countries, U.S. agencies must update their tools to analyze conflict-prone societies based on the latest available research.
Expand Research Partnerships

Several measures could help remedy these analytical gaps in fragile environments. Research partnerships have been repeatedly identified as one of the best ways to increase the use of evidence by policymakers. To ensure that strategy, policy and program decisions in conflict countries are informed by cutting edge political-economy analysis, the State Department and USAID could seek opportunities to share best-practices on analytic and assessment tools with other leading donor agencies. State and USAID as well as intelligence agencies should also explore relationships with research institutions, perhaps modeled on the research and development strategy that the U.S. government has developed in the field of global health, or the research partnerships it has established in the agriculture and health sectors. Conflict and violence prevention and stabilization should also feature prominently in USAID’s Research Technical Assistance Center consortium. It is critical that agencies and U.S. missions around the world also invest in research platforms that are inclusive of local researchers in fragile states, such as the Resolve Network. Agencies could also bring leading experts into government agencies through short-term details to develop new training programs for diplomats and program and policy guidance based on cutting-edge conflict research.
Build the U.S. Government’s Evidence Base on Conflict

Secondly and relatedly, agencies should update their monitoring and evaluation (M&E) and learning systems to ensure they are positioned to adapt policies and programs in response to fast-evolving political dynamics in fragile states. Research on flexible and adaptive M&E mechanisms, including a recent project funded by the UK government at the Graduate Institute Geneva, builds on political settlement analysis and is developing practical analytical tools for field-based staff. More broadly, State and USAID should seize on the Evidence Act and USAID Administrator Samantha Power’s commitment to evidence-based policy to ensure they are prioritizing evidence in conflict settings. For example, the new Office of Behavioral Science and Experimental Economics at USAID should include a strong team of political economists and conflict experts. USAID should also prioritize partnerships with policy innovation labs that run quasi-experimental research projects on the impacts of stabilization and conflict prevention policies, such as the Abdul Latif Jameel Poverty Action Lab.

Staff up for Success

Finally, the bottom line is that no progress will be possible without additional resources to improve the U.S. government’s analytical capabilities in conflict settings. As the latest SIGAR report makes clear, it is unrealistic to expect agencies to improve their own capabilities if only 5 percent of their budget can be invested in such reforms, the spending cap provided under the recent Global Fragility Act (GFA). Conducting context-specific local analysis is staff-intensive. Congress should support USAID’s request for funds to ensure the agency has the staffing posture it needs to improve its evidence base and hire experts in the field, including by lifting the GFA spending cap, if necessary.

As we continue to reflect on the lessons from Afghanistan, it will be important to identify achievable reforms that can help the U.S. government improve the outcomes of its policies in fragile states around the world. Improving the USG’s analytical capacities to develop a better understanding of conflict environments is low hanging fruit. We should seize it.

No comments: