Using a Participatory Theory Driven Evaluation Approach to Identify Causal Mechanisms in Innovation Processes
AbstractApplied agricultural research institutes play different roles in complex agricultural innovation processes, contributing to them with other actors. To foster learning and usable knowledge on how research actions influence such lasting innovation processes, there is a need to identify the causal mechanisms linking these actions and the effects of the changes they enable. A participatory, theory ‐driven, ex‐post evaluation method, ImpresS, was developed by the French Agricultural Research Center for International Development (Cirad). ImpresS reconstructs the innovation history and its impact pathway by analyzing ...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Genowefa Blundo ‐Canto,
Agathe Devaux‐Spatarakis,
Syndhia Mathé,
Guy Faure,
Claire Cerdan Tags: Research Article Source Type: research
What Is This Thing Called a Mechanism? Findings From a Review of Realist Evaluations
AbstractRealist evaluation has, over the past two decades, become a widely used approach in evaluation. The cornerstone of realist evaluation is to answer the question: What works, for whom, under what circumstances, and why. This is accomplished by explicating the causal mechanisms that, within a particular context, generate the outcomes of interest. Despite the central role of mechanisms in realist evaluation, systematic knowledge about how the term mechanism is conceptualized and operationalized is limited. The aim of the present chapter is to examine how mechanisms are defined and applied in realist evaluations. Inform...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Sebastian Lemire,
Alexander Kwako,
Steffen B. Nielsen,
Christina A. Christie,
Stewart I. Donaldson,
Frans L. Leeuw Tags: Research Article Source Type: research
Keeping it Real: Using Mechanisms to Promote Use in the Realist Evaluation of the Building Capacity to Use Research Evidence Program
AbstractThis chapter explores the use of mechanisms within the realist evaluation of the Building Capacity to Use Research Evidence (BCURE) program, a £15.7 million initiative aiming to improve the use of evidence in decision‐making in low and middle‐income countries. The evaluation was commissioned to establish not just whether BCURE worked but also how and why capacity building can contribute to increased use of evidence in policymaking in the very different contexts in which the program operated. This chapter argues that using mechanisms helped provide nuanced and robust insights into these questions, while also st...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Melanie Punton,
Isabel Vogel Tags: Research Article Source Type: research
Using Mediation Analysis to Uncover Psychological Mechanisms of Attitude Change in a Development Volunteer Program
AbstractOne aim of development volunteer services is to foster positive attitudes in participants toward those they encounter in their host countries. However, the psychological mechanisms underlying such attitude change remain underexplored. This is true despite intergroup contact theory proposing that contact between groups causes more positive intergroup attitudes through the mechanism of increased mutual knowledge, improved perspective ‐taking, and higher levels of empathy. A theory‐based multi‐method evaluation of the Germanweltw ärts development volunteer service included a quasi ‐experimental component to a...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Martin Bruder,
Jan Tobias Polak,
Kerstin Guffler,
Laura Scheinert Tags: Research Article Source Type: research
No Mechanism Without Context: Strengthening the Analysis of Context in Realist Evaluations Using Causal Loop Diagramming
AbstractRealist evaluation is an approach with a strong emphasis on causal mechanisms and the context in which they are triggered. However, recent reviews of published realist evaluations show that context is often understudied. This is problematic, as a thorough understanding of the relationship between context and causal mechanisms is crucial in assisting policymakers to make appropriate and targeted decisions that improve the intervention. Therefore, we set out to test whether combining realist evaluation with the “systems thinking” approach and, more specifically, causal loop diagramming, could help strengthen the ...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Dimitri Renmans,
Nathalie Holvoet,
Bart Criel Tags: Research Article Source Type: research
Unpacking Mechanisms in Climate Resilient Agriculture Interventions
Discussions (FGD) were conducted with smallholder farmer project beneficiaries. The FGDs revealed what the beneficiaries themselves considered the key mechanisms generated from the intervention, thus forming the bridge between the interventions and outcome. The resu lt was an evaluation design enabling deeper insight into attribution claims. The findings offered novel insights for policymakers about how the climate‐resilient interventions worked for the people themselves and shedding light on the inner workings of the climate‐smart technologies. Finally, th ey provided key stakeholders (commissioning agency and impleme...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Sam Barrett,
Stefano D'Errico,
Simon Anderson,
Bayu Nebsu Tags: Research Article Source Type: research
Leveraging Experimental Evaluations for Understanding Causal Mechanisms
AbstractExperimental evaluations —especially when grounded in theory‐based impact evaluation—can provide insights into the mechanisms that generate program impacts. This chapter details variants of experimental evaluation designs and also analytic strategies that leverage experimental evaluation data to learn about causal mec hanisms. The design variants are poised to illuminate causal mechanisms related to program implementation and the contribution of selected components of multifaceted programs. The analysis strategies lend themselves to illuminating causal mechanisms related to participants’ responses to progra...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Laura R. Peck Tags: Research Article Source Type: research
The Causal Mechanism Claim in Evaluation: Does the Prophecy Fulfill?
AbstractDespite increased discussions in the community and a common understanding about the virtue of mechanism ‐based explanation, little is known about the true benefits and challenges of applying causal mechanism analysis in practice. This chapter aims to introduce the reader to the topic of causal mechanisms and synthesize significant findings on this special issue. It begins by laying out definitions a nd concepts of causal mechanisms in evaluation literature and proposes a two‐way classification of causal mechanisms along which the chapters to this issue are structured. The chapter continues by introducing theCau...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Johannes Schmitt Tags: Research Article Source Type: research
Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms
AbstractThis chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence i...
Source: New Directions for Evaluation - October 6, 2020 Category: Universities & Medical Training Authors: Barbara Befani,
Stefano D'Errico Tags: Research Article Source Type: research
Getting Practical With Causal Mechanisms: The application of Process ‐Tracing Under Real‐World Evaluation Constraints
AbstractOver the past decade, the field of development evaluation has seen a renewed interest in methodological approaches that can answer compelling causal questions about what works, for whom, and why. Development evaluators have notably started to experiment with Bayesian Process Tracing to unpack, test, and enhance their comprehension of causal mechanisms triggered by development interventions. This chapter conveys one such experience of applying Bayesian Process Tracing to the study of citizen engagement interventions within a conditional cash transfer program under real ‐world evaluation conditions. The chapter bui...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Estelle Raimondo Tags: Research Article Source Type: research
Using a Participatory Theory Driven Evaluation Approach to Identify Causal Mechanisms in Innovation Processes
AbstractApplied agricultural research institutes play different roles in complex agricultural innovation processes, contributing to them with other actors. To foster learning and usable knowledge on how research actions influence such lasting innovation processes, there is a need to identify the causal mechanisms linking these actions and the effects of the changes they enable. A participatory, theory ‐driven, ex‐post evaluation method, ImpresS, was developed by the French Agricultural Research Center for International Development (Cirad). ImpresS reconstructs the innovation history and its impact pathway by analyzing ...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Genowefa Blundo ‐Canto,
Agathe Devaux‐Spatarakis,
Syndhia Mathé,
Guy Faure,
Claire Cerdan Tags: Research Article Source Type: research
What Is This Thing Called a Mechanism? Findings From a Review of Realist Evaluations
AbstractRealist evaluation has, over the past two decades, become a widely used approach in evaluation. The cornerstone of realist evaluation is to answer the question: What works, for whom, under what circumstances, and why. This is accomplished by explicating the causal mechanisms that, within a particular context, generate the outcomes of interest. Despite the central role of mechanisms in realist evaluation, systematic knowledge about how the term mechanism is conceptualized and operationalized is limited. The aim of the present chapter is to examine how mechanisms are defined and applied in realist evaluations. Inform...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Sebastian Lemire,
Alexander Kwako,
Steffen B. Nielsen,
Christina A. Christie,
Stewart I. Donaldson,
Frans L. Leeuw Tags: Research Article Source Type: research
Keeping it Real: Using Mechanisms to Promote Use in the Realist Evaluation of the Building Capacity to Use Research Evidence Program
AbstractThis chapter explores the use of mechanisms within the realist evaluation of the Building Capacity to Use Research Evidence (BCURE) program, a £15.7 million initiative aiming to improve the use of evidence in decision‐making in low and middle‐income countries. The evaluation was commissioned to establish not just whether BCURE worked but also how and why capacity building can contribute to increased use of evidence in policymaking in the very different contexts in which the program operated. This chapter argues that using mechanisms helped provide nuanced and robust insights into these questions, while also st...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Melanie Punton,
Isabel Vogel Tags: Research Article Source Type: research
Using Mediation Analysis to Uncover Psychological Mechanisms of Attitude Change in a Development Volunteer Program
AbstractOne aim of development volunteer services is to foster positive attitudes in participants toward those they encounter in their host countries. However, the psychological mechanisms underlying such attitude change remain underexplored. This is true despite intergroup contact theory proposing that contact between groups causes more positive intergroup attitudes through the mechanism of increased mutual knowledge, improved perspective ‐taking, and higher levels of empathy. A theory‐based multi‐method evaluation of the Germanweltw ärts development volunteer service included a quasi ‐experimental component to a...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Martin Bruder,
Jan Tobias Polak,
Kerstin Guffler,
Laura Scheinert Tags: Research Article Source Type: research
Editor's Note
New Directions for Evaluation, Volume 2020, Issue 167, Page 7-10, Fall 2020. (Source: New Directions for Evaluation)
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Johannes Schmitt Tags: Editors' Notes Source Type: research
