No Mechanism Without Context: Strengthening the Analysis of Context in Realist Evaluations Using Causal Loop Diagramming
AbstractRealist evaluation is an approach with a strong emphasis on causal mechanisms and the context in which they are triggered. However, recent reviews of published realist evaluations show that context is often understudied. This is problematic, as a thorough understanding of the relationship between context and causal mechanisms is crucial in assisting policymakers to make appropriate and targeted decisions that improve the intervention. Therefore, we set out to test whether combining realist evaluation with the “systems thinking” approach and, more specifically, causal loop diagramming, could help strengthen the ...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Dimitri Renmans,
Nathalie Holvoet,
Bart Criel Tags: Research Article Source Type: research
Unpacking Mechanisms in Climate Resilient Agriculture Interventions
Discussions (FGD) were conducted with smallholder farmer project beneficiaries. The FGDs revealed what the beneficiaries themselves considered the key mechanisms generated from the intervention, thus forming the bridge between the interventions and outcome. The resu lt was an evaluation design enabling deeper insight into attribution claims. The findings offered novel insights for policymakers about how the climate‐resilient interventions worked for the people themselves and shedding light on the inner workings of the climate‐smart technologies. Finally, th ey provided key stakeholders (commissioning agency and impleme...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Sam Barrett,
Stefano D'Errico,
Simon Anderson,
Bayu Nebsu Tags: Research Article Source Type: research
Leveraging Experimental Evaluations for Understanding Causal Mechanisms
AbstractExperimental evaluations —especially when grounded in theory‐based impact evaluation—can provide insights into the mechanisms that generate program impacts. This chapter details variants of experimental evaluation designs and also analytic strategies that leverage experimental evaluation data to learn about causal mec hanisms. The design variants are poised to illuminate causal mechanisms related to program implementation and the contribution of selected components of multifaceted programs. The analysis strategies lend themselves to illuminating causal mechanisms related to participants’ responses to progra...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Laura R. Peck Tags: Research Article Source Type: research
The Causal Mechanism Claim in Evaluation: Does the Prophecy Fulfill?
AbstractDespite increased discussions in the community and a common understanding about the virtue of mechanism ‐based explanation, little is known about the true benefits and challenges of applying causal mechanism analysis in practice. This chapter aims to introduce the reader to the topic of causal mechanisms and synthesize significant findings on this special issue. It begins by laying out definitions a nd concepts of causal mechanisms in evaluation literature and proposes a two‐way classification of causal mechanisms along which the chapters to this issue are structured. The chapter continues by introducing theCau...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Johannes Schmitt Tags: Research Article Source Type: research
Letting Evidence Speak for Itself: Measuring Confidence in Mechanisms
AbstractThis chapter argues that the credibility of causal mechanisms can be greatly increased by formulating them as statements that are both empirically falsifiable and empirically confirmable. Whether statements can be so depends on the potential availability of the relevant evidence (e.g., no evidence exists that can prove or disprove the existence of God, but good quality evidence is potentially available in many other cases). The Bayes formula can be used to measure the extent to which a given set of empirical observations supports or weakens the belief that a causal mechanism exists. With this approach, confidence i...
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Authors: Barbara Befani,
Stefano D'Errico Tags: Research Article Source Type: research
Issue Information
New Directions for Evaluation, Volume 2020, Issue 167, Page 1-6, Fall 2020. (Source: New Directions for Evaluation)
Source: New Directions for Evaluation - October 5, 2020 Category: Universities & Medical Training Tags: Issue Information Source Type: research
Radical Inquiry —Liberatory Praxis for Research and Evaluation
AbstractConventional social science research and evaluation too often replicate unjust and oppressive narratives and assumptions about young people of color's capacities, abilities, and needs, treating them primarily, and sometimes solely, as risk, problem, or disease. Overemphasis on “metrics of compliance,” such as personal behavior change, self‐efficacy, and resilience perpetuate this burden while ignoring those of survival, fortitude, and resistance in the context of structural/historical subjugation, discrimination, and state‐sanctioned violence communities of color have endured as part of US nation‐building...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Kanwarpal Dhaliwal,
Jill Casey,
Kimberly Aceves ‐Iñiguez,
Jara Dean‐Coffey Tags: Research Article Source Type: research
Blest Be the Tie That Binds
AbstractThis chapter explores the relationship between program evaluation and social change through the lens of key historical periods in the development of the field. Also profiled is the relationship between the practice of program evaluation by evaluators, and the communities or contexts in which the evaluand is situated. And finally, the chapter explores the strong bonds that support group consciousness and a desire for change. “Oppositional Consciousness,” a sociological construct, serves as a heuristic that may be useful for program evaluators seeking to promote, support, or inspire social change. (Source: New Di...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Melvin E. Hall Tags: Research Article Source Type: research
The Black Perspective: A Framework for Culturally Competent Health Related Evaluations for African Americans
AbstractThis chapter presents the Black Perspective Health Evaluation (BPHE) framework as a cultural lens for conducting evaluations with Federally Qualified Health Centers (FHQC). The consideration of the health encounter as a cultural encounter is central. The evaluator has a role in advocating for social justice and igniting change in health outcomes for marginalized and vulnerable populations served by FHQCs. The six principles of the Howard University School of Social Work's Black Perspective: (1) affirmation, (2) strengths, (3) vivification, (4) diversity, (5) internationalization, and (6) social justice are featured...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Tamarah Moss,
Sandra Edmonds Crewe Tags: Research Article Source Type: research
A Language Justice Framework for Culturally Responsive and Equitable Evaluation
AbstractEvaluation interconnects with myriad social needs and must be responsive to diverse, intersecting identities and experiences, including language. Over 25 million people in the United States speak English less than “very well,” according to the U.S. Census. Many are from marginalized communities that face multiple forms of oppression, including racism, xenophobia, and linguicism. We propose an evaluation framework grounded in language justice—simply defined as the right to communicate in the language in which one feels most comfortable. This right is threatened by structural inequities that include unexamined ...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Susan Ghanbarpour,
Ana Paula Noguez Mercado,
Ada Palotai Tags: Research Article Source Type: research
Nation ‐to‐Nation in Evaluation: Utilizing an Indigenous Evaluation Model to Frame Systems and Government Evaluations
AbstractEvaluation scholars have offered culturally responsive evaluation theories, methods, and frameworks, but few have applied them to systems or governance evaluations. Culturally responsive and systems evaluation literature does not address the unique legal and political components of sovereign Tribal/First Nations Governments. This chapter addresses literature and practice gaps through an emerging Nation ‐to‐Nation (N2N) Systems Evaluation Framework. Applying Tribal Critical Theory (TCT) to systems and governance evaluations, the author builds on an emerging Tribal Critical Systems Theory (TCST) to consider futur...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Nicole Bowman Tags: Research Article Source Type: research
Promoting Culturally Responsive and Equitable Evaluation With Latinx Immigrants
AbstractCulturally Responsive and Equitable Evaluation requires evaluators to understand the social and intersectional identity experienced by Hispanic/Latino/Latinx communities, inclusive of immigrants in the United States. This chapter introduces relevant issues for evaluators to consider when working with Latinx immigrants. Evaluators must simultaneously address biases that people have toward Latinx immigrants; heed intergroup diversity among Latino immigrants, and increase Latinx immigrants' knowledge about their existing environment and their ability to advocate for themselves. Likewise, they must advocate for change ...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Diana Lemos,
David Garcia Tags: Research Article Source Type: research
Issue Information
New Directions for Evaluation, Volume 2020, Issue 166, Page 1-6, Summer 2020. (Source: New Directions for Evaluation)
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Tags: Issue Information Source Type: research
LatCrit Theory as a Framework for Social Justice Evaluation: Considerations for Evaluation and Evaluators
AbstractThis chapter introduces the use of Latino Critical Race Theory (LatCrit) as a framework for inciting change through Social Justice Evaluation (Mertens& Wilson, 2012; Neubauer, 2015, 2017). Informed by the foundational work of Valdes (1997) and Solorzano and Yosso (2001), the authors suggest that LatCrit informed evaluation is a transdisciplinary process that links theory with practice, scholarship with evaluation learnings, and our institutions with community, action, and change. The chapter provides an overview of the theory, synthesizes related literature, and describes a guiding framework anchored in four functi...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Andrea D. Guajardo,
Grisel M. Robles ‐Schrader,
Lisa Aponte‐Soto,
Leah C. Neubauer Tags: Research Article Source Type: research
Is Inciting Social Change Something Evaluators Can Do? Should Do?
AbstractThis volume set out to consider what must occur for evaluators and evaluation to incite social justice ‐oriented change in the communities where there is a need. It takes an affirmative posture and, in each chapter, builds a case for engaging in social change. In a variety of contexts, exploration of the social consciousness of evaluators and the larger evaluation profession, has led to the encoura ging possibility that promoting oppositional consciousness could more reliably lead to the desired change. However, some argue that moving evaluation in this direction is a mistake and goes too far. This chapter examin...
Source: New Directions for Evaluation - July 9, 2020 Category: Universities & Medical Training Authors: Leah C. Neubauer,
Melvin E. Hall Tags: Research Article Source Type: research
