Impact Evaluation Toolkit
A fundamental question for development practitioners to ask is: did our program work? That is, did our program have an effect on a specific outcome, and if so, what is the magnitude of the effect? Answering this question credibly requires rigorous impact evaluation (IE) studies that identify the changes in outcomes that occur as a direct result of the program. Evidence generated by such studies enables both learning and accountability, and can increase the capacity for evidence-informed decision making. Ultimately, by expanding the state of knowledge on what development solutions work, what don’t, and why, policy makers have greater scope to combat poverty through the design of effective social programs.
However, impact evaluations are a costly and complex tool that cannot answer all questions asked by every stakeholder. As such, being able to evaluate the right questions, at the right time, and in a rigorous manner is essential. The Impact Evaluability Toolkit (IET, hereafter) offers a comprehensive framework for formulating an impact evaluation strategy. The objective of the IET is to guide users on the Selection Process, which is defined as the steps required to shortlist and ultimately select the evaluations that the user decides to commission for an evaluation. The Selection Process involves (1) setting an impact evaluation agenda, and (2) identifying the technical requirements necessary for an impact evaluation to make a credible, causal claim. Such information is relevant to a diverse group of actors including governments, NGOs, academics, donors, and any other organization that endeavors to carry out purposeful, strategic, and informative impact evaluations.
Section 1 of the IE Tool, “What to Evaluate”, discusses the process of characterizing the usefulness of an evaluation, defines criteria for when to do an evaluation, and sets out steps to build a theory of change. Section 2, “How to Evaluate”, presents a technical overview of impact evaluation design to help users determine the feasibility of evaluations, and also includes budgetary considerations to factor into decision making as well as steps on how to manage and commission an impact evaluation. The Appendix provides useful resources on generating an Impact Evalubility Activity Assessment, crafting a theory of change, finding the right methodology to use in an evaluation, and drafting terms of reference for evaluating agencies.