Training Course on Project Impact Evaluation-Rwanda

About the Course

All project and programmes are aimed at creating change to the target beneficiaries. Whether or not these changes are actually achieved is a crucial question but one that is not often examined. Impact evaluation assesses the changes that can be attributed to a particular intervention, both the intended ones, as well as the unintended ones. This course is designed to equip the participants with adequate skills to conduct rigorous impact evaluations for their interventions. The course covers all aspects of planning, designing and implementing a successful impact evaluation.

Target Participants

This training course is recommended for researchers, project staff, development practitioners, consultants and decision makers. It is ideal for those who wish to be at par with the latest practices in determining the impact of project interventions.

What you will learn

By the end of the course the learner should be able to:

• Understand why impact evaluation is important in projects and programmes
• Learn about theories and practices of impact evaluation
• Learn about evaluation problem: attribution, selection and placement biases
• Understand IE designs: randomized, quasi and non-experimental
• Conduct a rigorous impact evaluation of a project, programme or a policy
• Learn and practice on sampling, power calculation, econometric techniques for impact evaluation such as Propensity Matching, Difference-in –Difference and many more.
• Report and disseminate impact evaluation findings

Course duration

5 days

Course Outline

Fundamentals for Impact Evaluation

• Results Framework
• M&E Project Indicators
• Logical Framework
• Performance Evaluation

Introduction to Impact Evaluation

• Defining impact evaluation
• Deciding whether to evaluate
• Attribution in impact evaluation
• Contribution vs attribution

Determining points of Impact Evaluation

• Types of evaluation questions
• Theories of Change
• The results chain
• Evaluation hypothesis
• Impact evaluation indicators

Counterfactuals in Impact Evaluation

• Causal reference
• Comparison groups and counterfactuals
• Estimating counterfactual in impact evaluation
• Impact Evaluation when a Comparison and Baseline Data are not available

Impact Evaluation Research Designs

• Experimental Design (Randomized Control Trials)
• Quasi experimental designs (Regression Discontinuity; Difference-in-Difference; Matching)
• Non-experimental designs

Sampling in Impact Evaluation

• Sampling techniques in Impact Evaluation
• Sample size calculation,
• Sampling weight
• Statistical power
• Constructing valid comparison groups

Quantitative Data Analysis for Impact Evaluation

Randomized Impact Evaluation

• Setting the counterfactual
• Statistical Design of Randomization
• Impacts of Program Placement
• Impacts of Program Participation
• Capturing Both Program Placement and Participation
• Measuring Spillover Effects

Propensity Score Matching Technique (PSM)

• Introduction to PSM
• Application of PSM in Impact Evaluation
• Constructing propensity scores
• Matching Analysis (Nearest-Neighbour, Stratification, Radius, Kernel )
• Common support
• Doubly Robust Estimation

Difference in Difference Method

• What is DID?
• Application of DID in Impact Evaluation
• Estimating DID estimator: Single DID, DID with covariates, Quantile DID
• DID for Cross-Sectional Data
• Balancing Tests
• Diff-in-Diff with Propensity Score Matching

Qualitative Approaches in Impact Evaluation

• Most Significant Change
• Outcome Mapping
• Outcome Harvesting
• Appreciative Inquiry

Producing and Disseminating Impact Evaluation findings

• Impact Evaluation Plan
• Impact Evaluation Reports: Baseline report, Impact Evaluation Reports
• Case studies