Impact Evaluation Technical Course 2016

SHIPDETHow to Manage, Design, and Conduct Impact Evaluations
Aug 1 - 12, 2016

CLEAR-SHIPDET Center
The Asia-Pacific Finance and Development Institute (AFDI) 
Shanghai, China 

 

s

Course Overview

This technical course introduces impact evaluation as a key instrument for determining project/program effectiveness, informing policy development and improving program designs. It covers commonly used econometric and statistical methods to evaluate the impacts of social and other programs in developing countries. The course includes both randomized and non-randomized methods. We will also cover other practical aspects, such as managing an impact evaluation team, planning and implementing data collection, and disseminating the results of the evaluation.

Delivery of the course consists of lectures, class exercises, and group work. Many of the exercises will involve case studies using data from actual programs that are discussed in the lectures. Participants will also have an opportunity to determine options for evaluations for their own programs.

This course is highly technical. In order to benefit from the course, participants should review the prerequisites before applying. Please send any questions about the technical nature of the course to the course organizers to determine if this is a good fit for you. Though the course is technical, it will also address policy implications and practical applications.


Course Objectives

At the end of the course participants will be able to:

  • Understand what impact evaluation is and the different techniques used for impact evaluations

  • Make judgments about the situations (and/or interventions) when impact evaluation is and is not appropriate to use

  • Know which kind of evaluation or decision-making questions that impact evaluation is best suited to answer

  • Establish a management plan for having impact evaluations conducted

  • Review the “parameters of interest” (ATE-average treatment effect, ATT-average impact of treatment on the treated, ITT-intention-to-treat, etc.)

  • Describe in detail how randomized trials can estimate parameters of interest

  • Explain how to use regression methods that do not require randomized data (Difference-in-differences and within estimators, Instrumental variable methods, Regression discontinuity methods, Matching methods)

  • Present recommendations for designing high quality questionnaires

  • Provide advice on how to choose the sample size and attain statistical power Discuss the main issues concerning data collection and data management


Instructors

g Dr. Paul Glewwe,Professor, Department of Applied Economics, University of Minnesota
Paul is a Professor of Applied Economics at the University of Minnesota. His interests are economics of education, poverty and inequality in developing countries, and applied econometrics. Since receiving his Ph.D., he has worked in the following countries: Cambodia, China, Cote d'Ivoire, Ghana, Honduras, Jamaica, Jordan, Kenya, Laos, Malaysia, Morocco, Philippines, Sri Lanka, Turkey, Viet Nam and Peru. His recent publications have appeared in the Handbook of the Economics of Education, Economic Development and Cultural Change, Journal of Development Economics, Journal of Economic Literature, Journal of Political Economy, Handbook of Development Economics, Journal of Public Economics and World Bank Economic Review.
 
t Petra ToddProfessor, Department of Economics, University of Pennsylvania
Petra is also a Research Associate of Penn’s Population Studies Center. She serves as an Associate Editor for the American Economic Review and the Journal of Human Capital. Her main fields of research are social program evaluation, labor economics, and microeconometrics. She has published papers on the determinants of cognitive achievement, testing for discrimination in motor vehicle searches, sources of racial wage disparities, and methods for evaluating and optimally designing conditional cash transfer programs. She is currently working on implementing a large-scale randomized school incentive program in Mexican high schools, on analyzing the effects of school vouchers in Chile and on assessing the effects of government regulation on the operation of the privatized pension market in Chile.
 
c Dr. Qihui Chen,Associate Professor, College of Economics & Management, China Agricultural University
Qihui is an Associate Professor of College of Economics & Management at China Agricultural University. He received his Ph.D. degree in Applied Economics from the University of Minnesota in 2012. He also holds a B.S. degree in Urban and Regional Planning and a M.A. degree in Regional Economics, both from Peking University. Dr. Chen's research interests include applied microeconomics, human capital, development, and program evaluation. He examines education, health and welfare issues in developing countries, especially those related to the intergenerational transmission of human capital. In particular, he applies econometric methods to study the causal effects operated from parental resources and family configuration to children’s educational and health outcomes.
 

Course Prerequisites

To fully benefit from the course, participants need to be familiar with statistics, multiple regression analysis and possess some knowledge of a statistical software package (e.g., STATA, SAS, SPSS, or R). Additionally, knowledge of econometrics is highly beneficial.

  • All participants applying for the course need to complete the application form.

  • Priority will be given to teams (2 or more individuals) that have impact evaluations underway, recently completed and/or are considering, designing, or implementing impact evaluations. Members of the team may include technical staff, as well as policymakers, policy analysts, and program and evaluation managers who have the required technical skills to benefit from the course. Each individual will have to apply separately, but should reference the team in the application form.

  • All participants must be proficient in English.


Applications

Applications will be considered as they are submitted. If you are certain that you want to take this course please submit your application earlier than the deadline. Please note that submitting an application does not mean automatic enrollment in the course. Applicants will be notified of their enrollment status within one week following the application deadline. We will acknowledge receipt of your application within two business days of your submission. If you do not receive a note from us indicating that we received your application, please contact us with Ms. Amy Chen (amychen@afdi.org.cn) or Ms. Annie Wu (wuningqin@afdi.org.cn).

s

Application Procedures

We have a two-step process.

Step 1: Complete the application form (Online applications are required, but you may see the full application by clicking this PDF document).

If you would like to see what questions are asked in the application, please click here. But please submit your information using the web-based application/quiz.

Step 2: Next, via e-mail send documents listed below to Ms. Amy Chen (amychen@afdi.org.cn) or Ms. Annie Wu (wuningqin@afdi.org.cn).

  • Resume or CV
  • Optional, additional materials related to the impact evaluation/s that you are currently envisioning or implementing now (e.g., concept notes, proposals, evaluation designs, evaluation reports, etc.)

When saving your files, please use the following naming convention for the files that you will be submitting as part of the application package:

  • IECourse_China_FileName_YOURNAME (For example, IECourse_China_Resume_YOURNAME, IECourse_China_AdditionalMaterials_YOURNAME etc.)


Cost

Apply early to save! For applicants who apply on or before May 31, the organizers will offer an additional Early Bird Discount of $300.

The base price for the course fee is $3,300 USD. The bulk of the fee is for (almost) two weeks of housing and tuition. Discounts will be given to participants from certain organizations, as noted below. Payment policies and procedures will be provided to selected applicants.

Base Fee

Discount

Total

Applies to Participants from

$3,300

$0

$3,300
($3,000 if registering by May 31)

International organizations (e.g., multi-lateral banks, foundations, international NGOs), bi-lateral agencies from developed countries, the private sector, and others

$3,300

-$1,000

$2,300
($2,000 if registering by May 31)

*Government sector employees in developing countries, local NGOs

*Note: the organizers reserve the right to determine whether the discount applies to each applicant seeking a discounted rate.

Included in the fee:

  • Tuition

  • Training booklets

  • Housing at the SNAI residential facility from July 31 (afternoon) - to August 13 (morning), 2016

  • Transportation to and from the Shanghai airport upon arrival and departure.

  • Breakfast, lunch and dinner each day

  • Welcome reception/dinner

  • Wifi during the stay and use of SNAI recreational facilities

  • Limited visa assistance (e.g., letters of invitation)

The following items are NOT included in the fee:

  • Transportation from your home to Shanghai (airfare, public transportation, train, taxi, etc…)

  • Transportation within the city of Shanghai

  • Per diem

  • Travel insurance (for airfare, etc.)

  • Visa and passport application fees

  • Other miscellaneous costs (shopping, telephone charges, etc.)

s

 

Scholarships

Thank you for your interest in this course.  All scholarships have been awarded and no more scholarships are available.  We are still accepting applications to attend the course (but without scholarship).

3ie3iehas kindly agreed to sponsor a limited amount of scholarships, also selected competitively. 3ie will cover all costs included with the base fee, as well as airfare to and from Shanghai.

The selection process will be determined through the application process and will be based on the following criteria: potential benefit to you/your team in participating and applying the lessons to ongoing or planned impact evaluations, Asia regional priorities of our scholarship funders, inability to pay the course fees. Please indicate if you are applying for a scholarship in the application. Applications will be reviewed and a subset of participants will be contacted about the possibility of scholarships.


Certification

Participants will receive a “CLEAR” certificate of attendance.
 

About CLEAR-SHIPDET and Contacts

CLEAR-SHIPDET is run by AFDI and situated at the Shanghai National Accounting Institute (SNAI). SNAI was established by the Chinese government in 2000 to provide advanced education for financial professionals. SNAI has world-class teaching facilities, together with an information center, a recreation center, an international conference center, faculty apartments, student apartments, and dining facilities. The coordinators of this course are:

Amy Chen (Ms.)
Tel: (86 21) 6976 8060
Fax: (86 21) 6976 8016
E-mail: amychen@afdi.org.cn
Annie Wu (Ms.)
Tel: (86 21) 6976 8021
Fax: (86 21) 6976 8233
E-mail: wuningqin@afdi.org.cn

Address of SNAI: 200 Panlong Road, Qingpu District, Shanghai, China 201702, Tel: (86 21) 69768000

For more information, please refer to http://www.afdi.org.cn/

s

Course Modules Descriptions

Below is a description of each of the modules of the course:

  • Impact Evaluation in the policy and program cycles: This session gives a general overview of how evidence and impact evaluations are implemented within the policy cycle, highlighting both its strengths and weaknesses. In this discussion, we explore the development of theories of change for programs. We also identify which types of evaluation questions and policy decisions best align to impact evaluation.

  • The need for a counterfactual: This session provides an introduction to counterfactuals. Impact evaluations seek to answer the questions: What would the situation be with or without the intervention? What is the causal impact of the program (or project or policy) on the outcomes of interest?

  • Overview of Impact Evaluation Techniques: This session walks participants through a range of popular impact evaluation techniques, including randomized experiments, difference-in-difference, regression discontinuity, matching methods, regression estimators and systematic review.

  • Threats to Validity: This session will review impact evaluations that are very instructive regarding how evaluations should be implemented and what types of “threats” can arise regarding both internal and external validity, and how to address these threats.

  • Qualitative and Quantitative Methods: This session reviews key concepts related to these methods, as well as the differences and complementarities between quantitative and qualitative methods when conducting impact evaluations.

  • Impact Evaluation Management: This session looks at the kinds of competencies and the background required for a team undertaking impact evaluation. The session also provides an overview of the issues on how to disseminate the results of impact evaluations and the kinds of knowledge products that can be demanded in order to make impact evaluation more accessible to support the use of findings for policy and management.

  • Ethics and Politics: This session presents ethical and political considerations that must be taken into account when conducting impact evaluations.

  • Regression Analysis: Regression analysis is the key statistical tool used in almost all impact evaluations. This session provides a refresher for regression analysis and introduces the basic commands in the software for several regression models that are often used in impact evaluation, including linear regression models, probit/logit models, and random/fixed effects models.

  • Designing Questionnaires and other Data Collection Instruments: This session will provide an overview of the different types of questionnaires and other data collection instruments, and will provide practical advice and examples in survey processes, which are an essential part of many impact evaluations.

  • Sample Size, Sample Design, and the Power Calculations: A key issue in any randomized evaluation is how large of a sample is needed, and how the sampling should be done. The session provides detailed information on how to answer these questions, and explains some trade-offs that the evaluator will face.

  • Introduction to Regression Estimators: The session will present the four simplest estimators, as well as the assumptions required regarding the process by which people are selected and participate in impact evaluations to ensure that they yield unbiased estimates of impacts when program participation is not randomized. The four estimators are: cross-section, before-after, difference-in-differences, and within estimator.

  • Implementing Randomized Evaluations: In theory, randomized control trials (RCTs) provide accurate estimates of program impacts. Yet, in practice there are many things that need to be considered to obtain accurate estimates from RCTs. Also, many things can go wrong with RCTs. This session provides more detailed advice on how to conduct an RCT and discusses potential solutions to problems that may arise.

  • The Difference-in-Differences and Within Estimators: This session will explain the importance of the fixed effect errors structure for before-after and difference-in-difference methods, and discuss other issues relating to these two estimators (given that the assumptions needed for the cross-section estimator to be valid are unlikely to hold).

  • Introduction to Matching Methods: This includes Cross-sectional matching estimators; Rosenbaum and Rubin (1983) theorem for addressing curse of dimensionality; implementation of propensity score matching estimator; nearest-neighbor methods; nonparametric methods; and determination of the overlap support region.

  • Data Collection, and Survey and Data Management: This session will focus on data collection and data management issues, including finalizing the questionnaire design, field operations, data entry, and data “cleaning”.

3
Multi Day Event
No
Image Thumb
e
Event Information
China Time Zone (UTC+8:00)
200 Panlong Road, Qingpu District, Shanghai, China 201702