Study on the Demand for and Supply of Evaluation in Zambia

Submitted by on Tue, 04/12/2016 - 17:50

This report seeks to present in relation to evaluation in Zambia: (i) the conditions under which demand is generated for evidence; and (ii) the areas in which supply can be strengthened to meet and foster this demand. This report shows that there is currently active, latent and potential evaluation demand and supply in Zambia. It is argued that in Zambia each entry-point for evaluation is partial and is mediated by aligned interest groups rather than a neutral role-player seeking to expand evidence-based practice. This demand is set within a context where there is a high degree of political competition between political parties and various interest groups. In the political economy, loyalties to informal networks of power are in many cases more important than performance.

The latent and potential demands are nested within the Executive, while there is active demand for evaluation in the Ministry of Finance. This demand is conditioned by the political economy where both formal and informal agendas determine how policy is implemented. Supply could be strengthened through the various think tanks, the newly merged Evaluation Association, and through work with the well-resourced Centre of Excellence at the University of Zambia.

Evaluation supply is limited in the country. International consultants, rather than local ones, often lead evidence gathering or evaluation exercises. The expertise to undertake evaluations in Zambia has some islands of well-connected evaluation practice that works both in the formal and informal political domains. The university sector has some research capacity, especially in the social science sector, but indications are that there are structural challenges in the sector with qualified staff leaving for better paid positions elsewhere.

There are entry points in policy processes for improving evaluation capacities in Zambia with ongoing policy initiatives of the government as well as the Sixth National Development Plan. Functional sector working groups are lacking but are needed to feed into the governance of evaluations.

The approach for evaluation capacity development suggested in this study is to work across multiple entry points simultaneously to improve evaluation practice. This has the advantage of being able to interact with different points in the political economy, while also only resourcing different stakeholders to the degree that resources can be absorbed. This means that some institutions, such as Parliament, may require sensitisation activities and modest technical support rather than large-scale interventions. Meanwhile, entry-points, such as the Ministry of Finance, can be worked with to respond to new demands in a systematic manner that already builds on their successes.


Pdf as plain

Centre for Learning on Evaluation and Results Anglophone Africa (CLEAR-AA)
Graduate School of Public and Development Management,
University of the Witwatersrand, Johannesburg
Drafting team: Mr. Osward Mulenga (Country Consultant), Stephen Porter (CLEAR-AA/Wits)
Management Team: Stephen Porter, Osvaldo Feinstein, Salim Latib, Anne McLennan, David Rider Smith
Reference Group: Michael Bamberger, Derek Poate, Zenda Ofir, Robert Picciotto, Nidhi Khattri, Howard
White, Jessica Kitakule-Mukungu, Ian Goldman
December, 2013
Entire publication © Graduate School of Public and Development
Management, University of the Witwatersrand, Johannesburg
This report is an output funded by the UK Department for International
Development (DFID) as a public good. The views expressed are not necessarily
those of DFID.

For further information on this study please contact The content of this study is the
responsibility of the team alone, and should not be ascribed to the University of the Witwatersrand, DFID, or any other
organisation or individuals.
This report seeks to present in relation to evaluation in Zambia: (i) the conditions under
which demand is generated for evidence; and (ii) the areas in which supply can be
strengthened to meet and foster this demand. This report shows that there is currently
active, latent and potential evaluation demand and supply in Zambia. It is argued that in
Zambia each entry-point for evaluation is partial and is mediated by aligned interest groups
rather than a neutral role-player seeking to expand evidence-based practice. This demand is
set within a context where there is a high degree of political competition between political
parties and various interest groups. In the political economy, loyalties to informal networks
of power are in many cases more important than performance.
The latent and potential demands are nested within the Executive, while there is active
demand for evaluation in the Ministry of Finance. This demand is conditioned by the
political economy where both formal and informal agendas determine how policy is
implemented. Supply could be strengthened through the various think tanks, the newly
merged Evaluation Association, and through work with the well-resourced Centre of
Excellence at the University of Zambia.
Evaluation supply is limited in the country. International consultants, rather than local ones,
often lead evidence gathering or evaluation exercises. The expertise to undertake
evaluations in Zambia has some islands of well-connected evaluation practice that works
both in the formal and informal political domains. The university sector has some research
capacity, especially in the social science sector, but indications are that there are structural
challenges in the sector with qualified staff leaving for better paid positions elsewhere.
There are entry points in policy processes for improving evaluation capacities in Zambia with
ongoing policy initiatives of the government as well as the Sixth National Development Plan.
Functional sector working groups are lacking but are needed to feed into the governance of
The approach for evaluation capacity development suggested in this study is to work across
multiple entry points simultaneously to improve evaluation practice. This has the advantage
of being able to interact with different points in the political economy, while also only
resourcing different stakeholders to the degree that resources can be absorbed. This means
that some institutions, such as Parliament, may require sensitisation activities and modest
technical support rather than large-scale interventions. Meanwhile, entry-points, such as
the Ministry of Finance, can be worked with to respond to new demands in a systematic
manner that already builds on their successes.
EXECUTIVE SUMMARY.....................................................................................................................................II
LIST OF TABLES, FIGURES AND BOXES..........................................................................................................................III
ABBREVIATIONS AND ACRONYMS ................................................................................................................. IV
1. INTRODUCTION............................................................................................................................................1
1.1 METHODOLOGY................................................................................................................................................2
2. DEVELOPMENT CONTEXT.............................................................................................................................4
2.1 PLANNING, BUDGETING AND MONITORING AND EVALUATION FRAMEWORK.................................................................6
2.2 POLITICAL ECONOMY OF EVALUATION.................................................................................................................11
3. MAPPING OF THE EVALUATION SUPPLY AND DEMAND SYSTEMS..............................................................14
3.1 PRINCIPALS ....................................................................................................................................................14
The Executive: The Presidency, Vice President and the Cabinet..................................................................15
Civil Society Organisations ..........................................................................................................................18
The Legislature - Parliament .......................................................................................................................20
Development Partners.................................................................................................................................22
3.2 GOVERNMENT EVALUATION AGENTS ..................................................................................................................24
Ministry of Finance and Sector Ministries...................................................................................................25
The Central Statistical Office .......................................................................................................................28
3.3 EVALUATION AGENTS.......................................................................................................................................29
Think Tanks .................................................................................................................................................29
Evaluation Associations...............................................................................................................................31
4. PATHWAYS, OPPORTUNITIES AND CHALLENGES........................................................................................32
4.1 DEMAND.......................................................................................................................................................32
4.2 SUPPLY .........................................................................................................................................................34
APPENDIX 1: SAMPLE BUDGET TRACKING TOOL ............................................................................................37
List of Tables, Figures and Boxes
TABLE 1: MAPPING OF STAKEHOLDERS IN EVALUATION SUPPLY AND DEMAND ..................................................3
TABLE 2: SUMMARY OF ZAMBIA’S ECONOMIC INDICATORS, WORLD BANK (BANK 2013)....................................5
TABLE 3: AID TRENDS IN ZAMBIA ...........................................................................................................................5
FIGURE 1: MONITORING AND EVALUATION INSTITUTIONAL ARRANGEMENTS ..................................................11
TABLE 4: MEMBERS OF PARLIAMENT BY PARTY ..................................................................................................13
BOX 1: FARMER INPUT SUBSIDIES PROGRAMME.................................................................................................16
TABLE 5: DEVELOPMENT PARTNERS AND THE NATURE OF THEIR ASSISTANCE ..................................................22
BOX 2: EVIDENCE IN THE BUDGET CYCLE .............................................................................................................26
TABLE 6: SELECTED MINISTRIES AND EVALUATIVE EXERCISES UNDERTAKEN......................................................27
BOX 3: HIV AND AIDS RESPONSE ..........................................................................................................................27
TABLE 7: CENTRAL STATISTICS OFFICE SURVEYS ..................................................................................................28
Abbreviations and Acronyms
ACBF African Capacity Building Foundation
AfrEA Africa Evaluation Association
AIDS Acquired Immune Deficiency Syndrome
ART Anti-Retro-viral Therapy
ARV Anti-Retro-Viral drug
CCJDP Catholic Commission for Justice and development and Peace
CLEAR-AA Centre for Learning on Evaluation and Results for Anglophone Africa
CSO Central Statistical Office
CSPR Civil Society for Poverty Reduction
DFID Department for International Development
ECD Evaluation Capacity Development
EMIS Education Management Information System
FISP Farmer Input Support Programme
FNDP Fifth National Development Plan
FRA Food Reserves Agency
GDP Gross Domestic Product
GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit
GoZ Government of Zambia
GW-MES Government Wide Monitoring and Evaluation System
HIPC Highly Indebted Poor Countries
HIV Human Immune Virus
HMIS Health Management Information System
IAPRI Indaba Agricultural Policy Research Institute
IIAG Index of African Governance
IMF International Monetary Fund
INESOR Institute for National Economic and Social Research
JCTR Jesuit Centre for International Development
LCMS Living Conditions Measurement Survey
MDGs Millennium Development Goals
MESSY Zambia Evaluation Association and Monitoring Evaluation Support System
MMD Movement for Multiparty Democracy
MOE Ministry Of Education
MOFNDP Ministry of Finance and National Development
MTEF Medium Term Expenditure Framework
NDCC National Development Coordinating Committee
NGOCC Non-Governmental Coordinating Council
OSISA Open Society Initiative for Southern Africa
P&DM The Graduate School of Public and Development Management
PAC Policy Analysis and Coordination
PEMFA Public Expenditure Management and Financial Accountability
PMRC Policy Monitoring and Research Centre
PRSP Poverty Reduction Strategy Paper
PS Permanent Secretary
RDP Research and Development Programme
SAG Sector Advisory Group
SNDP Sixth National Development Plan
SWAps Sector Wide Approaches
TI-Z Transparency International Zambia
UK United Kingdom
UN United Nations
UNDAF United National Development and Analysis Framework
UNDP United National Development Programme
UNFPA United Nations for Population Analysis
UNICEF United Nations Children s Educational Fund
UNIP United National Independence Party
UNZA University of Zambia
UPND United Party for National Development
USD United States Dollars
WB World Bank
ZDHS Zambia Demographic and Health Surveys
ZEA Zambia Evaluation Association
ZIPAR Zambia Institute for Policy Analysis and Research
ZNFU Zambia National Farmers Union
1. This report presents a mapping of the parties involved in evaluation practice in Zambia,
namely the principals (demand), government agents (commissioners) and evaluation
agents (supply). In Africa there is now evidence of emerging country-led demands for
evaluation (Porter and Goldman 2013), which is consistent with the general emphasis of
the Paris Declaration on the use of country-owned systems. It is hoped that national
stakeholders and those supporting them will use this report to better structure their
assistance to evaluation capacity development (ECD). Consequently, the main audience
of this study is intended to be those interested in ECD in Zambia. The Regional Centre for
Learning on Evaluation and Results for Anglophone Africa (CLEAR-AA)2
conducted this
study for the UK Department for International Development (DFID) as one of a set of
cases covering five countries3
. Fieldwork for this study took place between 17 and 21
June 2013.
2. A secondary audience of this study is people who are in general interested in the
development of evaluation systems. The analysis in this report presents a range of
opportunities for capacity development in relation to government agents (central and
line departments), the evaluation agents (consultants, think tanks and universities), and
principals (development partners, Parliament, the Executive and civil society). In
undertaking the mapping of the evaluation context in Zambia, the study identifies latent,
potential and actual demand, the conditions under which demand is generated and
potential sources of supply, with an explicit sensitivity to the political economy of the
3. This study refers to evaluation as covering both evaluation and evaluative research and
therefore uses the terms interchangeably. The primary objectives of the study are to
1) The conditions under which demand is generated for evaluation evidence; and
2) The areas in which evaluation supply can be strengthened to meet and foster this
The guiding questions that interface with these objectives are:
(i) On the demand side:
a) What has been the actual demand for evaluation from principals?
b) Where is there latent and potential demand for evaluation?
c) How is evaluation demanded in the current organisational arrangements?
(ii) On the supply side:
a) What is the range and capacity of entities supplying evaluation services?
b) How relevant are the managers and producers of evaluation to the actual
demand for evaluation?
(i) On matching evaluation supply and demand:

CLEAR-AA is based at the Graduate School of Public and Development Management at the University of the
Witwatersrand in Johannesburg. CLEAR-AA aims to enhance development anchored in learning, evaluation and results.
Beyond Rwanda, the other case countries are Ethiopia, Ghana, Malawi and Zambia.
c) Where can evaluation supply (actual, latent and potential) be
strengthened so that it meets and fosters demand?
4. This report finds that in Zambia there are multiple entry points to improving evaluation
demand and supply. However, the entry-points are partial and mainly represent aligned
interest groups rather than neutral role-players seeking to expand evidence-based
practice. There is active demand for evaluation within some government ministries,
especially the Ministry of Finance, and some potential demand to explore within the
Cabinet Office. Outside of government there are well-connected organisations who seek
to use evaluative information to engage in advocacy with government. This demand is
set within a context where there is a high degree of political competition both between
political parties and various interest groups. In the political economy, loyalties to
informal networks of power in many cases are more important than performance.
5. The supply of evaluation expertise in Zambia is diverse in its quality. In response to the
historical demand from development partners and locally commissioned activities (using
development partner funding) a range of consultancy companies and individuals have
arisen with particular areas of strength. In the health sector, for example, a lot of
resources have been put into the generation of evidence, particularly in regards to
HIV/AIDS and malaria. The university sector has some research capacity, especially in the
social science sector, but indications are that there are structural challenges in the
sector with qualified staff leaving for better paid positions elsewhere, as President
Michael Sata (2013) observed at the opening of Parliament “our universities and
colleges do not only have dilapidated infrastructure but are also faced with shortage of
staff and apt teaching and learning materials.”
6. When conducting evaluations in Zambia there needs to be an awareness that the official
objectives contained in development plans are only partial articulation of the informal
objectives of interest groups. In some cases personal interests are pursued through the
state. Within these constraints there are examples of evaluation being successful at
informing strategy and policy and contributing towards development results. This means
that when supply is being strengthened the emphasis is not only technical, but also on
identifying entry points in the state where development agendas are being pursued for
the benefit of the population.
1.1 Methodology
7. This study was carried out through a combination of desk review, including an analysis of
existing evaluation products, and direct semi-structured interviews with a selection of
informants from across critical stakeholder groupings. The rest of this section presents
the methodology of the study, and provides an overview of the stakeholder groups that
were engaged with in the study. The study methodology encompassed the following
overlapping stages:
1. Establishing study commitment and support from key stakeholders;
2. Collating and analysing primary and secondary data and information of the
evaluation system (including available academic and popular literature);
3. Conducting a series of interviews with actors; and
4. Producing a draft paper.
Each of these stages is discussed in more detail below.
8. Establishing support from key stakeholders: Given the nature of the study, an
important initial step was to identify relevant national stakeholders who would be
engaged prior to, during and after the collection of data. Whilst the study was
conducted independently, it is helpful that there is some level of active buy-in from key
stakeholders to support the use of the study. In Zambia, letters were sent to the
Government via the Evaluation Association to ask for interviews, while the country
researcher made contact with a range of stakeholders.
9. Collating and analysing secondary data and information: The collation and analysis of
secondary data covered policy, academic and grey literature relating to the political
context and the demand and supply sides of evaluation. Included in this was data on the
size and scope of evaluation initiatives within government and the supply that emanates
from outside of government. Following the country research phase, further primary and
secondary documentation was referred to in order to substantiate the claims of the
interviews and further develop findings.
10. Interviews with key informants: A series of interviews were arranged with key incountry
stakeholders (see Table 1 for details). The design of these interviews drew upon
the literature reviewed and responded to themes that emerged in the country mapping
process. In particular issues around the political economy, potential and latent demand
and the ability of supply to invoke demand were explored through the interview
process. Data collection took place in a semi-structured fashion that allowed people to
narrate their story – with some probing taking place based upon the guiding supply and
demand questions. The data from interviews was analysed during the fieldwork with
emerging conclusions refined.
11. Production of a draft and final country report: Following the completion of first draft of
this study, it was reviewed by the reference group. In addition this case was reviewed by
the DFID country office and then also by the Government of Zambia. During these
processes adjustments were made to the report to aid the clarity and accuracy of the
core findings.
Table 1: Mapping of stakeholders in evaluation supply and demand
Government Agents Evaluation Agents Principals
Ministry of Health Zambia Institute for Policy
Analysis and Research (ZIPAR);
Policy Monitoring and Research
Parliamentary research
Central Statistical Office The University of Zambia –
INESOR & Population Studies
Department Centre of Excellency
for M&E
Civil Society Organisations –
Ministry of Local
Government and Housing
Private Institutions and
individual Consultants engaged
in M&E
Development Partners - GIZ;
UNDP; AfDB; EU Delegation;
Irish AIDA; Norwegian
Ministry of Finance Evaluation Associations – ZEA
and MESSY Group
The cabinet office
12. The findings of the country of case of Zambia are presented in the following manner:
first, the development context is described in relation to evaluation; second, the state of
the current evaluation demand and supply is mapped; third, illustrations of how the
development context interacts with evaluation supply and demand are detailed; and
finally, pathways to improve the national evaluation context are suggested in alignment
with the objectives of the study.
13. Zambia gained its independence on 24 October 1964, with Kenneth Kaunda as the first
president. Although Zambia has not had a major civil war, its geographical positioning
placed it at the heart of liberation struggles on the continent. This difficult position,
along with internal politics, has meant that its development path has witnessed large
scale declines since the 1970s before revivals in the 2000s. Zambia is regarded as one of
the richest African countries in mineral wealth, but its development path has been
stunted, with the country being unable to grow GDP per head and lengthen life
expectancy until recently. On average people in Zambia today are poorer than they were
in 1965 and live shorter lives than they did in 1971. The UNDP noted (2001: 1) that
Zambia “is the only country in the world for which data on the human development
index is available with lower human development indicators in 1997 than in 1975”
14. After independence, Kenneth Kaunda and the United National Independence Party
(UNIP) declared a one -party state in 1972. During the late 1970s, the price of copper,
which is Zambia’s primary export, weakened worldwide resulting in a major contraction
of the Zambian economy. In the late 1980s the one party state of UNIP became
increasingly unpopular. In June 1990 riots against Kaunda accelerated. Many protesters
were killed by the regime. In 1990 Kaunda survived an attempted coup, and in 1991 he
agreed to re-instate multiparty democracy and following multiparty elections Kaunda
was removed from office.
15. The elections were won by the Movement for Multiparty Democracy (MMD, which was
born out of a mix of trade unions and civil society organisations) and Frederick Chiluba
became president. In order to try managing its debt burden, in 1991 Zambia undertook a
range of economic liberalisation policies, such as privatisation, trade liberalisation,
subsidy cuts and public sector wage freezes. The Joint Evaluation of Support to AntiCorruption
Efforts (NORAD 2011: xv), points out that although there is
…no empirical evidence on the causes and drivers of corruption in Zambia. Studies
nonetheless agree that liberalisation in the 1990s introduced a ‘new culture of
corruption’ in the country. A presidential slush fund, infamously used to obtain
political buy-in during Frederick Chiluba’s regime, and privatisation, supported by
World Bank-led structural adjustment programmes, have allegedly triggered
high-level corruption.
16. Frederick Chiluba was replaced by Levy Mwanawasa (also MMD) after failing to win
support for a third term. Levy Mwanawasa undertook efforts to curb corruption.
Mwanawasa’s death in 2009 led to Rupiah Banda becoming the President until 2011.
Both Chiluba and Banda were indicted for corruption following their terms in office.
Chiluba’s indictment was withdrawn prior to his death in 2011 although a court in
London found him guilty of stealing more than USD 40 million. In 2011 there was a
change of ruling party from the MMD to the Patriotic Front, led by a former UNIP and
MMD politician, Michael Sata.
17. Zambia's economic record since the oil price shocks of the 1970s was woeful until
around 2005. Real GDP per capita fell from US$1455 in 1976 to US$1037 by 1987, an
average of -3.6 per cent per year. This decline stabilised from 1987 to 1991, before the
economy entered a massive recession again in 1992, the year an extensive programme
of structural adjustment began. By the mid-1990s Zambia had one of the highest per
capita foreign debts. After the external economic shocks suffered in the early 1970s,
Zambia's total external debt rose from US$814 million to US$3,244 million by the end of
the decade. The situation then further deteriorated with Zambia's external debt more
than doubling to US$6,916 million by the end of the 1980s. By 2000, real GDP per capita
had fallen to US$892 (Focus 2004).
18. From 2002 onwards, however, the country has been enjoying consistent economic
growth. Table 2 below is taken from the World Bank and illustrates the change in the
various economic indicators and life expectancy over 2002 and 2011. Meanwhile
19. Table 3 shows that aid has consistently become less important to the Zambian economy.
Table 2: Summary of Zambia’s Economic Indicators, World Bank (Bank 2013)
Table 3: Aid Trends in Zambia
20. Since 2006, Zambia achieved single-digit inflation along with GDP growth of roughly 6%
per year. In 2010 the World Bank (WB) named Zambia one of the world's fastest
economically reformed countries. However, the evidence is this economic growth has
not cascaded down to the rural areas. Equitable distribution of the national wealth still
remains a challenge. In Zambia, the World Bank’s poverty headcount (measured by the
percentage of people living under $2 per day) is one of the highest in the world at 86.6%
in 2010. This has risen since 2006 when it was 82.6% (Bank 2013). This reinforces key
statistics in relation to Zambia’s development which generally show a similar picture of
inequitable distribution of development. The United Nations Development Programme
(2013: 10) reports that, “Zambia’s Gini coefficient, which measures income inequality,
has worsened, from 0.60 in 2006 to 0.65 in 2010.” The UNDP Millennium Development
Goals (MDGs) report stresses that economic activity is largely centralised in urban areas,
leaving rural areas underdeveloped and unable to access economic ventures because of
unequal opportunities. Zambia has a life expectancy of 56 years (Bank 2013), which is
comparable to relatively poorer countries like Rwanda and Uganda. Recent
Indicator 2002 2003 2010 2011
GNI per capita PPP (current international $) 950 1000 1390 1510
Population total (in millions) 10.6 10.9 13.2 13.6
GDP (current US$) (in millions) 3711.3 4341.8 16190.2 19204
GDP growth (annual %) 3.3 5.1 7.6 6.8
Life expectancy at birth total (years) 43 45 55 56
Indicator 2008 2009 2010 2011
ODA (Millions US$) 1116 1267 914 1046
Aid (as % of GNI) 8.4 10.2 6.2 5.8
Aid per Capita (US$) 90 99 69 77
improvements in life expectancy appear to be related to advances in response to the
generalised HIV epidemic.
21. Zambia’s 2013 UNDP Report shows that the country has made progress towards
reaching its targets for some of the MDGs. The report identifies
the country’s remarkable progress over the past decades in achieving certain
targets”, but emphasises “the need to sustain these gains, while greater policy
focus and investments are required to improve progress on other targets. (UNDP
2013: 56)
22. Zambia needs to pay particular attention to finding ways in improving MDGs related to
secondary education, reducing child and maternal mortality, ensuring environmental
sustainability, and reducing hunger (UNDP 2013: 53). Zambia’s efforts to reduce the
spread of HIV/AIDS are on target, while efforts to reduce malaria in Zambia are offtarget
following a reduction in funding (UNDP 2013: 39-40). So while there have been
increases to access to health and education, Zambians have yet to benefit from more
efficient and effective service delivery systems. The reasons cited for this are that: there
are human resource constraints and that all sectors are characterised by weak
accountability for results at the local level; the complementary role of non-state actors
in service delivery has not been fully harnessed; and there is heavy dependence on aid
for education and health services (UNDP, 2011).
23. The judicial system in Zambia lacks the capacity to deal effectively with the proliferation
of litigation and cases, which tend to drag on for years before they are finalised. This
source also notes that this situation has deterred foreign investment due to fear on the
part of investors that the judicial system is so badly run down that it cannot offer
adequate legal protection for investments.
24. Zambia is ranked at 12th out of 52 countries on the Ibrahim Index of African Governance
(IIAG). The largest improvement in the index in Zambia has been in the health sector,
which has shown an improvement of +11.6% over a 6-year period
( According to the IIAG, Zambia has witnessed a decline
in rule of law since 2006, while accountability as remained pretty much stable with a
slight increase of 0.5%.
2.1 Planning, Budgeting and Monitoring and Evaluation framework
25. There appears to be broad consensus on three strategic challenges facing Zambia
between government, civil society and development partners. First, that economic
growth at a rate over 6% needs to be maintained over a 25-year period in order for
Zambia to become a prosperous middle-income country (GoZ 2005). Second, the
Zambian economy remains too dependent upon copper exports that make up around
60% of the country’s total exports (Economics 2013). Finally, it is recognised that the
decade of economic growth has not benefited the masses or translated into improved
human development (Front 2011; UNDP 2013; USAID 2011).
26. In responding to these challenges, the current strategic response focuses very much on
the provision of infrastructure, although official documents point to a range of other
priorities. Currently slightly different drivers of policy can be identified in three main
documents, namely: The Sixth National Development Plan (incorporating Vision 2030),
the Patriotic Front’s manifesto, and the national budget. An analysis of all of these
documents shows that although there are overlaps they do not completely align.
27. The Sixth National Development Plan (SNDP) is the current official development
planning framework dated as effective from January 2011, having been released before
the elections in September 2011, and the subsequent change in government. National
planning, which was undertaken during the single party state, but not during the 1990s,
came back into existence initially in the form of the Poverty Reduction Strategy Paper
and then the Fifth National Development Plan for the period 2006 - 2010. The final
evaluations of the Fifth National Development Plan (FNDP) are currently being
28. The strategic focuses of the SNDP are infrastructure, rural development, and human
development. Within these focuses the priority sectors are agriculture, livestock and
fisheries, mining, tourism, manufacturing, and trade and commerce. The SNDP is cited as
the basis for development partners’ assistance. For example, the Joint Assistance
Strategy, signed by fifteen bilateral and multilateral agencies, supports the SNDP. The
largest allocation of the itemised support goes to infrastructure development (the
largest package of overall assistance is project grants, followed by general budget
support) (Partners 2013).
29. However, the challenge with treating the SNDP as an operational policy is twofold. First,
implementation mechanisms for national plans have historically not been consistently
put into practice. For example, the mid-term evaluation of the Fifth NDP noted that
“there is a vacuum in the institutional set up because there is no planning authority to
ensure that sector planners followed the FNDP” (MoF 2009: 152). Further to this point
there is no evidence that the National Development Coordinating Committee (NDCC), a
core body for supporting development between various actors, has actually been
convened. Additionally, sector advisory groups are not consistently convened or if they
are a number have been recorded as ineffective (Leiderer and Faust 2012: 167; USAID
2011: 154). Second, following the election the Patriotic Front according to interviews has
emphasised that their manifesto takes precedence over the SNDP. This has led to a
review period of the SNDP to align it to the Manifesto. The current Medium Term
Expenditure Framework (MTEF), for example, recognises both as sources for policy,
stating that the MTEF has “been guided by the... [SNDP] and the aspirations of the
Government as set out in the Patriotic Front Manifesto”. In the body of the MTEF the
key strategies for growth are mainly aligned to the SNDP.
30. The second document that can be considered to be a source of strategic policy direction
for Zambia is the Patriotic Front’s manifesto. Central to the Patriotic Front’s manifesto is
a core programme based on the aspiration to create pro-poor growth, generate access
to quality education and health care and implement decentralisation of the
government’s administration (Front 2011). Within some of these core programmes
there are indications that the manifesto is being implemented. For example, a bill has
been introduced to create a higher education authority in Zambia, a stated priority for
tertiary education in the manifesto. In another example, the tax regime for mining
companies has been reviewed, which has increased revenue. Further, there is
momentum on decentralisation of administration power with the launch of a new policy
initiative. However, the constitutional reform process, which the Patriotic Front
government had promised to complete within 90 days of coming to power in 2011,
seemed to be stalled at the end of 2013.
31. The final document that could be taken to represent the strategic priorities for the
country is the national budget. The theme of the 2013 budget address in October 2012
was “Delivering Inclusive Development and Social Justice”. Within the budget overlaps
can be noted with the Patriotic Front’s Manifesto and the SNDP, for example, in the
areas of infrastructure, decentralisation, agriculture, tourism, and manufacturing.
Education and health feature less strongly than the Patriotic Front’s manifesto implies,
while the emphasis on infrastructure development is increased. The issue with taking
the budget as a firm statement of intent is that the execution is very different. Issues
with budget execution are further discussed below.
32. Zambia has undergone a range of public finance reforms linked to planning and
monitoring and evaluation (M&E) over the past decade. Three phases of reforms have
been undertaken since the year 2000. These are: (i) initial reforms according to the
Highly Indebted Poor Countries (HIPC) Initiative; (ii) Poverty Reduction Budget Support
and the Fifth National Development Plan; and (iii) National planning and budgeting
reforms along with implementation of the Sixth National Development Plan. In
reviewing this history some important progress can be identified with some on-going
33. Phase 1 - HIPC 2002 - 2005: Following the unsuccessful third term re-election bid of
Chiluba, the new government applied for the HIPC Initiative. In the initial phases of
financial reforms in the early 2000s it was recognised by the IMF (IMF 2005) that
Zambia’s public expenditure management system was not a functional tool for the
implementation of fiscal policy. An activity-based budgeting system and MTEF were
implemented in 2003 and 2004 to financially remedy this situation. Both of these
systems are still in place today. In order to qualify for the HIPC Initiative, a Poverty
Reduction Strategy Paper (PRSP) was implemented. At the end of this first round of
public finance and planning related reforms a range of issues emerged. In this first phase
the International Monetary Fund (IMF) (2005) reported the dominant challenge that was
identified by implementing agencies was the inadequate and untimely disbursement of
funds. It was also noted that at the end of the period poverty rates remained largely
unaffected. Furthermore there was no monitoring and evaluation of targets. One of the
chief recommendations of the IMF was that there is a need to carry out regular
evaluations to assess progress on outcome and impact indicators.”
34. Phase 2 - Poverty Reduction Budget Support and Fifth National Development Plan
2005 – 2010: This second phase of national planning and finance reform saw an
increased emphasis on budget support mechanisms with implementation defined
through a national development plan. Framing this process has been Vision 2030, an
aspirational document with general development aims. During the period 2005 – 2010,
external and domestic debt in Zambia was brought down to moderate levels,
international reserves increased significantly, and inflation reduced to single digits
(Kemp et al. 2011). Reform of the budget calendar was undertaken during this period to
support Parliament to have oversight of a full twelve months of implementation. With
support from the Ministry of Finance, “the de facto independence of the institution [of
the Auditor General] has increased” (Kemp et al. 2011: 108). An assessment of public
financial management noted
significant improvements in various dimensions of [Public Finance
Management]…in particular an improved basis for strategic budgeting, greater
comprehensiveness of fiscal information (including systematic reporting of
arrears), improvements in internal auditing and improved oversight, yet, all in all,
Zambia’s… system continues to exhibit significant weaknesses.” (Kemp et al. 2011:
113 & 15)
35. Improvements in public finance in 2008 – 2009 have been difficult to maintain since
Banda’s election. Getting a more complete picture on budget execution has been
hampered by a declining level of transparency. These issues are discussed in Box 2.
Financial releases are reported to be a continued problem, while the decentralisation
plan was not implemented. Further, challenges in in the distribution of funds and
coordination continued to affect the efficiency and effectiveness of delivery. It was
recognised in a mid-term review of the FNDP that the statistical services were not up-todate
and that there is a lack of coordination and harmonisation of M&E systems
between ministries. Importantly,
two Annual Progress Reports on the FNDP revealed the inability of several sectors
to report against the [key performance indicators]. Six sectors could not report
against any of the indicators…, while other priority sectors, notably Agriculture
and Water and Sanitation consistently appear to have difficulties in reporting
against some of their major KPIs.4
36. Phase 3 - New Reforms 2011 – Present: The current reform phase is being undertaken
in the context of a 2010 evaluation of the implementation of the Paris Declaration. The
evaluation report of implementation of the Paris Declaration states that “the findings
highlight the need for a more balanced donor-recipient relationship in Zambia which
requires fundamental changes in the underlying principles and incentives of aid.”
(Chigunta and Matshalaga 2010: vii). In particular it noted that there was a tendency for
development partners to entrench their own positions through parallel forums in the
context of the government of Zambia not being assertive about development priorities.
37. As part of a response to these issues a draft National Planning and Budgeting Policy (GoZ
2013) was being circulated for comments in 2013. This policy (GoZ 2013: 6), recognises
that challenges have persisted across the two previous waves of reforms, stating that
there is:
…weak linkage between budgeting and development planning procedures;
ambiguous and variable processes used in practice for preparing MTEFs, budgets
and development plans; and no legally binding institutional structures in place to
undertake budgeting and development planning procedures in a manner that
ensures informed participation by relevant stakeholders and effective oversight by
the National Assembly.
38. In particular the new draft policy notes that the implementation of activity based
budgeting was not undertaken with a significant mind-set change to support a more
results/performance orientation, while the credibility of the annual budget itself has
been undermined for many years by wide in-year variations between appropriated
amounts and actual expenditures incurred by ministries, provinces and other spending
agencies. In their analysis a challenge with the earlier phases was that accountability

4, accessed
mechanisms were not included in the reforms. The draft policy notes that the
“effectiveness of SAGs has been compromised by the perception that they are often
donor- driven” (GoZ 2013: 10). Reporting on budget execution, the draft policy states
that “significant variances are belatedly regularised by submitting large supplementary
budgets to the National Assembly at the end of the financial year, in the last week of the
National Assembly session” (GoZ 2013: 11). The challenges noted in the analysis above
were confirmed in civil society submissions responding to the draft policy. A recent
report by Zambia Institute for Policy Analysis and Research (Cheelo and Banda 2012)
that analysed the Auditor General’s reports notes that financial irregularities in Zambia
have increased 2.5 times from 2008 to 2010 (which also demonstrates challenges in
Parliament’s oversight capacities to call departments to account). These issues appear to
continue to this day according to reports in the Zambian press.
As a result, the policy
proposes a variety of technical and political fixes to better align the national planning
cycle with the vagaries of a five year electoral cycle.
39. In this third phase, the SNDP is an important document for undertaking M&E within
Zambia. As can be seen from Figure 1, according to the SNDP the Ministry of Finance is
at the heart of the planning, budgeting and M&E system in Zambia. All information from
sectors and lower tiers of government are collated in the Ministry of Finance, before
being reported to the cabinet and Parliament. (i) Budget Execution Monitoring; (ii)
Project spot monitoring; (iii) Analysis of administrative data or Management
Information Systems; (iv) Surveys; and (v) Research and Development. Evaluation in the
SNDP document appears only being undertaken mid and end-term. However, in reality
the Research and Development Programme in the Ministry of Finance has been
established to undertake ex-ante and ex-post evaluative exercises of sectors in relation
to the SNDP. The role of the Research and Development Programme is discussed further
in the mapping section.

Figure 1: Monitoring and Evaluation Institutional Arrangements
Source: Sixth National Development Plan
40. Separate to the SNDP, the proposed policy framework (GoZ, 2013) suggests further
evaluation mechanisms. For example, it suggests that major capital projects and
recurrent programmes will undergo appraisals and/or evaluations (2013: 15). These may
well compliment the existing annual review processes of the National Development
Plan. Indeed, currently the Ministry of Finance has produced Annual progress review
reports is finalising a range of evaluations. This process will be discussed further in
reference to government agents.
41. Three challenges to implementing the M&E plans of the SNDP have been noted. Firstly,
there is still no National Development Coordinating Committee to review information
and guide implementation. Secondly, the capacity for M&E at all levels is quite low
especially at provincial and district level. For example, data quality challenges in
reporting and setting targets have all been cited as current issues with the system in
interviews. Thirdly, while funding might be in place, it is rarely actually expended on
42. In summary, there have been advances in planning, budgeting and M&E in past decade.
Most notable is the currently stable macroeconomic environment and improved
financial management processes. However, there is also a continuity of challenges over
the past decade. Chief among these is that poverty has not decreased. Meanwhile, there
has been an ongoing challenge of budget execution. Across the phases efforts have
continued to increase rule-based accountability. However, the challenge may not solely
lie in a lack of rules, but with tensions that exist between differing aims of interest
groups. This is further discussed below.
2.2 Political Economy of Evaluation
43. In Zambia stated policy intent is strongly mediated in implementation by the informal
objectives of a variety of stakeholders. In other words, although policy intent may
appear developmental, actual implementation is negotiated between interest groups,
often to the benefit of the powerful. This leads to a situation where loyalty often
matters more than performance (Leenstra 2012). Formal rules and systems can act as a
façade for interests within an informal system of rules and decision-making that is
broadly dispersed amongst those with power. This has otherwise been called neopatrimonialism.
A range of commentators and a number of interviews highlighted that
pecuniary interest and the extension of state power competes with poverty reduction in
the objectives of powerful interests within the state (FODEP 2011; Leenstra 2012;
Rakner 2012). For evaluation this means that openings for evidence to inform policy are
much more tenuous than they may appear at face value with change likely to be very
44. This means that the constitutional values of democracy, transparency, accountability
and good governance are subordinate to the informal power structures within the state.
Perceiving that formal rules often become vehicles for informal objectives helps to
unravel a variety of seeming contradictions that inhibit or enable Zambia’s ability to
improve learning through evaluation.
45. An in depth case study of the Zambian health sector that was conducted by Leenstra
(2012) provides an illustration of the connections between the formal and informal
system within Zambia. This study connects the micro-level behaviours in the health
system to the macro context. The author finds that while rules do exist and are followed,
these are also on occasions warped and used to justify behaviours that are against the
public interest, such as health professionals “stealing government time” (Leenstra 2012:
11) to pursue their own business interests. In interviews during this study it was
confirmed and accepted that professionals use their government position to pursue
their own interests. However, there is no simple pattern: people can choose to follow
patronage-driven or rational legal behaviours, or even combine them. As Leenstra (2012:
302) argues in relation to donor interventions in Zambia, this means that
the results of …interaction can never be planned or foreseen: the arena is complex,
made up of competing and conflicting interests, and what is articulated is never a
complete and accurate representation of real interests.
A structure or process can become ordered or disordered so as to enable personal,
group, or institutional benefit to be derived. On the one hand, disorder such as poor
record keeping and information management can be not due to a lack of capacity, but a
strategy to avoid accountability. Similarly, order, such as in the form of public service
reform programme can provide an opportunity for consolidating power, diverting assets
and resource accumulation. On a large enough scale these behaviours transcend the
personal interests and become a way groups accumulate entitlements.
46. The interplay of formal and informal logics can be identified in mediation of presidential
power, the differentiated approach to public debate, and within divisions of civil society.
Since 1991, presidential power has been both supreme and severely curtailed in Zambia.
The constitution of 1991 gives the President supreme power within the state over the
allocation and oversight of the business of government. The President appoints the
heads of the judiciary, the public service, the Auditor General, eight members of
Parliament out of 150 and the attorney general. Although all of the appointments have
to be vetted by Parliament, the President can either overrule this or secure the votes of
opposition MPs to get to a majority of support (in addition to the 8 MPs he has the
power to appoint). Further there is very limited decentralisation of decision-making and
political power. Decentralisation has been discussed in Zambia since the 1990s, and a
renewed policy was put in place earlier this year6
. Yet historically little has been
achieved on this front, which places a limit on the entry points for evaluation practice
into the main central ministries.
47. The political manoeuvres since 2011 highlight the power of the President over the
legislature in the political economy. In 2011 just following the general election there
were 18 deputy ministers (Times 2011), while today there are 41 deputy ministers (P. o.
Zambia 2013). According to the Open Society Initiative for Southern Africa (OSISA), 13 of
these deputy ministers are formerly opposition MPs who have resigned their seats and
triggered by-elections of which the government has won five out of six (Geloo 2013). As
Table 4 shows, this has led to an outright majority for the Patriotic Front by May 2013,
even without opposition ministers voting for the government. In effect, the President
has the power to limit the actual power of parliamentary oversight. Yet, in spite of the
supreme power granted by the constitution, two presidents have been removed from
office by popular processes when they sought to extend their rule: Kaunda by civil
society and then an election, and Chiluba by members of his own party and civil society
(Phiri 2003). In both cases it appears that the President seems to have alienated
important power networks.
Table 4: Members of Parliament by Party
48. Contradictions between the operations of formal rules are also in evidence in regards to
reporting in the public space. Although within the constitution of Zambia the founding
values of transparency and freedom of speech are enshrined there is evidence that
control over public debates is sought by politicians especially in regards to the media.
According to Freedom House, all governments have harassed critics within Zambia’s
press since the end of one party rule. The government also broadly controls the media
through the issuing of licences through the Ministry of Information. Currently Freedom
House rates the Zambian press as “Partly Free”, although is it on the borderline of “Not
Free” (House 2013). Rakner (2012: 12) reports that “there is clear evidence that licenses
are used in order to control the private media.” Defamation laws and website blocking

Party Seats Following
General Elections
Seats as of May
Patriotic Front 60 (+8 appointed) 68 (+8 appointed)
Movement for Multi-party Democracy (MMD) 55 43
United Party for National Development (UPND) 28 30
Independents (IND.) 3 2
Forum for Democracy and Development (FDD) 1 1
Alliance for Democracy and Development (ADD) 1 1
Vacant 2 5
Total 150 150
are other tools used to control negative reporting. For example, the Zambian Watchdog,
a critical website, has been shut down through a combination of court action and
blocking of the website (Carlucci 2013). On the other hand, government officials and
ministers are available and accessible. Political parties can also advertise in the daily
newspapers and there is debate in public spaces, which was reportedly more thoroughly
curtailed during Kaunda's regime (Kasoma et al. 1997).
49. One final role-player that requires consideration in the political economy is civil society,
especially organised interests. Civil society has sections with political power, although
these tend to represent organised interest groups. Civil society has played an important
role in bringing an end to the one party state and in opposing Chiluba’s attempt to run
for a third term (Leenstra 2012). A well-cited example of civil society momentum was
the OASIS forum, a grouping of NGOs and opposition parties formed to oppose Chiluba’s
bid for a third term. It is currently reported that civil society activism is weak in Zambia
(Rakner 2012). It was perceived during interviews that Zambian civil society weakness is
based upon a highly polarised political environment with different civil society
organisations supporting different political parties. A sign of this is the report by
Coalition for the Defence of Democratic Rights to the Commonwealth Secretariat
outlining a range of reported abuses by the Patriotic Front government (Bandow 2013)
A further issue that arose that during the research is that the rooting of sections of civil
society in the country is shallow and responds rather to donor funding. One respondent
referred to the ‘Lusaka Consensus’ in policy processes.
50. Against these challenges in civil society there remain strong interest-based organisations
that have insider links to government, such as the Farmers Union and the churches.
These examples, which are further discussed in the body of the report, demonstrate that,
just as with other institutions, the power of civil society is based upon its ability to
access both formal rule-based mechanisms and informal relational spaces. Where they
are unable to do this their power relies upon those who do have access and form
alliances to further specific agendas.
51. Understanding that complex barely visible forces whose objectives are focused on
securing group interests have a range of consequences for identifying and interpreting
the supply and demand for evaluation. It means that ECD work needs to recognise that
policy influence, even when there appears to be a clear cut opening, is tenuous and
incremental. Issues in state performance cannot be understood solely as a lack of
capacity to implement formal bureaucratic norms, because more rules will only have a
limited effect. ECD therefore needs to recognise that the use of evaluation is achieved
by working in both the formal and informal spaces. The informal space is, however, very
difficult to penetrate.
3.1 Principals

7; (since first accessing
this site it has now become infected with malware);
52. Principals are stakeholders that in the main demand evaluation (although can supply
and mange evaluations on occasions) as a result of their political power and positioning.
In this study, principals include the Executive, civil society, the legislature and
development partners.
The Executive: The Presidency, Vice President and the Cabinet
53. There is some latent demand for evaluation in the Executive. Transforming this to
consistent actual demand within the political context will be challenging as a variety of
other considerations come into play. The latent demand for evaluation is embedded in
an Executive that demands on-going monitoring reports of implementation. Large
changes in policy on important issues can spring from the Executive without public
discourse. This curtails transparency and access to information. Seemingly in
contradiction, the Cabinet Office emphasises transparency of participation in policy
processes and is open to the use of evidence. This apparent contradiction highlights the
meeting of different objectives within the policy process from the more political and
bureaucratic parts of the Executive given the political economy.
54. In the interviews it was reported that the Presidency has been demanding monitoring
reports of programmes currently being financed and implemented by government. For
example, according to the Zambia Daily Mail “the President could not comprehend why
it is hard to obtain up-to-date developmental information on provinces when
permanent secretaries exist.”8 On-going monitoring takes place through a budget
tracking tool, a sample of which is shown in Appendix 1. The nature of the demand for
monitoring, however, needs further investigation. First, evidence could not be found of
poor performance being censured as a result of the monitoring. Second, it was reported
in interviews that it was felt that this monitoring was not necessarily development
results-orientated, but focused on priorities of the presidency. Third, it needs to be
ascertained whether the political Executive has interest to go beyond monitoring and to
establish through evaluation the quality of outcomes.
55. The Cabinet Office, which is under the Presidency, oversees and coordinates all policy
emanating from government ministries through the Policy Analysis and Coordination
(PAC) Division. A Permanent Secretary, who the President has the power to appoint,
heads this division. The formulation of policy can be based on some evidence defined at
the sector level, but it is not standardised to conduct evaluations or identify evidence in
the policy process. It was reported by the Cabinet Office that the process of policy
formulation is consultative involving a wide range of stakeholders. There were different
perspectives on the extent of the consultation processes. Some reported that it was
limited to Lusaka. Other reports, however, emphasise bottom-up aspects of the Zambian
policy process9
. Ostensibly, the centralised policy process supports transparency and
participation in decision-making. This means that the Cabinet Office is a potential entry
point for demanding evaluations within policy consultation processes.
56. In contrast to the open processes of the Cabinet Office, there is evidence of restrictions
which have been put in place by the Presidency on the flow of information from

different sources. For example, challenges accessing the Zambian Watchdog website10
are reported to be a result of government action11
. Rapid and important policy decisions
are sometimes made by the Executive which are then debated in the public media and
general society, for example, changes to fuel and maize subsidies in the month of May,
2013. Major civil society stakeholders were not consulted in the decision process.
According to two media statements attributed to the President, Mr Michael Sata, the
removal of fuel and maize subsidies is justified on the grounds that the subsidies do not
actually benefit the poor and, therefore, funds saved from subsidies if applied to
construction of infrastructure such as roads, schools and health facilities could benefit
the poor more (Chellah 2013). This case is considered in Error! Reference source not
57. According to interviews there are certain sectors that the Presidency pays more
attention to if undertaking evaluations. One such is infrastructure development,
particularly roads. Recently the Roads Agency has been centralised in the Presidency,
with the contracts from the previous administration cancelled. The evidence from the
interviews suggests that procurement, especially of infrastructure, is an area where the
political Executive remains quite hands-on and so may be a sensitive area for evaluative
Box 1: Farmer Input Subsidies Programme
The current national debate on maize subsidies highlights the interplay and possible entry
points for evaluation that arises in the interplay between various interest groups around
policy processes. Although actual informal agendas are difficult to pin down there is enough
information around this issue to demonstrate points of entry for evaluation in the debate. A
national debate in the media between a various interest groups was sparked by an
announcement by the President of a change in the subsidies regime (widely reported as
subsidies removal) on the basis that the money could be better spent on infrastructure
Farming subsidies in Zambia cut across the value chain with three main forms of
intervention. The first is the Food Reserves Agency (FRA), which purchases maize from
farmers. The second is the farmer input support programme (FISP), which provides targeted
households with fertiliser. The third are subsidies to millers of maize aimed to help control
the price of maize. Together these three programmes account for 80% of government
spending on agriculture and suffer regular and very large overruns, as reported by the
…during 2012, the budget allocation for the FISP was K500 billion against actual expenditure of
K1.181 trillion, representing a budget overrun of K681.2 billion; while during the year 2011,
K485 billion was budgeted against actual expenditure of K1,354.70 trillion, representing a
budget overrun of K869.7 billion….In addition, during 2010, K100 billion was budgeted under
the Food Reserve Agency (FRA) maize marketing programme against actual expenditure of
K2.6 trillion, representing a budget overrun of K2.5 trillion; while in 2011, K150 billion was
budgeted against actual expenditure of K3.2 trillion representing a budget overrun of K3.0
These subsidies operate to achieve multiple objectives. Ostensibly, the aim of these
interventions is to provide cheaper food and reduce poverty in rural areas, but according to

10 A controversial website that contains allegations of government mismangement
research and reports actual benefits accrue to more specific groups (Mason et al. 2013).
Purchasing of food by the FRA has provided above-market prices for maize for a number of
years. For example, FRA purchased 83% of the marketed surplus between 2010 and 201213
Mason and Myers (2013: 203) show that that the FRA’s intervention in maize
“…raised mean prices between July 2003 and December 2008 by 17–19%...which assisted
surplus maize producers but adversely affected net buyers of maize in Zambia, namely urban
consumers and the majority of the rural poor”.
As a consequence the study concluded that:
…the increase in maize price stability is unlikely to have had substantial welfare effects on poor
households. In contrast, relatively wealthy producers are likely to have benefited from the
higher average and more stable maize prices resulting from FRA policies.
Research on FISP has found very little evidence of poverty reduction through the maize
subsidies (Mason et al. 2013; Ricker-Gilbert et al. 2013). In investigating the net impact of the
scheme on food prices, Ricker-Gilbert found that doubling the size of the input scheme would
reduce maize prices by a maximum of 1.6%. On the other hand research by the same group
(Mason et al. 2013: v) found that the FISP was “being disproportionately allocated to betteroff
households above the $1.25/day poverty line.” Further, prior to 2011 constituencies won
by the MMD received “significantly more subsidised fertiliser than those in areas lost by the
ruling party”. Media, Auditor General and civil society investigations into the allocation of
FISP have confirmed challenges in its targeting with their own investigations14
The organisation Civil Society for Poverty Reduction (CSPR) is opposing the reduction in
subsidies claiming that it is critical to smallholder farmers. The Zambia National Farmers
Union (ZNFU) has claimed that hunger will increase with a change in subsidy, arguing that
“providing support to poor rural farmers in form of agricultural inputs is not a subsidy but a
mandatory responsibility on the part of Government.”
15 Finally, subsidies to millers have not
been passed onto consumers in the form of cheaper maize meal from larger millers. Instead
the informal sector has been able to process maize meal at lower costs leading to a reduction
in price, though large millers are able to negate competition through the subsidy16
Against this background, where the benefits of subsidies currently consistently accrue to
those who are better off rather than the poor, government has been attempting to reform
maize subsidies. For example, this has resulted in the introduction of a barter system and
electronic distribution for the beneficiaries of FISP. The government has subsequently been
reported as stating their support for FISP, rather than a focus on the long-term removal of
. The 2013 citizens’ budget, which is a simplified version of the annual budget,
highlighting popular expenditure and revenues, notes that the 2013 FISP and FRA are still
budgeted to receive K800 billion between them18
What this case illustrates is the gap between formal policy intent (poverty reduction) and
actual implementation (rents accruing to more powerful interests) and the complex array of
interests mobilised around a policy shift. Given this, there are potential entry points for
evidence to retarget the subsidy, and also to remove the subsidy, by conducting evaluations
with civil society, farmers or the government. On the other hand, long-term research advice
is harder to come by generally, but in this instance was provided by Indaba Agricultural
Policy Research Institute (IAPRI), which is supported by SIDA and USAID.

However, what actual policy implementation ends up looking like is difficult to predict given
the array of vested interests and the potential that actual interests are only being partially
articulated publically as those who gain through large budget overspends on the programme
will use the language of poverty reduction to defend their interests.
Civil Society Organisations
58. Civil Society and its role in politics in Zambia has been shaped by a history of
authoritarian traditions, first during the colonial area, and later on during one-party rule.
For several decades, civil society was largely restricted to activity within the agendas and
control of the state, until it coalesced on two separate occasions into a major opposition
movement (1991 and 1999). This means that civil society has shifted between
supporting and opposing government in various phases. Though civil society has some
broad challenges (discussed below) which reduce its ability to engage in advocacy
through evaluation, there are some organisations identified during this study who are
well equipped for presenting their interests in both the formal and informal political
economy, namely: Civil Society for Poverty Reduction (CSPR); the Jesuit Centre for
Theological Reflection (JCTR); Transparency International (TIZ); and the Zambia National
Farmers Union. This list could be expanded, but in the scope of this research it gives a
feel for some of the entry points for evaluative activity by civil society and its
connections with policy.
59. A robust and balanced participation of the civil society is still impeded by five factors
even though the role of the civil society has continued to grow19
. First, there is a lack of
institutionalised mechanisms for citizens’ participation in decision-making other than
elections and political parties. An example of this is the current constitutional
development process. The Oasis forum, a body made up by a range of civil society
groups, recently bemoaned the lack of participation in the process that has been
undertaken over six years20
. Second, the relationship of government and civil society has
been confrontational with regard to issues of governance, participation, human rights
and the rule of law. An example of this is the recent NGO Act that was enacted in a short
space of time without consultation and has led to tension between civil society and
. Third, a large number of civil society organisations are based at national
level, and have limited local representation. For example, Civicus (2010) found that 49%
of civil society organisations in Zambia operated in Lusaka. The rest of the operations are
spread out over the 9 other provinces with no more than 13% of the civil society
organisations working in any other single area. This analysis reinforces a term that
emerged during interviews that there can be a ‘Lusaka consensus’ in decision-making.
Fourth, organisational and monetary constraints often force organisations to serve their
own survival needs and compromise on larger goals of development and change
(Civicus, 2010: 12). For example, Civicus (2010: 73) found that the focus of civil society
more or less mirrored the development partner priorities and that organisations were
almost entirely dependent upon funding from these agencies. Finally, there are
challenges around the coordination and collaboration within and between actors. For
example, the organisational survey findings from the Civicus (2010: 45) study shows that

19 The following list has drawn from Maitra’s (1998) observations and recent work by Civicus (2010).
networking and communications tends to be more within the membership of umbrella
groupings, rather than across the looser networks or alliances formed to address topical
60. These issues mean that although there are some stronger civil society organisations,
broadly the sector has little collective weight and reach through which formal or
informal policy networks are influenced. Concisely put, civil society lacks many
institutional entry points with government or the formation within itself to act as a
persuasive agent. An example of this is the constitution process in which civil society has
been involved that has not been finalised in six years. There are some examples that
have more reach, which are discussed below.
61. CSPR, established in 2000, is an anti-poverty advocacy network of over 140 organisations,
whose objective is to work for pro-poor development in different parts of Zambia. CSPR
seeks to ensure that civil society participates effectively and meaningfully in design,
formulation, implementation and monitoring through sourcing evidence-based data.
Budget tracking and service delivery monitoring are listed as current projects. Although
no project reports have been updated on the website since 2010, CSPR provides regular
commentary in the media on poverty reduction efforts and budget allocations. CSPR also
provided inputs to the recent draft policy for planning and budgeting following a
submission to Parliament on the same issue. Given this ongoing activity, CSPR is an entry
point for demanding evaluation activities to inform advocacy and policy influencing
activities of civil society. In this regard, ECD activities could focus on identifying and
demanding evaluative activities and perhaps managing these in a manner that could
further feed into CSPR’s and other similar organisations’ policy-influencing work.
62. JCTR is a research, education and advocacy organisation that promotes study and action
on issues linked to Christian faith and social justice in Zambia. JCTR began operating in
1988 as a project of the Zambia-Malawi Province of the Society of Jesus with similar
orientation and activities as other Jesuit social centres around the world. JCTR is an
important research advocacy organisation politically, based upon their history and the
prominent position of Christianity in the Zambian constitution. In the preamble to the
constitution, the country is defined as a Christian nation and society. According to
Afrobarometer, an independent, nonpartisan cross-country research project that
measures the social, political, and economic issues, religion is very important for 93% of
people in Zambia, the joint second highest out of 31 African countries22
. It is reported
that for Kaunda and Chiluba, losing the support of the churches was a major blow to
their efforts to remain in power (Phiri, 2003).
63. The JCTR produces the basic needs basket analysis. This analysis compares a basket of
basic goods with the take-home pay of Zambians. This analysis is conducted in both rural
and urban areas and is used for advocacy purposes with government, communities and
. The rural analysis is important as it provides external verification of ongoing
poverty issues in Zambia. The JCTR used the basic needs analysis as a base for inputs into
major policy decisions. Recently, it has provided input to the new draft planning and
budgeting policy discussed in the previous section.

64. Transparency International Zambia (TI-Z) was established in 2001. TI-Z is a professional
membership organization focused on anti-corruption, integrity, leadership and good
governance. TI-Z has been a vocal campaigner on corruption issues in Zambia. For
example, it has openly criticised the current administration and powerful actors such as
the Police Service. A major publication towards which the local chapter contributes is
the corruption barometer. In addition, TI-Z is pushing towards commissioning more
advanced evaluation activities such as public expenditure tracking surveys.
65. The Zambia National Farmers Union (ZNFU) is an organisation that has high-level access
and the ability to influence policy based on its representation in the formal and informal
agrarian sectors, representing a sizable economic and employment interest. Its breadth
of membership makes it one of the few organisations in Zambia to be able to claim
country-wide reach. Its website claims that apart from churches, the ZNFU is one of the
oldest associations in Zambia, having existed from the early 1900s.
66. One of the main evaluation and advocacy mechanisms used by the ZNFU is the
Agriculture Production Survey for which “data was collected from a sample of 4,000
households.”24 Further the ZNFU makes detailed budget submissions drawing upon a
range of evidence sources25. It was reported in interviews that the ZNFU enjoys highlevel
access to the government with ministers regularly attending ZNFU events.
Notwithstanding the recent reduction in the maize subsidy the ZNFU has been able to
defend a variety of protections enjoyed by the farming industry. For example, a wheat
import ban still remains in force, while the ZNFU is advocating strongly on payments to
farmers for government maize purchases and for quotas for trade with the Democratic
Republic of Congo26
67. Overall this mapping shows that there are a variety of entry points for evaluation
activities through civil society organisations in Zambia, although there are issues in the
structure of civil society. Organisations can be identified that can demand, manage and
implement a variety of types of evaluation exercises and take these into both formal and
informal policy spaces. There is no single champion within civil society: instead, as with
policy influencing in general, there are multiple entry points with a variety of skills that
may be better placed for different roles and interests in supporting opportunities for
The Legislature - Parliament
68. Across a wide number of sources consulted during this study the message consistently
arose that the legislature in Zambia has limited power of oversight in relation to the
Executive (Rakner, 2012: 16; Civicus, 2010: 31). The turnover in Parliament has been
high, for example, in 2011, 60% of parliamentarians were new, especially in the ranks of
the Patriotic Front27
. Subsequent by-elections have increased the number of new
members in the chamber. This means that the Parliament is currently operating with a
relatively inexperienced number of representatives with an assertive Executive. Further,

key committees such as the Parliamentary Accounts Committee are reconstituted every
year. This means that only half the members of the committee serve for the full term of
the Parliament28
. Parliament provides limited oversight on the Executive through its
parliamentary committee system, as well as through the Vice President’s (rather than
President’s) question time every Friday when Parliament is in session.
69. There are some mechanisms for evaluation in the legislature. For example, there could
be some ECD opportunities within the Research Section of the legislature. The Research
Section of the National Assembly of Zambia was established to serve parliamentarians in
two main ways: i) provide information and evidence to Members of Parliament (MPs)
who wish to make a private members motion; and ii) prepare background papers (briefs)
for MPs who may attend local or international workshops or seminars. The Research
Section has twelve researchers. For the most part, the Research Section does not
establish its own research agenda. It responds to MPs’ information requests and
requirements. Recently, however, it has started conducting research in anticipation of
the information that might be required and needed by the MPs. The Research Section is
currently conducting evaluative research to ascertain the effectiveness of some of the
aspects of the parliamentary reforms whose implementation begun towards the end of
2002. The funding of this research is coming from the United Nations Development
Programme (UNDP).
70. The Public Accounts Committee reviews and makes recommendations on the reports
from the Auditor General (which receives around 70% of its budget from Norwegian Aid).
The reports of the Auditor General, with the responses by Parliament, are released
electronically more than a year after the period reviewed. For example, the report for
the year ending 31 December 2011 was released on 18 July 2013. The Auditor General
also undertakes audits of other public entities, such as parastatal bodies. Among
parastatals there are failures to convene boards of directors, failure to prepare financial
statements and poor management of infrastructure (A. G. Zambia 2011). Among
museums, there are no strategic plans, weaknesses in the documentation of artefacts,
and transportation is not maintained. Responsibility for the remediation of the issues
rests with the Secretary to the Treasury, who must enforce these with line departments.
The main challenge is that the Treasury struggles to enforce change. This will be
discussed in the next section. Although Parliament can only offer recommendations
there might be scope for improved use and follow-up of the audit products as well as
increasing the scope of performance audits. Given the rapid turnover in committee staff,
capacity development efforts might well focus on the staff of committees.
71. Although firm oversight is limited, the legislature does have a role in convening
processes to discuss policy with multiple stakeholders, although final decisions are made
in the Cabinet Office. For example, digital migration for the SADC region was
implemented in Zambia through participation of relevant stakeholders such as the
media, civil society organisations and interested individuals.

28 Implementing the Good Practice Principles and SADCOPAC conference resolutions: the Case of Zambia, presented by Hon
Vincent Mwale, Chairperson of the Public Accounts Committee of Zambia, Mpumalanga, South Africa 3-6 September, 2012.
Development Partners
72. There is limited evidence that evaluations in Zambia are conducted as country-led
process. This finding is in alignment with the Second Phase Evaluation of the Paris
Declaration, which notes that mutual accountability mechanisms are weak between
development partners and the Government29
. However, there are some existing
mechanisms that are being used to support ECD which could be built upon.
73. Interviews with evaluators conceded that most evaluations were commissioned and
managed by development partners. Recent evaluations related to cash transfers30
European cooperation31
, the joint assistance strategy32 and anti-corruption efforts33
were all commissioned and managed by development partners, and led by evaluators
external to Zambia, with some cooperation from government. These findings highlight
those important evaluations on development, which have resource implications are
regularly conducted exogenously to the state. On the other hand, development partners
have supported a range of activities related to completing important surveys and
developing statistical capacity.
74. There are development partner efforts that are directly and indirectly supportive of ECD
within Zambia. The largest amount of efforts quantitatively seek to engage in
performance and accountability related reforms, especially related to public finances
(including the Auditor General), with 8 out of the 11 development partners reviewed
supporting this area of work (see Table 5 below). This work should be coordinated under
the Joint Assistance Strategy as there is an outcome focused on the area. Another focus
of development partners that provides indirect support to evaluation is in funding civil
society organisations ability to demand accountability and to advocate with
government. Here 6 out of 11 development partners have related funding.
75. A challenge with leveraging this support is in understanding how the political economy is
interacting with these programmes. The evaluation of the Public Expenditure
Management and Financial Accountability (PEMFA) Programme, a major initiative that
lasted six years and cost USD 74 million, highlights the risks, especially of public finance
interventions. The evaluation concluded that:
In all of the twelve technical components, goals and objectives have either not
been achieved at all or they have been delivered much later than programme
design had envisaged. Where PEMFA has attempted directly to improve PFM
processes, procedures and human resource capacities, in many cases this has not
been achieved.34
The challenges around this large and important public sector reform programme
highlight that case-by-case analysis (perhaps from a political economy perspective)
needs to be undertaken to understand the value of each reform initiative to ECD.
Table 5: Development partners and the nature of their assistance

29 - 68
34 Public Expenditure Management and Financial Accountability (PEMFA) Programme Evaluation - Overview Report - 3

# Development
Related Support to Monitoring and Evaluation
1 UN system Direct:
35 Contributed funding to evaluation of the Fifth National
Development Plan; Demographic Health Survey (UNFPA)
Indirect: The Technical Facility for Strategic Response – This is aimed
at enhancing the attainment of national development results and is
implemented by the Ministry of Finance. This programme has been
responsible for completing the country MDG report – No budget
recorded; Governance programme - This initiative is largely focused on
technical skills development, pay reform and right-sizing the public
service - US$7,280,000
2 World Bank Indirect:
36Research on economy, poverty and governance.
3 African
37Governance Reform Support Programme – Through
Improved budget execution and oversight.
4 Irish Aid Indirect:
38 Increasing accountability to citizens – E1.1m (approx. 2012)
5 Swedish
Indirect39: Strengthened democratic accountability and transparency,
and increased awareness of human rights – USD6.4million (2012)
6 Norwegian
40Transparency International – NOK3.5million (2012-2013)
Non-Governmental Organisations Coordinating Council (NGOCC) –
NOK18.8 million (2010- 2013)
Office of the Auditor General – NOK45 million (2010 – 2014)
7 Embassy of
Phased out governance support projects in 201341
8 GIZ Direct:
42Strengthening Good Financial Governance in Zambia, including
support to strengthen M&E systems in order to provide evidence-based
development planning, budgeting and programme implementation
(coordination of the evaluation of the FNDP, Gender analyses, and
support to national strategy for the development of statistics, LCMS
2010 etc.)
Civil society participation in governance reform processes and poverty
Indirect: Support of the Zambian Decentralisation Process
9 EU Delegation Direct: Public Expenditure Management and Financial Accountability
(PEMFA) programme (programme ended)
New strategy currently being formulated
10 DFID Direct:
43Budget and Public Service Delivery Monitoring by Civil Society
– GBP662k (2006- 2014)
Public Financial Management Zambia – GBP1,805k (operational 2011 -
2014) GBP138k (planned)
Support to Zamia 2010 Census – GBP3,150k (2010 – 2013)
76. Direct support to developing evaluation demand amongst principals is somewhat limited
in scope. The three main areas of support have been to the evaluation of the FNDP
through the Ministry of Finance (to be discussed later), the upcoming demographic
health survey, and a centre of M&E excellence at the University of Zambia that mainly
conducts training. Only the support to the evaluations of the FNDP has some local
demand-side linkages.
77. Two additional support mechanisms exist for evaluation through the coordination
mechanisms of development partners, Sector Wide Approaches (SWAps) and the new
Joint Assistance Strategy. First, there does seem to be some promise in Sector Wide
Approaches (SWAps) to support ECD. The potential of SWAps to support ECD is
emphasised within the evaluations on the Paris Declaration (Chigunta and Matshalaga
2010). All of these documents emphasise that SWAps have produced improved
coordination and lead to more evaluative activities taking place. Second, the Joint
Assistance Strategy 2011 – 2015, which is aligned to the SNDP, is a point of reference for
development partners. For example, both USAID refers to the strategy in outlining their
own country strategies (USAID 2011). In this joint assistance strategy there is an
emphasis placed on “using systematic information to improve decision-making”
(Partners 2013: 14). It is intended that capacity development efforts are to be
undertaken with the Ministry of Finance and Central Statistics Office. In addition, the
partners have formed an M&E group through which monitoring and evaluation issues
are discussed, strategy developed and funding coordinated to some extent.
78. The above analysis reinforces that a number of entry points for supporting ECD do exist
in Zambia. The challenge is that many of the opportunities are latent within larger public
service reform efforts. Further, there are limited functioning coordinating mechanisms,
such as SWAps or Sector Advisory Groups. This means that entry points need to be
nurtured in a context where there is a limited history of country-led evaluations and the
potential support to evaluation is in the main indirect. The direct work in ECD, with the
Ministry of Finance, University of Zambia, and Central Statistics Office are considered
separately below.
3.2 Government Evaluation Agents
Indirect: Public Sector Performance Programme – GBP535k (planned)
Management of Public Services Zambia – GBP1,833k (2006 – 2014)
Economic Advocacy Programme – GBP – 3,454k (2010 – 2016)
Zambia Anti-Corruption Programme – GBP6,318k (2009 – 2015)
Democratic Empowerment and Accountability Zambia – budget in
Direct: M&E Centre of Excellence University of Zambia – USD5.6million
(2013 – 2016)
Demographic Health Survey (2013)
Strategic Information Central Initiative (2012-2015)
Indirect: Assistance in the management of public resources - USD300k
Increase the ability of civil society organisations to advocate and
engage with government on service delivery - USD500k (2013)
Ministry of Finance and Sector Ministries
79. In restructuring the Ministry of Finance 2008 a separate M&E Department was
established with a mandate and responsibilities to monitor and evaluate the national
development plan. Since 2010, a separate programme for evaluations and research
exists, Research and Development Programme (RDP), which has coordinated sector
specific analyses but also the final evaluation of the FNDP. Within the Ministry of
Finance there is an actual demand for evaluations to support with policy processes,
especially to ensure that the MTEF and annual budgets are based on analytical research
and M&E results. Responding to this, within the RDP an annual research and evaluation
plan is issued in conjunction with the line ministries. In interviews it was emphasised
that the Ministry of Finance has decent technical skills, a degree of management buy-in
to evaluation and a commitment to implementing policy in accordance with rules
(although at times these are circumvented). In other words, the Ministry of Finance was
considered to have a commitment to development in alignment with formal rules and
80. There are three overarching challenges to implementing the annual research and
evaluation plan, the first is human, the second is financial capacity, the third,
institutional. Interviewees reported that currently, the M&E Department spends a good
amount of time on the monthly monitoring exercise as part of supporting the new
government wide M&E system (GW-MES) demanded by State House using the tool
provided as a sample at Appendix 1. Stakeholders reported that the monthly reporting
system has posed a challenge as most other ministries have limited capacity to populate
it with the required data and information. This means that staff from the M&E
Department need to spend time following-up with line Ministries. The issue here is that
monitoring might crowd out attention to evaluation. It is also important to note that the
implementation of Government Wide M&E is a new demand and represents a potential
expansion of the mandate of the Ministry of Finance. However, it was reported in
interviews that there is ambiguity around who owns the M&E system as there is an M&E
unit in the cabinet office. This study could not find any evidence of their activities.
81. The human resource challenge is reinforced as currently the coordination of the
evaluations of the FNDP is carried out by an external consultant. The consultancies to
undertake the evaluations of the FNDP have been paid for by development partners (EU,
UN, JICA and GIZ) and only one component by the Ministry of Finance. The contribution
of the Government / Ministry of Finance is significant for all consultations, technical
committee, publications and dissemination of results; the Cooperating Partners were
approached to sponsor the consultancies only. This demonstrates a commitment within
the Ministry of Finance to support ongoing evaluative exercises.
82. Institutionally, the Ministry of Finance in Zambia is still developing compared to other
very strong centre of government Finance Ministries in other African countries (e.g.
Rwanda, South Africa). Consequently, it may not have the institutional legitimacy to be
able to implement, advocate for and maintain a government-wide evaluation function.
The challenges around the authority of the Ministry of Finance have been raised
explicitly and implicitly. Explicitly, Leiderer and Faust (2012: 167) state that “in contrast
to several of its counterparts in the region, the Zambian finance ministry is relatively
weak politically and, as a consequence, has not been able to involve the sector
ministries in an effective reform of public financial management.” Implicitly, the
challenges around exerting power from the Ministry of Finance can be identified in
ongoing expenditure management challenges. Although this judgement does not
directly extend to the M&E unit, it does raise questions about the institutional strength
of the Ministry. This is explored in Box 2.
Box 2: Evidence in the Budget Cycle
A review of the documents and announcements surrounding the budget cycle present a
differentiated picture. The Open Budget Survey of 2012, for example, rates Zambia in the
global bottom ten of its ranking with a score of 4 out of 100. Previously in 2010, Zambia
scored 36 out of 100. In the current budget cycle, it appears that improvements are being put
in place, which may facilitate improved entry points for evidence in the budget cycle.
The national budget cycle in Zambia has four main stages: a) budget preparation by the
Ministry of Finance based on submissions by ministries and departments and by various
stakeholders, including the civil society and private sector; b) budget authorisation or
approval by Parliament; c) budget execution by ministries and departments; and d) audit by
the Auditor-General’s office with follow-up by Parliament.
A number of documents to improve the budget process became publicly available in 2013. A
citizens’ budget has been put in place; public submissions to the budget process are being
; the MTEF, and the full activity-based budget, are available on the Ministry of
Finance’s website; and budget performance figures are available for the first quarter45
The national budget as prepared by the Ministry of Finance has the potential to be informed
by a range of recent surveys produced with the Central Statistical Office and development
partners, such as, the Census, Crop Survey’s and the Living Conditions Monitoring Survey
Reports. It was reported in interviews that analyses like the gender assessments have fed
into budgeting processes. The results of the Annual Progress Reports (APR) of the National
Development Plan are utilised in drafting the MTEF and the annual budgets, when available.
As noted in the earlier section there are civil society organisations that play a role, in
monitoring the implementation of the national budget.
However, there remains a gap in transparency around government expenditure until the
Auditor General report becomes available, up to 18 months after the end of the budget year.
This usually leads to additional budget votes to make expenditure legal.
During the preparation stage there is potential for the influence of evaluations. Those making
submissions such as the JCTR, Economics Association of Zambia, CSPR and the ZNFU have the
burden to prove their case to the Ministry of Finance. Past this initial stage, it is difficult to
track implementation. According to interviews there is some evidence of technical demand
for evaluation to feed into the budget process, for example, through the current round of
evaluations and research being conducted by the Ministry of Finance.
83. For line ministries and agencies there is no consistent system and mechanism to
evaluate policy performance beyond annual performance reviews, which mainly focus
on monitoring information. This study was unable to identify any electronic copies of
annual programme review documents although evaluations do confirm that they have
taken place46

46 Evaluation of Paris Declaration
84. The Ministries of Health and Education have recognised M&E systems. The Ministry of
Health has an active M&E Unit in the Planning Department which is also home to the
Health Management Information System (HMIS). However, the M&E unit is much
stronger on monitoring, which is undertaken monthly and reported on quarterly, than
evaluation. The Ministry of Health undertakes a Joint Annual Review with their partners
which have evaluative dimensions as well conducting mid-term reviews of its strategic
plans. HMIS enables the Ministry to collect data and information on three major
diseases, namely: malaria, tuberculosis (TB) and HIV/AIDS. The Ministry of Education
also manages the Education Management Information System (EMIS).
85. The mid-term and the end-term review/evaluation of a policy are seldom undertaken
except in situations where a development partner is nudging and financing. As a case in
point is the Ministry of Education HIV/AIDS Work Place Policy which was financed by
UNICEF. Table 6 highlights some of the evaluations that have been undertaken beyond
routine information collection. Box 3 explores the links between evidence and policy.
Table 6: Selected ministries and evaluative exercises undertaken
Source: Compilation by authors
Box 3: HIV and AIDS Response
In Zambia there is a history of developing an evidence-based response to HIV with work
between development partners, civil society and the government. This shows that
development partner led efforts can support development results and capacity development.
The development of the evidence base over the last 25 years has meant that the drivers of
the epidemic have been understood. HIV transmission in Zambia is primarily through
heterosexual contact and this mode of transmission is exacerbated by the high prevalence of
sexually transmitted infections (STIs), the poor socio-economic status of women and highrisk
sexual practices (such as age at first sex debut, multiple and concurrent partners and
condom use).
Evidence is then taken into practice through a consultative process that involves the National
AIDS Council (which is a form of Sector Advisory Group). For example, in November 2009,
stakeholders at the National HIV Prevention Conference reviewed the drivers of the epidemic
and these were included into the National HIV and AIDS Strategic Framework 2011 – 2015,
which included prevention and control of tuberculosis, treatment of other opportunistic
infections, access to anti-retroviral (ARV) drugs, utilisation of alternative and/or traditional
medicines, promotion of appropriate nutrition, support to the infected and affected, support
to orphans and vulnerable children as well as support to high risk and vulnerable groups
(disabled groups, commercial sex workers, prisoners, refugees and long distance truck
drivers). While all these measures have been implemented, their success rates differ from
one to the other with the most successful being provision of free ARVs through the ART
programme. Appreciable gains have been made in the area of HIV and AIDS with the
assistance of evidence.
However, there still remain some key challenges to accessing the many services that are
available in the country. Again evidence is available to help shape the changing nature of the
Ministry of Education Review of the Ministry of Education
HIV/AIDS Work Place Policy 2006-2011
Ministry of Health Zambia Country Report for the United
Nations General Assembly Special
Session (UNGASS) on HIV/AIDS
Global Fund review of the M&E System 2007
Annual Programme Reports
epidemic. For example, it is known that HIV and AIDS related stigmatization and
discrimination threaten the effectiveness of HIV prevention, care and support programmes.
Much of the HIV and AIDS work is funded by development partners yet it has led to a range of
research publications that have involved Zambians and helped build local capacities in
evidence-based participatory policy definition. This shows that development partner efforts
can be a pre-cursor to endogenous activities, for example, through the new Centre for
Excellence in M&E at the University of Zambia.
The Central Statistical Office
86. The Central Statistical Office, the custodian of official government data, has conducted a
range of important statistical surveys. Organisationally, it is a department that reports to
the Ministry of Finance. There is evidence of some of these products being used in public
debate and for government planning documents. However, the Office has capacity
issues and needs external assistance to complete products. The Office completes regular
inflation figures and GDP figures. Some of the surveys undertaken in the last four years
are listed in Table 7.
Table 7: Central Statistics Office surveys
87. Central Statistics Office gets both financial and technical assistance from development
partners to complete these surveys, particularly USAID, DFID, GIZ and Macro
International. While the Census received a large amount of government funding
(approximately USD 50 million) there was a DFID contribution of GBP 3.5 million.
88. Some staff members are provided with advanced training in statistics especially by
Macro International and Measure Evaluation. The M&E Centre of Excellence at the
University of Zambia’s Department of Population Studies also provides some training on
statistical approaches.
89. A number of challenges with the management of statistics were identified. During
interviews it was reported that datasets often required substantial work before they
could be used for analysis. This account is substantiated by other sources. For example,
the GDP indicator has been rebased. Further, in supporting the completion of the LCMS,
DFID reported that the 2006 data needed to be rekeyed. Additionally, issues around the
project management of both the LCMS and Census led to delays in the release of data. It
seems to be agreed that the Central Statistical Office needs long-term support, but the
extent to which this is in-country or remote needs further clarification.
90. In spite of the challenges there does appear to be demand from the Ministry of Finance
to improve and use statistical information. For example, it was reported that the rekeyed
data from 2006 was used in the formulation of the SNDP and the new data from
2010 was used in the final report of the FNDP. Both LCMS 2006 and 2010 have also been
used in the micro-macro simulator ZAMMOD, which is used for forecasting impact and
expenditure of major GRZ policies and programmes. Further the health surveys are used
in formulating development partner and the ministries responses. However, there is no
Survey Last time completed
Zambia Sexual Behaviour Survey 2009
Living Conditions Measurement Survey (LCMS) 2010
Population Censuses 2010
Zambia Demographic Health Survey 2013 (ongoing)
report of high-level political demand for using survey information or systems that entail
review by politicians.
3.3 Evaluation Agents
Think Tanks
91. In Zambia there is an interesting emerging array of think tanks conducting policy
relevant analysis in important and sensitive policy areas. The Zambia Institute of Policy
Analysis and Research (ZIPAR) undertakes good quality, policy relevant economic
analysis, linked to government agents and provide evaluative recommendations to
principals. ZIPAR’s focus is on the production of high quality analysis that feeds into
public debate and public education on economics. ZIPAR was set-up with support from
the previous MMD Government and only became operational in 2011, as the
government was changing, although it still received financial support from the public
funds and development partners into 2012. This formation led to the impression that it
was an MMD think tank. After the 2011 election when the Patriotic Front formed a
government a new Director was appointed and has been working to establish ZIPAR as a
non-partisan think tank
92. ZIPAR’s strategy is to build slowly. During interviews with ZIPAR it was emphasised that
they focus upon undertaking in-depth analysis rather than rapid reviews. This approach
is verified in their products, for example, they have produced detailed public analysis of
the auditor general’s reports, the 2012 budget, and of the implications for the reduction
in fuel subsidies for the poor. ZIPAR does undertake commissioned work. For example,
they undertook an analysis of the constituency development fund for the Presidency,
recommending and starting to develop an allocation formula. Further they have
presented work to the Committee of Estimates in Parliament on Mining Taxation.
93. Managing this range of work was reported to be challenging as the capacity on both the
demand and supply sides is limited. For example, although they are interested in
engaging in experimental and quasi-experimental impact evaluations, they have not had
the opportunity to do so, nor do they have the capacity to lead one. Other than the
Executive Director, who is a PhD holder, other staff members are at master’s degree
level and are having to learn quickly on the job.
94. ZIPAR reported that in their experience there are some pockets of demand for
evaluation, but the Government does not seem to be organised to actually commission
evaluation studies. Although in their work they have not undertaken many projects with
the Ministry of Finance, they recognise that there is political will within the Ministry for
supporting evidence in policy processes.
95. The Policy Monitoring and Research Centre (PMRC) is a relatively new think tank that
was incubated in the structures of the Patriotic Front with some core funding from DFID.
Since the 2011 election, it has received funding from the government. The composition
of its board includes Wynter Kabimba, the Secretary General of the Patriotic Front and
current Justice Minister. The management of the PMRC state that they have
independence from the government and the party and have on occasions produced
information that is critical of government performance and plans. For example, on the
implementation process of the SNDP. The PMRC has access to government agents and
political principals.
96. The PMRC generate produces Policy Analysis, Infographics, and a variety of special
discussions named Series and Hard Talk47
. Policy briefs have been presented to a range
of Ministers. Hard Talk is a recorded interview with Ministers where they are presented
with and questioned on various challenging pieces of information related to their
departments. PMRC appears to have good access to political principals and is able to
leverage these to enter into discussions.
97. PMRC, in contrast to ZIPAR, have focused on generating a large quantity of products on
a wide range of subjects. A lot of these products use cartoons and graphics as part of
their style of communication making them quite accessible. PMRC, similar to ZIPAR,
does not have postdoctoral researchers and relies on people with masters degrees. In its
work with government, PMRC had identified health and education as organised and
interested in using evidence.
98. The Indaba Agricultural Policy Research Institute (IAPRI) is a non-profit company limited
by guarantee and collaboratively works with public and private stakeholders. IAPRI exists
to carry out agricultural policy research and outreach, serving the agricultural sector in
Zambia so as to contribute to sustainable pro-poor agricultural development. Their
website currently lists seven researchers in addition to management structures. Their
main funding comes through USAID and SIDA. Michigan State University support their
research development. They publish technical, policy and working papers. Their research
products are often based on quantitative studies. During the recent food subsidies
debate their evidence featured prominently in the national media.
99. It is difficult to get an overall picture of the state of university research and education in
Zambia as the Higher Education Council is only now going to be set-up and the
universities do not keep records of their research output. In Zambia the Universities of
the Copper Belt and Zambia are the only ones to offer their own PhD degrees. The third
public university, Mulungushi, was founded in 2008 and is accepts only undergraduate
students. There are also a number of private Universities that offer postgraduate
qualifications either through linkages to public universities or foreign universities.
100.Zambian universities’ ability to produce research is limited. The Southern African
Regional Universities Association (SARUA) reported that in Zambia “the majority of
universities are characterised by low research output and few scientific publications (as
is common in most African countries)”. The SARUA 2012 profile of Zambian universities
noted that there were only 11 PhD students enrolled in universities. Out of 1024
university staff, only 255 (about 22%) have doctoral degrees (Kotecha et al. 2012).
Currently Zambia has only one active journal publication in the last year identified in this
study, the medical journal was last published in 201248. The Schimago Institutions
Ranking, that produces a ranking against some main dimensions of research
performance, rates only the University of Zambia. Currently the University of Zambia is
top of the country ranking, 51 in the Africa region and 1881. The University of Zambia
has increased its research output from 353 articles over 2004–2008 to 510 articles from

2007– 201149
. This is the only current collated information on research output in Zambia
(SARUA, 2012).
101. Against this background the “demand for higher education institutions is very high
and is far from being satisfied” (SARUA, 2012: 108). A recent World Bank report found
Zambian higher education to be regressive, only being open to a rich segment of the
population. The challenge with the sector was recognised by the President who noted
during his first opening of Parliament that “our universities and colleges do not only
have dilapidated infrastructure but are also faced with shortage of staff and apt teaching
and learning materials.”
50 A higher education bill has been introduced to Parliament to
help to address the coordination and resource challenges.
102. Although there are challenges, there are social scientists who can engage in
evaluation, specifically in health, agriculture and demography. The Institute of National
Economic and Social Research (INESOR) is active in research. INESOR has qualified staff
with PhD and masters level qualifications and it is headed by a professor who is the
director of the institute. Currently they undertake research on agriculture, economics,
governance, health, urban development and socio-cultural research51
. It is the research
and consultancy arm of the University of Zambia. INESOR has been involved in a number
of high profile evaluations including evaluation of some components of the FNDP.
103. Meanwhile, the M&E Centre of Excellence has received a sizable grant of USD 5.9
million over five years that can help develop the research and teaching based for
evaluation in the country. In achieving this aim the University has four short courses on
data analysis, epidemiology for data use, the integration of population variables to
development planning and planning monitoring and evaluation. In addition they have a
module for fourth year undergraduate students on M&E and are planning to launch a
Master’s degree in Monitoring and Evaluation.52

Evaluation Associations
104. The Zambia Evaluation Association (ZEA) has been in existence for close to 10 years.
Most of the members are independent evaluators. However, its membership base has
failed to expand over this period. ZEA is not established by an Act of Parliament and,
therefore, it has no powers to mete out or enforce professional conduct among its
members like other professional bodies. However, it has managed to establish links with
some funding agencies. For example, GIZ funded the Association’s strategic plan and
some monthly dissemination meetings. The African Capacity Building Foundation (ACBF)
has funded the Association to publish a book on M&E practice in Zambia from the point
of view of various disciplines, HIV/AIDS and poverty among others, but only one paper
has been written and so the funds would be surrendered back to ACBF. While ZEA does
not itself undertake evaluation it does so through its members. Most of the people
interviewed on the supply side are not members of the ZEA.
105. MESSY is a grouping of M&E practitioners who have come together to support each
other professionally and also to create a platform for sharing M&E information. They

facilitate some presentations from among members and non-members on M&E topics of
interest. ZEA and MESSY are merging to form one evaluation association. Both are
affiliated to the Africa Evaluation Association (AfrEA). Most members of MESSY are in
active employment implementing monitoring systems and working as managers of
evaluations. The membership bases of a merging could therefore be complimentary.
106. This report finds that in Zambia there are multiple entry points to improving
evaluation demand and supply. However, each entry-point needs to be considered in
terms of the aligned interest groups. There is active demand for evaluation within some
government ministries, especially the Ministry of Finance who could play a broader role
clarifying standards and expectations from government. Outside of government there
are well-connected organisations who seek to use evaluative information to engage in
advocacy. This demand is set within a context where there is a high degree of political
competition between political parties and various interest groups.
107. The supply of evaluation expertise in Zambia is diverse in its quality and needs
further development. In response to the historical demand from development partners
and locally commissioned activities (using development partner funding) a range of
consultancy companies and individuals have arisen with particular areas of strength. In
the health sector, for example, there have been a lot of resources put into the
generation of evidence, particularly in regards to HIV and AIDS. The university sector has
some research capacity, especially in the social science sector, but indications are that
there are structural challenges in the sector with qualified staff leaving for better paid
positions elsewhere.
108. This report finds that in Zambia there are multiple entry points to improving
evaluation demand and supply. However, each entry-point is partial and often
represents aligned interest groups rather than a neutral role-player seeking to expand
evidence-based practice. Some of these entry points are outlined below.
4.1 Demand
109. Currently there is very little actual demand from Principals outside of development
partners and to some extent civil society. Development partners are the main managing
and commissioners of evaluations in Zambia. Even the current evaluations based within
the Ministry of Finance rely upon external development partner funding. Civil society,
meanwhile produce useful analysis that feeds into policy debate. Latent and potential
demands for evaluations are discussed further below.
110. The Executive: The Cabinet Office process for policy submissions provides a potential
entry demand for evaluation. Further investigation needs to be undertaken of the
monitoring processes being requested by the Presidency as it is not clear how the
demand for information feeds back into accountability and performance. The Presidency
is both the centre of formal and informal power and demand for evaluation could have
an important effect that serves broader development outcomes if it is received at the
right time. Evaluations that touch on sensitive areas of resource allocation, such as,
infrastructure need to be undertaken with consideration.
111. Civil Society Organisations: There are a variety of pathways for evaluation work to
be undertaken with civil society. The pathway chosen depends very much upon how
commissioners of evaluation wish to position themselves in the political economy. For
example, evaluations could be supported with existing influential interest groups such as
the Farmers Union that seek to establish evidence for the benefit of their members, or
with those groups with have a more hostile stance with the current government. If the
concern is in undertaking evaluations that will be used and useful in the policy process
for the broader aims of development then supporting organisations like JCTR who
already conduct evaluative activities and have strong formal and informal legitimacy
may prove to a useful entry point.
112. The Legislature: Currently, the legislature provides a limited entry point for ECD
activities. Focused work on specific research issues of interest to parliamentarians
especially on strategic issues could be helpful to generate an overall orientation to
evidence-based practice. Further, the current hearings and submissions on policy
although not currently far reaching do provide potential entry points to introducing
evaluation through external organisations.
113. Development Partners: This group is currently supportive of evaluation through
funding and in-kind support. Getting sector working groups and SWAps functional will
help to interact with the demand-side as they provide a useful entry point for the
identification and management of evaluations. This kind of support would need to be
undertaken in alignment with the findings from the Paris Declaration Evaluation. On the
supply-side there could be a greater emphasis by development partners on
commissioning in-country think tanks by themselves or as part of their own evaluation
114. Ministry of Finance and Sector Ministries: The Ministry of Finance demonstrates an
actual demand for evaluation. They are currently undertaking evaluations and are
interested in setting-up an evaluation function. An issue with the demand is that it
requires development partner assistance to undertake the evaluations, both financially
and technically. Yet, the Ministry of Finance can play a key leadership role in evaluation
in accordance with the SNDP in terms of setting out the rules and incentives around the
conduct of evaluation. In doing this the evaluations could get greater resonance with the
Executive if they focused on priority areas and overlaps between the SNDP and the
Patriotic Front’s Manifesto.
4.2 Supply
115. Think Tanks: Working with the think tanks profiled in Zambia can provide an entry
point to undertake evaluations and also provide ECD services. Both provide access to
high-level decision makers through their boards and so could present evaluations or
undertake orientation to the importance of evidence in decision-processes. Both provide
different kinds of entry points: ZIPAR research quality products and PMRC rapid
turnarounds on that are well communicated. For both organisations, the ECD activities
would need to be sensitive to the politics surrounding the organisations.
116. Universities: The Centre of Excellence initiative at the University of Zambia provides
a well-resourced entry point to strengthening the supply of evaluation in the country.
Working with this project would provide synergies with existing funding and support
mechanisms. A challenge will be to support the initiative so that gains can be extended
past the immediate funding period. This might require the centre to conduct evaluations
as well as research on evaluation. There are broader issues around the capacity of the
university sector in Zambia to be strengthened so that it fulfils research objectives rather
than just providing consultancy opportunities. This would require longer-term
interventions. Potential areas that could be considered are those where research
capacity is required in the current SNDP, for example, infrastructure development.
117. Evaluation Associations: With the merging of MESSY and ZEA there is now a
boundary crossing community of evaluators. This provides a useful entry point for the
discussion and clarification of standards and competencies for evaluators and an
ongoing site of engagement around evaluation activities in Zambia especially the diverse
array of consultancy companies and individuals have arisen with particular areas of
Bandow, Doug 'Democracy Is Under Challenge in Zambia',
toRecentOpeds+(Cato+Recent+Op-eds)%3E, accessed 14 October 2013.
Bank, World 'World DataBank', <, accessed
14 October 2013.
Carlucci, Paul (2013), 'Tongue Tied: How Zambia’s Defamation Laws Silence Critics', Zambia Reports,
14 October 2013.
Cheelo, Caesar and Banda, Bernard (2012), 'What the Auditor General’s Office has been saying about
Public Financial Management in Zambia', (Lusaka: ZIPAR).
Chigunta, Francis and Matshalaga, Neddy (2010), 'Evaluation of the Implementation of the Paris
Declaration in Zambia', (Lusaka: UNDP).
Civicus (2010), 'The Status of Civil Soceity in Zambia: Challenges and Future Prospects',
(Johannesburg: Civicus).
Economics, Trading 'Zambia Exports', <,
accessed 14 October 2013.
Focus, Africa 'Zambia: Condemned to Debt',
<, accessed 14 October 2013.
FODEP, Foundation for Democratic Process (2011), 'The State of Democracy in Zambia', (Lusaka:
University of Zambia).
Front, Patriotic (2011), 'Patrotic Front 2011 - 2016 Manifesto', (Lusaka: Patrotic Front).
Geloo, Zarina 'Saying bye (elections) to democracy in Zambia',
accessed 14 October 2013.
GoZ, Government of Zambia (2005), 'Vision 2030', (Lusaka: Government of Zambia).
--- (2013), 'Draft The National Planning and Budgeting Policy', (Lusaka: Ministry of Finance).
House, Freedom 'Freedom of the Press 2013', <,
accessed 14 October 2013.
IMF, International Monetary Fund (2005), 'Second PRSP Implementation Progress Report',
(Washington: IMF).
Kasoma, F, Eribo, Festus, and Jong‚ÄêEbot, William (1997), 'Press Freedom in Zambia', Festus Eribo
and William Jong-Ebot. Press Freedom and Communication in Africa, 135-56.
Kemp, A, Faust, Jorg, and Leiderer, Stefan (2011), 'Synthesis Report: Between high expectations and
reality: An evaluation of budget support in Zambia (2005-2010)', (D.I.E).
Kotecha, Piyushi, Wilson-Strydom, Merridy, and Fongwa, Samuel (2012), 'A Profile of Higher
Education in Southern Africa', (Johannesburg: SARUA).
Leenstra, Melle (2012), Beyond the facade: instrumentalisation of the Zambian health sector (African
Studies Centre, Leiden).
Leiderer, Stefan and Faust, Jorg (2012), 'Evaluation of budget support in Zambia: Implementation,
direct effects and political economy', (Bonn: Deutsches Institut für Entwicklungspolitik).
Mason, Nicole M, Jayne, TS, and Mofya-Mukuka, Rhoda (2013), 'A Review of Zambia’s Agricultural
Input Subsidy Programs: Targeting, Impacts, and the Way Forward'.
MoF, Ministry of Finance Zambia (2009), 'Mid-Term Review of the Fifth National Developemnt Plan',
(Lusaka: Ministry of Finance).
NORAD, Norwegian Agency for Development Cooperation (2011), 'Joint Evaluation of Support to
Anti-Corruption Efforts: Zambia Country Report', (Oslo: NORAD).
Partners, Development (2013), 'Joint Assistance Strategy for Zambia 2011 - 2015', (Lusaka).
Phiri, Isabel Apawo (2003), 'President Frederick JT Chiluba of Zambia: the Christian nation and
democracy', Journal of religion in Africa, 33 (4), 401-28.
Porter, Stephen and Goldman, Ian (2013), 'A Growing Demand for Monitoring and Evaluation in
Africa', African Evaluation Journal, 1 (1), 9.
Rakner, Lise (2012), Foreign Aid and Democratic Consolidation in Zambia (United Nations University,
World Institute for Development Economics Research).
Ricker-Gilbert, Jacob, et al. (2013), 'What are the Effects of Input Subsidies on Maize Prices?
Evidence from Malawi and Zambia', (Michigan State University, Department of Agricultural,
Food, and Resource Economics).
Sata, Michael 'President Sata's Speech During the Official Openning of Parliament',
< Sata.html%3E, accessed
10 December 2013.
Times, Lusaka (2011), 'List of newly appointed Cabinet and Deputy ministers', Lusaka Times, 29
September 2011.
UNDP (2001), 'Human Development Report:Zambia', (Lusaka: United Nations Development
--- (2013), 'Millennium Development Goals: Progress Report Zambia', (Zambia: UNDP).
USAID (2011), 'Zambia: Country Development Cooperation Strategy 2011 - 2015', (Lusaka: USAID).
THE HOUSE ON 2 NOVEMBER, 2011', (Lusaka: Auditor General).
Zambia, Parliament of 'Latest Members List', accessed 14 October 2013.
Appendix 1: Sample Budget tracking tool

53 Changes with each year within the SNDP
Sector Programme
Budget (K)
Annual Budget/Allocations 201153 (K)
% of
Actual over
Comments (focus on
deviation from plan and
impact on sector programme)
First Half Second Half
Plan Actual Plan Actual
Regional Center Tag
Resource Type Tag
Language Type Tag
Image Thumb