Study on the Demand for and Supply of Evaluation in Malawi

Submitted by jyuan@worldbank.org on Tue, 04/12/2016 - 17:46
Image
a

This report provides a review of two dimensions of evaluation practice in Malawi: (i) the conditions under which demand for evidence is generated; and (ii) the areas in which supply can be strengthened to meet and foster this demand. The review serves to highlight the prevalence of active, latent and potential demands for evaluation. The latent and potential demands are nested within requests for evidence from principals and government agents. This demand is not necessarily only conditioned by development partners, but results endogenously from government, based on articulated development objectives. Supply could in the short-term be strengthened through work with the main research centres of the Universities of Malawi (Centre for Social Research, Economics Department) and Lilongwe University of Agriculture and Natural Resources.

The major thrust of the review is that, within Malawi, there is a stronger demand for evidence for policy contestation than there is for systematic evaluations. This demand is circumscribed by politics of patronage, except on matters of common destiny or clearly science dominated fields like health. Evaluation studies are, in contrast to other forms of evidence, seen as a tool applied to support accountability for management or political leadership and sometimes used to explore complex areas on performance, be it good or bad. On the whole, both evaluation supply and demand is very limited in the country.

International consultants usually lead major evidence gathering or evaluation exercises, with local consultants playing a support role or taking the lead for smaller in-country assignments. As a result of which, for there to be a growth in systematic evaluations, as a useful tool for development, capacity enhancement efforts need to emphasise that evaluation can support learning that enables policy makers and programme implementation agents to understand how public investments could be improved. The limited supply of evaluative expertise shows some legitimate and good practice, and it is mostly linked to country-led demand by government, local donors and non-governmental organisations. Within this context, there are some examples of evaluation being applied to inform policies and development strategies.

There are a number of individual entry points for improving evaluation capacities in Malawi within the framework of the evolved National Development Monitoring and Evaluation System (NDMES). The existence of functional sector working groups, involving a number of institutions both in demand and supply, is perceived to be a resource that can be used to enhance the governance of evaluations and its growth as a tool for development policy. Malawi’s central government uses a large array of donor-funded technical assistance to supply evaluative expertise. Some of the demand is endogenous and there are cases where evidence does feed into decision-making, especially on matters where substantial donor funds are at stake. It is unlikely that if donors withdrew resources, Government would identify substantial internal resources to support evaluation activities beyond programmes that have little popular support.

In undertaking evaluation capacity development (ECD), the central identifiable challenge from the review is the establishment of a coordinating framework or institutional mechanisms that would serve to foster evaluation capacity and be the target of genuine transfer of skills programs over the long term. These would strengthen the University Sector and other higher level technical institutions to deliver appropriate certified training in this field. The study brings to the fore the importance of shaping supply to fit with capabilities within the demand space and the importance of enhanced coordination in government and institutional development for evaluations within the wider society.

Pdf as plain
STUDY ON THE DEMAND FOR AND
SUPPLY OF
EVALUATION IN MALAWI





Centre for Learning on Evaluation and Results Anglophone Africa (CLEAR-AA)
Graduate School of Public and Development Management,
University of the Witwatersrand, Johannesburg
Drafting team: Dr. Hannock Kumwenda (Country Consultant), Salim Latib (CLEAR-AA/Wits)
Management Team: Stephen Porter, Osvaldo Feinstein, Salim Latib, Anne McLennan, David Rider Smith
Reference Group: Michael Bamberger, Derek Poate, Zenda Ofir, Robert Picciotto, Nidhi Khattri, Howard
White, Jessica Kitakule-Mukungu, Ian Goldman
December, 2013
Entire publication © Graduate School of Public and Development
Management, University of the Witwatersrand, Johannesburg
This report is an output funded by the UK Department for International
Development (DFID) as a public good. The views expressed are not necessarily
those of DFID.
i
STUDY ON THE DEMAND AND SUPPLY OF
EVALUATION IN MALAWI1

1
For further information on this study please contact CLEAR.AnglophoneAfrica@wits.ac.za. The content of this
study is the responsibility of the team alone, and should not be ascribed to the University of the
Witwatersrand, DFID, or any other organisation or individuals.
ii
EXECUTIVE SUMMARY
This report provides a review of two dimensions of evaluation practice in Malawi: (i) the
conditions under which demand for evidence is generated; and (ii) the areas in which supply
can be strengthened to meet and foster this demand. The review serves to highlight the
prevalence of active, latent and potential demands for evaluation. The latent and potential
demands are nested within requests for evidence from principals and government agents.
This demand is not necessarily only conditioned by development partners, but results
endogenously from government, based on articulated development objectives. Supply could
in the short-term be strengthened through work with the main research centres of the
Universities of Malawi (Centre for Social Research, Economics Department) and Lilongwe
University of Agriculture and Natural Resources.
The major thrust of the review is that, within Malawi, there is a stronger demand for
evidence for policy contestation than there is for systematic evaluations. This demand is
circumscribed by politics of patronage, except on matters of common destiny or clearly
science dominated fields like health. Evaluation studies are, in contrast to other forms of
evidence, seen as a tool applied to support accountability for management or political
leadership and sometimes used to explore complex areas on performance, be it good or bad.
On the whole, both evaluation supply and demand is very limited in the country.
International consultants usually lead major evidence gathering or evaluation exercises,
with local consultants playing a support role or taking the lead for smaller in-country
assignments. As a result of which, for there to be a growth in systematic evaluations, as a
useful tool for development, capacity enhancement efforts need to emphasise that
evaluation can support learning that enables policy makers and programme implementation
agents to understand how public investments could be improved. The limited supply of
evaluative expertise shows some legitimate and good practice, and it is mostly linked to
country-led demand by government, local donors and non-governmental organisations.
Within this context, there are some examples of evaluation being applied to inform policies
and development strategies.
There are a number of individual entry points for improving evaluation capacities in Malawi
within the framework of the evolved National Development Monitoring and Evaluation
System (NDMES). The existence of functional sector working groups, involving a number of
institutions both in demand and supply, is perceived to be a resource that can be used to
enhance the governance of evaluations and its growth as a tool for development policy.
Malawi’s central government uses a large array of donor-funded technical assistance to
supply evaluative expertise. Some of the demand is endogenous and there are cases where
evidence does feed into decision-making, especially on matters where substantial donor
funds are at stake. It is unlikely that if donors withdrew resources, Government would
identify substantial internal resources to support evaluation activities beyond programmes
that have little popular support.
In undertaking evaluation capacity development (ECD), the central identifiable challenge
from the review is the establishment of a coordinating framework or institutional
mechanisms that would serve to foster evaluation capacity and be the target of genuine
transfer of skills programs over the long term. These would strengthen the University Sector
and other higher level technical institutions to deliver appropriate certified training in this
field. The study brings to the fore the importance of shaping supply to fit with capabilities
iii
within the demand space and the importance of enhanced coordination in government and
institutional development for evaluations within the wider society.
iv
Table of Contents
EXECUTIVE SUMMARY ..................................................................................................................................II
TABLE OF CONTENTS ...................................................................................................................................IV
ABBREVIATIONS AND ACRONYMS................................................................................................................................... V
1. INTRODUCTION .......................................................................................................................................... 1
1.1 METHODOLOGY ..........................................................................................................................................................2
2. DEVELOPMENT CONTEXT....................................................................................................................... 4
2.1 DEVELOPMENT STRATEGY AND PLANNING............................................................................................................5
2.2POLITICAL ECONOMY OF MALAWI...........................................................................................................................7
3. MAPPING OF EVALUATION IN MALAWI............................................................................................. 9
3.1 PRINCIPALS .................................................................................................................................................................9
The Political Executive.....................................................................................................................................................9
Legislative Structures....................................................................................................................................................10
Development Partners...................................................................................................................................................11
Civil Society ........................................................................................................................................................................12
3.2 GOVERNMENT AGENTS .......................................................................................................................................... 13
Lead Ministry - Economic Planning and Development..................................................................................13
Line Ministries...................................................................................................................................................................16
National Statistical Office............................................................................................................................................19
3.3. SUPPLY OF EVIDENCE AND EVALUATION AGENTS............................................................................................. 20
Professional suppliers....................................................................................................................................................20
Independent think tanks and research institutions........................................................................................20
Universities.........................................................................................................................................................................21
Evaluators and evaluation associations...............................................................................................................22
4. PATHWAYS, OPPORTUNITIES AND CHALLENGES ........................................................................24
4.1 BUILDING THE DEMAND FOR EVALUATION......................................................................................................... 25
4.2 STRENGTHENING EVALUATION THROUGH GOVERNMENT AGENTS ................................................................. 26
4.3 BUILDING THE WIDER SUPPLY SYSTEM ............................................................................................................... 27
REFERENCES...................................................................................................................................................28
APPENDIX A: INTERVIEW LIST ................................................................................................................30
v
Abbreviations and Acronyms
AfDB African Development Bank MoAI Ministry of Agriculture and Irrigation
CABS Common Approach to Budget
Support
MP Member of Parliament
CARD Centre for Agriculture
Research and Development
MPRS Malawi Poverty Reduction Strategy
CIHD Centre for International
Health and Development
MTEF Medium Term Expenditure Framework
CLEAR-AA Regional Centre for Learning
on Evaluation and Results for
Anglophone Africa
NCTP National Cash Transfer Programme
CPAF Common Performance
Assessment Framework
NDMES National Development Monitoring and
Evaluation System
CSOs Civil Society Organisations NGOs Non-Governmental Organisations
CSR Centre for Social Research NSO National Statistics Office
CWIS Core Welfare Indicators
Questionnaire Survey
ODA Overseas Development Assistance
DA District Assembly OPC Office of the President and Cabinet
DHS Demographic Health Surveys OSSREA Organisation for Social Science Research
in Eastern and Southern Africa
DIFD UK Department for
International Development
PEFA Public Expenditure and Financial
Accountability
DPs Development Partners PER Public Expenditure Reviews
DRC Democratic Republic of Congo PETS Public Expenditure Tracking Surveys
ECD Evaluation capacity
development
PFM Public Financial Management
EU European Union PRU Policy Research Unit
FISP Free Inputs Subsidy
Programme
PSIP Public Sector Investment Programme
GoM Government of Malawi SDU Service Delivery Unit
IHS Integrated Household Survey SP Starter Pack
IMF International Monetary Fund SWG Sector Working Groups
M&E Monitoring and evaluation TBAs Traditional Birth Attendants
MAMP Malawi Aid Management
Platform
TIP Targeted Inputs Program
MASEDA Malawi Socio-Economic
Database
TWC Technical Working Committee
MDGs Millennium Development
Goals
UN United Nations
MDHS Malawi Demographic and
Health Survey
USD United States Dollars
MEJN Malawi Economic Justice
Network
WB World Bank
MEMP National Development
Monitoring and Evaluation
Master Plan
MEPD Ministry of Economic
Planning and Development
MGDS Malawi Growth and
Development Strategy
1
1. INTRODUCTION
1. This study provides an analytical mapping of the agents involved in evaluation practice
and hence of the evaluation system in Malawi. It is inclusive of those agents involved in
establishing demand for evaluation, the actors involved in connecting demand with
supply, and those active in the supply of evaluations. The review and report has been
undertaken to serve as a reference point for those engaged in supporting evaluation and
evaluation capacity enhancement in the country. Consequently, the primary audience
for this study is intended to be those interested in evaluation capacity development
(ECD) in Malawi. The Regional Centre for Learning on Evaluation and Results for
Anglophone Africa (CLEAR-AA)2
conducted this study on evaluation/evaluative research
for the UK Department for International Development (DFID) as one of a set of cases
covering five countries3
. Fieldwork for this study took place from 17 to 21 June 2013.
2. A secondary audience for the report are individuals and stakeholders who have an
interest in the development of country-wide evaluation systems. The analysis in this
report presents a range of opportunities for capacity development in relation to
government agents (central and line departments), the evaluation agents (consultants,
think tanks and universities), and principal agents (development partners, Parliament,
the executive and civil society). In undertaking the mapping of the evaluation context
the study has identified latent, potential and actual demand, the conditions under which
demand is generated and potential sources of supply within the evolving political
economy of the country.
3. This study refers to evaluation as covering both evaluation and evaluative research and
therefore uses the terms interchangeably. The primary objectives of the study are to
explore:
1) The conditions under which demand is generated for evaluation evidence; and
2) The areas in which evaluation supply can be strengthened to meet and foster this
demand.
The guiding questions that interface with these objectives are:
(i) On the demand side:
a) What has been the actual demand for evaluation from principals?
b) Where is there latent and potential demand for evaluation?
c) How is evaluation demanded in the current organisational arrangements?
(ii) On the supply side:
a) What is the range and capacity of entities supplying evaluation services?
b) How relevant are the managers and producers of evaluation to the actual
demand for evaluation?
(i) On matching evaluation supply and demand:
c) Where can evaluation supply (actual, latent and potential) be
strengthened so that it meets and fosters demand?
4. This report finds that there is latent endogenous demand for evidence in Malawi. Supply
is generally limited even though there are a number of local organisations which have

2CLEAR-AA is based at the Graduate School of Public and Development Management at the University of the
Witwatersrand in Johannesburg. CLEAR-AA aims to enhance development anchored in learning, evaluation and
results.
3Beyond Malawi, the other case countries are Ethiopia, Ghana, Rwanda and Zambia.
2
undertaken good evaluation practice. Development partners largely support evaluation
exercises, but there is growing evidence that Government is increasingly willing to
support internally driven evidence gathering exercises. On the basis of the evidence, it is
concluded in this study that there is a stronger demand for policy related evidence,
especially quantitative, than there is for systematic evaluations. International rather
than local resources are the main funding conduit for evaluation studies, even where
there is strong government ownership. Evaluation activities are perceived to be a tool
for investigation on resource utilisation to support external accountability, rather than
as a tool for internal policy construction and development priority setting. Malawi is
perceived to be open to the conduct of any type of evaluation, seemingly as these
exercises come with further aid flow. However, in all of these, the central challenge is in
the uptake of results from evaluation. As a result of this, it is widely concluded that for
evaluation to be useful ECD efforts need to emphasise that evaluation promotes the
identification of better policies to accelerate development results, and also enables
managers or other implementing agents to be constructively engaged with.
5. The supply of expertise for quality evaluations is rather diffuse in the country, with
several examples of legitimate and good practice at the individual and corporate level.
The institutional capacity of Universities is largely reflected in the work of individual
academic and has generally been driven by active resource related demands. Within
these arrangements, there are examples of supply being successful at informing strategy
and in some instances wider policy interventions.
1.1 Methodology
6. This study was carried out through a combination of desk review, including an analysis of
existing evaluation products, and direct semi-structured interviews with a selection of
informants from across stakeholder groupings identified as central for the study. The
study methodology encompassed the following overlapping stages: i) establishing study
commitment and support from key stakeholders; ii) collating and analysing primary and
secondary data and information of the evaluation system (including available academic
and popular literature); iii) conducting a series of interviews with actors that fall within
the space established through the broad conceptual map; and iv) producing a draft
paper. Each of these stages is discussed in more detail below.
7. Establishing support from key stakeholders: Given the nature of the study, an
important initial step was to identify relevant national stakeholders who would be
engaged with prior to, during and after the collection of data. Whilst the study was
conducted independently, it is helpful that there is some level of active buy-in from key
stakeholders to support the use of the study. In Malawi, letters were sent to the
Government via the DFID Country Office to ask for appointments with senior
government officials, while the country researcher made additional contact with a range
of stakeholders at the technical level.
8. Collating and analysing secondary data and information: The collation and analysis of
secondary data covered policy, academic and grey literature relating to the political
context and the demand and supply side of evaluation. Included in this were studies on
evaluation initiatives within government and the supply that emanates from outside of
government. Following the country research phase further primary and secondary
3
documentation was referred to in order to substantiate the claims of the interviews and
further develop findings.
9. Interviews with key informants: A series of interviews were arranged with key incountry
stakeholders. The design of these interviews drew upon the literature review.
Issues of potential and latent demand and the ability of supply to invoke demand were
explored through the interview process. Data collection took place through semistructured
interviews that allowed people to narrate their story – with some probing
taking place based on the guiding questions. The data from interviews was analysed
during the field-work with emerging conclusions refined through synthesis.
10. Production of a draft and final country report: Following the completion of the first
draft of this study, the reference group reviewed the study report internally and
externally. During these processes adjustments were made to the report to aid the
clarity and accuracy of the core findings. Table 1 details the representatives of
stakeholders that were interviewed in Malawi.
Table 1: Interview Respondents
Government Agents Evaluation Community Principals (representative)
Ministry of Health Centre for Social Research Parliament –Budget and
Finance/Public Accounts
Committee
National Statistical Office Centre for Agriculture
Research and Development
Malawi Economic Justice Network,
PLAN Malawi, Water Aid, Tilitonse
Program
Ministry of Economic
Planning and
Development
Private Consultants
engaged in M&E
Development Partners (DFID,
Norwegian Embassy, African
Development Bank)
Ministry of Finance Economics Department The Office of the President and
Cabinet
National Aids
Commission
The United Nations Development
Programme, UNICEF
Source: Author
11. This report is structured as a product that stands alone from the overall research
synthesis and four other cases4
. The findings of the country case of Malawi are
presented in the following manner: First, the Malawian development context is
described in relation to evaluation; second, the state of the current evaluation demand
and supply is mapped; third, illustrations of how the development context interacts with
evaluation supply and demand are detailed; and finally, pathways to improve the
national evaluation context are suggested.

4
Those interested in a more detailed background, definitions of terms and the full methodological approach
should read the Inception Report (22 April 2013).
4
2. DEVELOPMENT CONTEXT
12. Malawi is a landlocked country with a surface area of approximately 118,484 km2
. The
estimated population of Malawi is 15.4 million (National Statistics Office, 2008). Over 80
percent of the population live in rural areas depending on subsistence farming for their
livelihood. Even though the country is one of the least urbanised countries in the world,
it has one of the highest urbanisation rates at 6 percent per annum (NSO, 2008).
13. Malawi’s economy has grown steadily since 2006, by an annual average of 7 percent per
annum (Malawi Government: Ministry of Economic Planning and Development, 2012).
This growth is largely attributed to investment in agriculture which accounts for nearly
34 percent of GDP (Reserve Bank of Malawi, 2012). The Free Inputs Subsidy Programme
(FISP) was the main driver for agricultural growth. Mining, construction and
manufacturing also grew strongly during this period of relative macro-economic stability.
Since 2010, the economy slowed down with real GDP growth declining from 6.5 percent
in 2010 to 4.3 percent in 2011 and 1.9 percent in 2012, mainly reflecting a contraction in
agricultural output (Reserve Bank, 2012). Adverse weather conditions in parts of the
country resulted in a decline in the output of maize (the main staple crop). Tobacco
output, the main export earner, also contracted sharply in 2012 reflecting poor price
incentives associated with the period of marked overvaluation of the exchange rate.
14. Malawi is one of the poorest countries in the world with a Human Development Index
(HDI) ranking of 171 out of 187 countries (United Nations Development Programme:
Human Development Report, 2012). Poverty remains high, widespread and
concentrated in rural areas. The poverty level is estimated at 50.7 percent, only
marginally down from the estimated 52. 4 percent in 2005 (NSO, 2004 and 2010).
Income remains very low and inequality has been increasing from 0.34 to 0.45 between
2005 and 2010. Poverty in rural areas is estimated at 56.6 percent relative to 17.3
percent in urban areas (Reserve Bank of Malawi, 2012). There are internal regional
variations in the distribution of poverty amongst the country’s three regions, with the
country’s centre ranked richest and the south as poorest.
15. Official development assistance (ODA) accounted for 40 percent of government revenue
in the 2010/11 fiscal year and 19 percent of GDP (International Monetary Fund, 2012).
Prior to the more recent reforms, the budget policy implemented in 2010 and 2011 was
expansionary, with a large budget deficit. In the 2011/12 fiscal year, the Government of
Malawi (GoM) instituted fiscal discipline measures by increasing the tax level and
domestic borrowing. The overall fiscal deficit widened from nearly 3 percent of GDP in
2010/2011 to an estimated 7 percent in 2011/12, with domestic financing rising from 1.7
percent of GDP to 5.6 percent in respective years (IMF, 2012). Table I below reflects the
general budget and donor contributions since the 2008-09 financial years.
16. Whilst the larger share of aid is delivered as direct project support, disbursement of
budget support has grown substantively, from USD 67 million in 2004 to USD 164 million
in 2009 (Resnick, 2012:6). Overall Overseas Development Assistance (ODA) in 2010/11
was estimated to be just under USD 800 million, with ODA accounting for more than 40
percent of government expenditure. Principal donors include DfID, European Union (EU),
USAID, China, the World Bank (WB), African Development Bank (AfDB), Norway, Ireland,
Germany and the United Nations (UN). The Aid provided is concenntraed in areas of
5
basic services and infrastructure development. Beyound some selected areas of pool
funding, donors tend to concetrate in areas of choice.
Table 1: Trends in FDA (US$ Millions)
Year Domestic
revenue
Total
expenditure
Gap External assistance and
loan Average
exchange rate
(USD/kwacha) External
assistance
Foreign
Loan
2008/9 950.4 1,751.8 801.4 539.0 92.2 0.0070
2009/0 1,240.0 1,713.3 473.3 426.7 46.7 0.0067
2010/1 1,333.3 1,897.4 564.1 403.8 70.5 0.0064
2011/2 727.3 993.9 266.7 48.5 54.5 0.0030
Source: IMF Country Report No. 12/221(2012) and No. 13/131 (2013). (Conversion into USD by authors)
2.1 Development strategy and planning
17. In 1999, the GoM (with assistance from UNDP and the World Bank) established Vision
2020 as a long-term developmental plan for Malawi. This process was followed by the
adoption of medium-term (three to five years) development strategies beginning with
the Malawi Poverty Reduction Strategy (MPRS) covering the 2002-05 period. Thereafter,
Malawi’s second generation PRS was renamed the Malawi Growth and Development
Strategy (MGDS) and covered the period 2006 to 2011. As of to date, the country has an
MGDS II covering the period 2011-2016.
18. The MGDSs are generally held to be more focussed on economic growth than the MPRS
and developed with a more results orientation. Targets and priorities are established
within the framework with linkages to the Millennium Development Goals (MDGs). The
MGDS II is generally used for the purposes of budget allocation and hence serves to
shape donor contributions, as reflected in the Medium Term Expenditure Framework
(MTEF).The MGDS contains a results matrix, detailing long-term as well as policy actions
and medium term goals.
19. Even though the MGDS serves as the framework for shaping budget allocation and
donor contributions, there are been concerns about the extent to which allocations
reflect actual implementation across government (Malawi Government: Ministry of
Economic Planning and Development, 2009). In addition to gaps between commitment
and actual contribution by government, adjustments often emerged because of revenue
overestimations and the impact of the exchange rate. A notable feature of planning
since the transition to democracy has been the multiplication of policy documents at the
sectoral levels and absence of real implementation beyond the very short term (Booth,
D, Cammack, D, Harrigan, J, Kanyongolo, E, Mataure and Ngwira, N (2006)).
20. Malawi has a framework for monitoring and evaluation that has only been partially
activated, in the sense that some coordinating institutions and information systems are
not working as was anticipated during the planning and design process. The system was
designed in 2004 as part of the MPRS implementation process and has its roots in the
national planning and budgeting systems. The budget system still has its own monitoring
system which has linkages to the national system. Prior to this, national planning and
6
budget systems had their own standalone monitoring systems. The national framework
has no legal instrument to support implementation.
21. At a formal level, and in response to PRSP requirements and some pressures from
donors on improved reporting systems, the GoM has been working on the development
of an overall M&E system since 2002. Prior to 2002, M&E was not specifically engaged
with in an organised manner (Chirwa 2004: 2). The National Development Monitoring
and Evaluation Master Plan (MEMP) first appeared in 2002 and was finalised towards
the end of 2004.The National M&E Framework is shown in Figure 1.
Figure 1: National M&E Framework
Source: Author based on the National Development Strategy Monitoring and Evaluation Master plan (2008).
22. Progression towards the implementation of a coherent framework for M&E has been
slow, although there have been periods of strong movement especially during MGDS 1.
Figure 1 is a diagrammatic representation of the system, as it was meant to work, rather
than what has in practice transpired. For example, the National Stakeholders Forum has
only been convened twice since the system was operationalised. Reporting to
Parliament is, in practice, not based on oral presentations, but unfolds through the
NATIONAL
STAKEHOLDERS
FORUM
NATIONAL CABINET NATIONAL ASSEMBLY
PRINCIPAL SECRETARIES FORUM
TECHNICAL WORKING COMMITTEE
SECRETARIAT
MINISTRY OF ECONOMIC PLANNING
AND DEVELOPMENT
SECTOR WORKING
GROUPS (SWAP)
LINE MINISTRIES NATIONAL
STATISTICS
OFFICE
DISTRICT
EXECUTIVE
COMMITTEE
DISTRICT
ASSEMBLIES
CIVIL SOCIETY
ORGANISATIONS
COMMUNITIES
MINISTRY OF LOCAL GOVERNMENT
AND RURAL DEVELOPMENT
7
circulation of published annual reviews of the national development strategy and
monitoring reports of individual programs. Sector Working Groups (SWGs) are a
relatively new addition (except the Health Sector one) and have been adopted for all
public sectors where development partners are active.
23. The central and most consistent outputs of the NDMES have been regular annual
reviews of the National Development Strategy, which is used to inform the planned
budgets for the following year. It is generated on the basis of resources allocated by
donors and government, for data collection on sector indicators, for the conducting of
consultative workshops to review the results and to write up the annual reports. A
further analytical output from the system is the generation of Public Sector Investment
Program (PSIP) reports between 2006 and 2008, the annual progress reports on MDGs,
Public Expenditure Tracking Surveys (PETS) in the big Ministries of Education, Transport
and Health. These reports are not widely circulated but used by the principal agents in
the respective Ministries and the Ministry of Finance in effecting budget and
procurement changes, and for allocating the national budget.
24. Figure 1 shows the framework for both monitoring and evaluation, but for the purposes
of evaluative research not all its components apply. Lower level institutions, covering
districts and communities, are generally not seen as part of the overall evaluation
system. This is not to say that evaluation does not involve these groups, but that its
demand elements seem to be currently limited to the higher levels of line Ministries and
other national level bodies, including NGOs. According to the Ministry of Economic
Planning and Development (MEPD), the evaluation dimension in M&E is largely
underdeveloped and hardly goes beyond the Principal Secretaries and National
Stakeholders Forum. Beyond that, where executive power rests (President and
Ministers) what matters most is simply to show evidence on the performance of policies
and programs. The line Ministries is where the Sector Working Groups are based and
these form the most active hub for evaluative research in addition to routine monitoring.
The Principal Secretaries Forum was only activated some six years ago but it has been
inconsistent ever since, and the National Stakeholders Forum has only been held once in
2008 (according to MEPD).
2.2Political Economy of Malawi
25. Malawi re-introduced multi-party democracy in 1993. Malawi started off with multiparty
democracy at independence in 1964 but this was quickly suppressed by Dr
Hastings Banda’s party within two years of winning elections. Malawi was ruled by a
single party government lead by Dr Hastings Banda for 30 years. The period under Dr
Banda is generally perceived as one with policy consistency and a high level of
professionalism in the public service (Booth et al 2006; Chirwa et al 2008). This is
contrasted with the last few years of Dr Banda’s rule and the term of office of Bakili
Muluzi (1994-2004). During this period there was no policy consistency reflected in rapid
change and frequent clashes between government policy preferences and contrary
donor approaches and perspectives (Harrigan 2003).
26. It was during Muluzi’s second term of office that Malawi adopted the Malawi Poverty
Reduction Strategies (MPRS) under the influence of International Monetary Fund
(IMF)/WB conditionality and the donor community. The adoption of the PRSP was a
move towards greater policy coherence and the opening of spaces for evaluation and
8
research, but continuing patterns of patronage politics suggested that evidence did not
really matter in the policy and decision making process (Booth et al, 2006). Whilst there
was no open hostility to receiving evidence, there was reportedly little chance that the
political leadership would learn from it and change established patterns of resource
distribution. During this period, the IMF described Malawi as willing to implement policy
reforms but unable to implement them due to lack of political will and capacity
constraints (IMF, 2004).
27. The election of President Mutharika in 2004 and the initial appointment of qualified
individuals within his Cabinet, raised hopes for the role of research, professional
expertise in policy design and implementation. It was during this initial period that there
were investments into systems for evidence-based policy making. Furthermore, the
surveys of the NSO became more regular and encompassing. An M&E system was
introduced in the central ministries and encompassed a framework to monitor and
evaluate development progress.
28. It is held by stakeholders that whilst the Muluzi regime did not care much for evidence,
the Mutharika regime did care but tampered with the results of surveys conducted. For
example, Mutharika failed to release the complete results of the third Integrated
Household Survey (IHS 3) by NSO.Outside of changes in political leadership, the
mentality of a single, dominant political party, and a centralisation of power continues
to be the norm. The value and relevance of evaluations and evidence for policy purposes
is constrained by the realities of a political culture of patronage based decision-making,
within which the focus is on the distribution of spoils, rather than informed policy
construction and the functioning of formal state structures(Joala 2012: 62).
29. The politics of making policies on the podium5
has generally been evident in all these
regimes since the transition to multi-party democracy. One consequence of the deep
patterns of patronage is the gap between formally articulated policy perspectives and
actual implementation practices (Booth et al 2006). Key initiatives are often adopted
without any serious consideration of their viability and personality politics tend to
prevent coordination. The general view is that politics embodied in patrimonial practices,
disempower the civil service and progressively undermine the capacity to generate
coherent technically grounded policy approaches.
30. Since coming into power following the death of Mutharika in April 2012, Malawi’s new
President Joyce Banda, has taken steps towards adjusting policies and opening the space
for M&E and more open policy construction. She has strengthened monitoring and
evaluation within the Office of the President and Cabinet by establishing a unit for
performance management. However, there are many who hold that she is equally
constrained by a deeply engrained culture of patronage. A key implication of the level of
patronage politics is the loss of policy autonomy, reflected in an inability of public
officials to carry out policies in accordance with public interest and to learn evidence
from analytical research and evaluation. Whilst there are some unfolding steps to
enhance policy capacity within the Presidency, the extent to which this would serve to
overcome an entrenched and diffused culture of patrimonialism is difficult to determine.
31. Although patronage runs deep in the politics and policy process, actual patronage
actions appear to be limited to areas that satisfy the largely economic and power

5 On the trail of Presidential public meetings and events.
9
interests of the ruling party. The general incentives for patrimonial action are thought to
be very low in some areas, such as health and education delivery. In practice, there are
also a number of competing patronage networks and which are amenable to the use of
any evidence to discredit policies and practices. This has happened in Parliament, where
those in opposition have appealed to findings from academic research to discredit the
policies of those in power.
3. MAPPING OF EVALUATION IN MALAWI
32. This section presents the mapping of evaluation within Malawi for both supply and
demand. Given the high level of ODA, nearly 40 percent of the budget in recent years,
the demand for evaluation is primarily rooted in the work of Development Partners
(DPs) and is evident in sector based interactions. Outside of a coordinated approach in
practice, donors have initiated evaluations and related research for accountability
purposes and as demanded by their own constituencies. Whilst some studies and
evaluations are initiated in partnership with the GoM, generally these take place outside
of a formalised process of feedback and generally unfold with the cooperation of
individual public servants in a diffused system where larger buy-in and appreciation has
not been fully established.
33. The descriptive overview that follows seeks to provide a more detailed analysis of the
actors involved and their capacities for managing evaluations, conducting evaluations
and using evaluations. The role and efficacy of stakeholders that express a demand for
evaluation or have latent or potential demand capacity (legislative structures, the
political executive, development partners and civil society) is explored in some detail.
The areas of demand are then followed by an analytical overview of those institutions
that are central to establishing a bridge between supply and demand - line Ministries
and the National Statistical Office. Embodied within the linking structures is also the
crucial role played by coordinating bodies. The final part focuses on those stakeholders
active in the supply of evaluation (think tanks, universities and non-governmental
evaluation organisations).
3.1 Principals
34. Demand within the political system is diffused and largely embedded within joint
programme implementation frameworks or within specific sector working group (SWG)
based interaction amongst donors and relevant government officials. Although the
NDMES Master Plan suggest a level of coherence in approach and the basis for active
demand, the evolving system reflects a combination of emergent and inconsistent
central push, coupled with diffused pulls based on energies, incentives and resources of
different actors within the wider pool of stakeholders.
The Political Executive
35. Malawi has a presidential type of political system, where Ministers, Heads and Deputy
Heads of Ministries are appointed by the President and are answerable to them. One
consequence of this is that the survival of Ministers and other senior government
officials depends on ‘toeing the line’ any dissent to the President’s wishes may result in
job loss. Furthermore, the centralised nature of political parties and party political
contestations creates an environment where the party leader’s priorities become party
10
policies. This tendency has historically served as a major constraint on constructive
criticism within the Executive and the willingness of Ministers and top level officials to
draw on evidence to change policies set by the ruling government. Policy direction is
very much centralised in the person of the President: any divergence from this scenario
may be due to the personality of the President rather than substantive positions. The
country’s Cabinet has not been recognised as a source of evaluation demand: at most
Ministers would encourage evaluations within their own portfolios as a means for
demonstrating performance.
Legislative Structures
36. The role and value for evaluation in legislative structures must be seen in the context of
past, current and future patterns on policy engagement within political parties. A recent
study by Mpesi and Svasand (2012:20) demonstrates that policies, ideology or
ideological perspectives are not significant elements in the existence or formation of
political parties. The party system is fragmented with individuals changing from one
party to another without substantive policy or ideological justification. Generally,
Malawian political parties remain entirely associated with the personalities of their party
leaders and offer little differentiation on policy issues (Resnick 2012: 3).
37. The fragmentation of party politics and the drive towards establishing parties for
positioning purposes often renders the focus on policy issues weak. Individual members
of parliament (MPs) engage on issues that impact on their constituencies, but not on
larger policy issues that affect the whole country. This weakness has been noted by
numerous actors and has reduced the value of legislative structures for policy relevant
evaluations and research. In some cases, this lack of appreciation is explained by limited
education and policy exposure amongst some of the serving MPs.
38. The National Assembly has historically relied on infrequent briefings by government and
non-governmental organisations (NGOs) on the performance of development policies
and their implementation. In practice, the scrutiny of the government budget, bills and
other public sector policies by the legislature has only been taken seriously when the
ruling party is in the minority in the National Assembly, like, for example, during Muluzi’s
and Mutharika’s first term. The lack of strong oversight by the National Assembly is
attributed to the absence of research capacity, limited information flows and
inadequate sitting time. Until the more recent construction of a new Parliament building
around 2008, parliamentarians did not have a permanent place to convene (Resnick
2012:7).
39. In theory, there is an expectation that Parliament will engage with polices and be a
source of demand, as it exercises oversight over implementation. Some level of
engagement does take place on the budget, although indications are that the time given
for debates is inadequate and would require changes in the manner in which the budget
is constructed. Further, there are indications that the Public Accounts Committee of the
Assembly engages periodically with information presented by Departments through the
audit process. Generally, however Parliament is considered weak with limited internal
technical capacity for policy engagement.
40. A number of donors have attempted to support the strengthening of Parliament, but the
support has not been consistent, in part due to the governance challenges between
11
2010 and 20126
. Whilst there are indications of possible support to strengthen the role
of Parliament in the oversight process and hence, by implication, on evaluations, the
reality of limited funding flows and donor dependence establishes a range of constraints
like inability to conduct regular meetings, seek technical inputs, among others. As large
portion of foreign aid comes either in the form of grants or off-budget support, which
does not require Parliamentary approval, it becomes difficult for MPs to engage in
pertinent evaluation processes.
Development Partners
41. DPs have historically been the main source of demand for evaluations and for evidence
based policy making. However, in practice this demand is mediated either through the
direct commissioning of studies for accountability purposes or through coordinative
engagements within established sector level structures. Development partners have
over the past decade initiated many independent studies on areas that they are
supporting. In 2006, external partners listed 27 analytical reports on the Country
Analytical Works website. As there is no repository for evaluations completed or
analytical studies, there is no point of reference to determine the coverage, quality and
number of studies completed. Box 1 demonstrates donors’ role in evaluation studies.
Box 1: National Cash Transfer Programme (NCTP)
Donors have a strong role in influencing policy but they do not always commission studies to
suit only their accountability or reporting requirements. The engagement of donors with
government on research on the impact of the cash transfer programme demonstrates that
the notion that ‘donor funding’ is equal to ‘donor driven’ may be too simplistic in Malawi.
Where it matters, government has the capacity to take strong ownership of a research study
and decide which recommendations to take on board.
Social cash transfers were positioned to become a major poverty reduction tool in Malawi’s
National Social Protection Policy. It is generally seen as the central effort to respond to
widespread poverty, vulnerability and the limited ability of households to deal with climate
related shocks. The NCTP started as a pilot programme in one district - Mchinji - in 2006 –
and involved nearly 3,000 households (2008) then. The Programme has as of to date spread
to cover all of Mchinji District and to more than seven districts benefitting about 24,000
households.
The Centre for International Health and Development (CIHD) at Boston University School of
Public Health, jointly with the Centre for Social Research in Malawi, conducted a series of
rigorous evaluations in 2008. The evaluations looked at the impact of the programme on
children, families and the community. Results from the CIHD’s impact evaluation revealed
that cash transfer recipient families had fewer missed meals, lower rates of underweight
children, fewer reported sicknesses, higher school enrolment, better access to medicines and
health care, and a reduction in young children working to help support their families. The
study also identified a number of weaknesses in the cash transfer program and made a
number of recommendations.
The findings from the CIHD’s evaluations were used by government to improve the
programme’s operations and beneficiary identification policies, as well as informing plans
developed for scale up to more districts in the country. However, it has not been all the
findings from the evaluation that were taken on board. In some instance, the

6
UNDP and Norway have been among such donors but they withdrew their support for various reasons. It has
been observed that Malawi’s MPs are often unable to critically analyse issues outside of Party parameters.
12
recommendations were considered unfeasible politically or because of the available
infrastructure. In this instance, despite the fact that evaluation was funded by a donor,
Government took strong ownership of the research results that despite donor pressure they
have chosen which recommendations to take on board in the expansion of the program.
Donors though are still heavily involved in the funding of the program expansion.
42. Whilst a relatively new development, and with the influence of donors, sector wide
approaches (SWAps) or sector working groups (SWGs) have been operationalised for a
number of sector and programme areas. Generally, they bring together all the major
stakeholders working within a particular sector: government departments and
institutions, development partners, civil society organisations, academia and community
level groups. The SWGs create several sub-working groups among which there is always
one that focuses on monitoring and evaluation. The lead Government Ministry for the
SWG would initiate the establishment of an M&E unit or at minimum, would incorporate
some element of M&E functionality to serve the purpose of collecting data and other
information to feed the policy and programme formulation processes.
43. SWGs are normally chaired by a Principal Secretary and co-chaired by a donor. In
practice, these are proving to be an important avenue for DPs to influence government
policies and act as a basis to identify research and evaluation opportunities. For example,
the agriculture and water sectors have established regular periodic consultative
meetings and one joint sector review held around November or January each year
where policy based studies form part of the agenda. Functionality and pressures for
evaluations vary between different sectors and are most often shaped by interests
established amongst key role players. Such structures seem to work on the basis of the
energy of donor funded initiatives within Ministries.
44. As a result of historical challenges on the flow of donor resources and participation
within coordinating structures like SWGs, evaluative research and capacity has been
relatively diffused in the system. Whilst some sectors, such as health and education have
developed their own systems and some advances have been made on collating studies
completed, coordination levels have been low and there has not been a defined
modality for exchange amongst donors, beyond the data that feeds into the Malawi Aid
Management Platform (MAMP). There is wider appreciation for the MAMP and its utility
in collating data and aping progress on aid but it does not incorporate, at present,
information on evaluations that have been completed or that are underway. Indications
are that this will be incorporated into future modules, but the primary focus of the
stakeholders is on tracking disbursements and reporting on spending within priority
sectors.
Civil Society
45. The Malawian civil society has demonstrated reasonable capacity to engage with
government and Parliament on policy issues, but it has not historically been a source of
demand for evaluations. There are a number of civil society organisations that have
been active in encouraging evaluations, like the Malawi Economic Justice Network
(MEJN), WaterAid, World Vision, but the focus of their studies has largely been in the
context of mobilising resources for advocacy work or operational programming.
Perhaps this is unavoidable given the high levels of poverty experienced and their
13
general engagements with communities in the country. MEJN uses local consultants to
carry out regular budget reviews every year with support from donors.
46. There are established networks and organised formations for civil society engagements
and consultations, but the main channel for expressing evaluation demands has been
the same as those utilised by DPs. Civil society organisations participate in the SWGs and
District Executive Sub-Committee on M&E on the basis of their areas of interest. Some
of the civil society organisations (CSOs), like Plan Malawi, even serve as the Chairpersons
of sub-committees of SWGs and where necessary have established networks to mobilise
resources or influence policies. For example, CSOs have formed their own network in the
water sector (Water Environment Sanitation (WES) Network) they are given the platform
at the SWG forums to present their perspectives on the implementation and delivery of
water services in the country.
47. Given the nature of diffused resource flows from donors and the fact that many projects
are initiated directly through non-governmental organisations, some CSOs are given the
responsibility to chair sub-committees of the SWG. For example, Plan International
chairs the Sanitation Sub-committee. CSOs also interact with the NSO in terms of
improving data protocols, provision and dissemination. WaterAid, for example, has
been facilitating a national taskforce that is refining definitions of water and sanitation
indicators used in the national surveys conducted by NSO.
3.2 Government Agents
48. Within the system of government, sector Ministries are crucial for establishing the link
between supply and demand. Not only are they generally entrusted with the
responsibility for collecting and collating information for their sector, but they stand at
the centre of providing the required information to those who demand it and are also
central to ensuring that evidence is generated for policy purposes. Even though they
may often also be viewed as integral to establishing demand and supplying information,
they are, in practice, primarily involved in the exercise of linking supply with demand.
Central to this process is also the role of the government established National Statistical
Office.
Lead Ministry - Economic Planning and Development
49. A national M&E system was adopted in 2002 under the leadership of the Ministry of
Economic Planning and Development. This system was relatively weak by the time the
Mutharika administration took power in 2004. Under the Mutharika administration, in
2007, the M&E system was extended to the district and community levels. Government
also took a direct role in providing funding and paying district M&E officers. It was also
extended to other central Ministries which had no M&E Units. These included the
Ministry of Local Government, the Ministry of Water, and the Ministry of Labour. Within
the framework of the M&E Master Plan, the Monitoring and Evaluation Division of the
Ministry of Economic Planning and Development serves as the Secretariat for the
national M&E system. It has the responsibility for coordinating outcome and impact
monitoring across all sectors and undertaking producing poverty analysis. The
Secretariat is, furthermore, responsible for disseminating annual national development
reviews.
14
50. The national M&E system envisaged the establishment of a representative Technical
Working Committee (TWC) to guide the development and operationalisation of the
monitoring and evaluation system throughout the public sector. As substantial resources
would be required to implement such a system, government and donors established a
programme – the Joint Programme Support for Monitoring and Evaluation – to support
the process of implementation. It was anticipated that this structure would consider
technical reports and recommendations from the Secretariat which would include
progress reports from implementing institutions. However, the desired ambition that
included drawing on external research centres, on a quarterly basis or when required,
has not been realised. The TWC has had limited impact because of funding and capacity
constraints facing member institutions. External technical assistance was used in the
early years when designing the system. This included producing manuals for training,
forms for collecting data at various levels, and conducting training itself. This was phased
out in 2007.
51. The main consistent outputs of the National M&E System to date have been regular
annual reviews of the national development strategy which is used to inform the
national budget the following year. Use is made of both donor and government funding
to carry out data collection on the sector indicators and conduct consultative workshops
to review the results and write up the annual reports. The other analytical reviews that
emerged out of the initial attempts at coherence in M&E are as follows:
a. Public Sector Investment Programme (PSIP) Reports: Between 2006 and 2008, a
number of individual projects in the key sectors of the economy (water, transport,
agriculture) were assessed in terms of implementation. Reports were written,
published, circulated widely and presented to the National Assembly. One such
report on the transport sector triggered debate in Parliament and led to
corruption related investigations.
b. Public Expenditure Tracking Surveys (PETS): Between 2008 and 2010, the system
was used to conduct public expenditure tracking in Education, Transport and
Health. These reports were not widely circulated but were used by the Principal
Secretaries in the respective Ministries, and the Ministry of Finance, in effecting
budget and procurement changes, and for allocating the national budget.
c. MDGs Annual Progress Reports: These have been produced on a consistent basis
and continue to be generated with limited external assistance.
52. The main persistent criticisms against the outputs of the M&E systems, both at the
national and sector level are twofold: the quality of data used and the quality of analysis.
Firstly, on the quality of data used: the annual reviews do not have many quantitative
indicators and as a result the reviews rely on subject expert opinion. The data for
quantitative indicators is also not updated on a regular basis. Secondly, the quality of
analysis: the reports are not seen to have adequate analytical depth so as to inform
national budget planning and other specific policy reforms. Issues of data and staff
capacity seem to be at play here. Analytical skills are underdeveloped in many nonresearch
oriented institutions even if adequately qualified personnel are available.
53. It was anticipated that the use of resources would be monitored through annual Public
Expenditure Reviews (PER). This has however not unfolded as expected. Although line
ministries provide financial reports to the Monitoring Section of the Ministry of Finance,
15
some Ministries do not comply and it has not proven possible to impose any form of
sanction. Compliance is often waived due to political pressures (Chirwa 2004: 4). Box 2
explores this challenge in more detail.
54. One of the main tests for the sustainability of government’s efforts on evidence-based
policy formulation and evaluation is the provision of an explicit budget line for these
functions in the national budget. This has happened for the Ministry of Economic
Planning and Development (MEPD), the Ministry of Finance, the Ministry of Agriculture
and Food Security, and the Office of the President and Cabinet. MEPD also carries the
extra burden of supervising M&E Units in line Ministries. However, in none of these
institutions is government funding adequate to carry out periodic field visits, collect
regular and reliable data without the support of donors. This constraint is what led
UNDP to establish a multi-donor trust fund in 2006 to support the development of M&E
functions in the public sector.
Box 2: Public finance management systems performance assessment
The current budget system does not always allocate resources according to the priorities of
the MGDS, and this problem has been outstanding even for earlier national development
strategies. There is need to improve the public finance management system for this to be
achieved by making adjustments to the core structure for accounting (Malawi Government:
MEPD, 2008). This is an issue that requires implementation more than new studies to assess
what is wrong, but government has remained receptive, perhaps to a fault, to persistent
demands from donors to regularly assess the performance of the public sector financial
management system.
The Common Approach to Budget Support (CABS) donors has been providing budgetary
support to Malawi on the basis of the introduction of reforms to the government financial
system. A June 2005 public expenditure financial accountability (PEFA) assessment revealed
that the new government, which came into power in May 2004, had inherited an economy in
fiscal crisis with high interest payments on domestic debt and significant arrears throughout
the Government. Since that time, the new government showed political commitment to
reverse the trend.
Government made some attempts to improve transparency and accountability in the
management of public resources. There were still outstanding issues government had to
address: weak Parliamentary oversight; the lack of linkage between the Government’s
Strategic Plan (Malawi Growth and Development Strategy) and the financial planning and
budgeting systems; the complexity of budget documents; delays in preparation of financial
accounts; the publication of Audit reports and follow-up of audit recommendations; and
concerns over continuing weaknesses identified within the GoM’s internal control systems.
Government and donors agreed on the need for a new PEFA assessment to follow up on
achievements on previous recommendations and identify any other new issues for
consideration. The new PEFA assessment was conducted in 2008.
The main objective of the 2008 PEFA assessment was to take stock of the progress made in
the public financial management (PFM) reform process since the previous exercise (MayJune
2006) and assist Government in determining its most urgent PFM priorities. The
assessment results were expected to inform CABS donors on their support through
budgetary support and technical assistance to PFM issues. The study was carried out by a
team made of only international consultants from Consortium POHL Consulting & Associates.
The study involved evaluating the performance of the PFM system against a set of 28 high
level performance indicators for the government, plus 3 additional indicators that measure
16
the performance of donors involved in the government’s budgetary processes. Consultants
interviewed government officials, donors, and private sector organisations.
The completed assessment found that government maintained good fiscal discipline at the
aggregate level in the last three years, but the credibility of the budget was undermined by
deviation from plans. Whilst the formulation of the MDGS outlined government’s priority
areas, weaknesses in the planning system made it difficult to direct resources to those areas.
In many significant areas the outcomes from the financial management system could not be
determined, resulting in ‘no-score’ because of the weaknesses in the data for monitoring
areas, unreported government operations, procurement, and predictability of direct budget
support. Thereby in an overall sense it was not possible to determine the impact of the
financial systems on efficient service delivery.
Government had already done its internal report on the matter and the results from this
study were nothing new. Senior government officials in Treasury believe that donors prefer
reports from their own consultants in order to learn about what is happening. They pointed
out that some issues recommended in the study have been on the drawing board for many
years, like the need to have a budget that has an appropriate policy based classification
structure to deliver on the MGDS programmes, and the capacity for the efficient delivery of
services to ensure financial resources allocated for priority development interventions are
spent.
The process on the assessment is revealing of the momentum that donors themselves
establish with respect to analytical studies and evaluations. In this instance, donors were
simply keen to have their own study or independent assessment. Relevant government
officials were aware that the study is unlikely to reveal anything new, but that they felt it was
difficult for donors to appreciate their position. The overall impact is that such initiatives
generally undermine the ability to take a united position on resolving problems: it promotes
a government versus donors’ mentality. The logic of having two studies is party rooted in
donor perspectives on the need for independence on evaluations studies.
Line Ministries
55. The NDME Master Plan envisages a substantive role for line ministries even though the
institutional framework remains relatively weak and much of the evaluative activities
unfold on an ad-hoc basis. Some line Ministries have M&E units that are officially
created and operational. However, most do not appear to have any established capacity
for evaluation. A study conducted in 2004 (Chirwa, 2004) revealed that even where line
ministries established M&E units, they do not appear to have the legal or administrative
foundations for collecting and collating data. A related challenge is the lack of
monitoring reports, even in the Ministries that have M&E units. Although some
Ministries indicated that they produce monitoring and evaluation reports, these are
often ad hoc and mostly related to donor-funded projects. There is generally no
evidence that the M&E units in various Ministries produce periodic and regular
monitoring reports.
56. Linkages among various Ministries with respect to the exchange of information are weak
or non-existent. Many of the M&E units do not send their monitoring reports to MEPD
or other line Ministries. It is generally held that the Monitoring and Evaluation Division
of MEPD lacks the political power or leadership skills to coordinate the activities of line
ministries and civil society organisations. For example, some of the line ministries are
developing their own monitoring and evaluation systems independent of the national
17
framework of poverty monitoring and evaluation – such as Department of Local
Government, Ministry of Agriculture and Ministry of Health.
57. The capacity of M&E Units in line Ministries is limited (Arcadis Euroconsult 2004). There
are no work plans for the M&E work and no separate budget for monitoring activities.
Apart from funding problems, many M&E units do not have adequate human resources
to discharge their duties effectively. Where M&E Units do not exist, for example, in the
Ministry responsible for Climate Change, the monitoring activities are undertaken by
planning officers who view monitoring as a secondary activity. Official estimates in the
public sector for vacancy rates at the professional grades (first degree entry point) are
between 50 to 60 percent suggesting that there are capacity gaps in M&E Units. The lack
of coordination also reflects the capacity in the number of staff and management skills
in the M&E Division. As a result, many line Ministries do not seem to appreciate the
central role MEPD plays in the coordination of monitoring and evaluation activities. This
view was expressed by MEPD itself and donors that support M&E work like UNDP, DFID
and UNICEF.
58.The case of traditional birth attendants reveals important lessons on the power and
limitations of the Presidency to tamper with established policies in a field dominated by
donor funding and vested interests of government officials (see Box 3). The other
lessons relate to the complexity of implementing policy based on evidence from other
countries, the problems of tampering with research findings, the conflicts between
bureaucrats and politicians in policy making, and the limitations of even good research
to provide unambiguous direction.
Box 3: The role of Traditional Birth Attendants in the Health Sector
Traditional birth attendants (TBAs) are mostly women who assist mothers to deliver outside
the formal health system. They generally form an important link in the chain of health
personnel that provide primary health care services in Malawi. Despite the establishment of
many hospitals and health centres across the country, it is to the TBAs that the majority of
women turn to in times of child-birth. Over 60 per cent of birth deliveries in the country take
place in villages and generally outside of health care facilities or hospitals.
Given the numbers of women that turn to TBAs, successive governments regarded TBAs as
an important component in the delivery of maternal health. The training of TBAs became an
essential part of the Maternal and Child Health Services, and the Ministry of Health began
registering those who had undergone trainingin 1976. By the time the TBA policy was
suspended in the late 1990s, there were over 5000 TBAs out of which the Ministry of Health
had managed to train just over 2000. According to the 2000 Malawi Demographic and Health
Survey (MDHS), TBAs assisted in 10.5 percent of live births in urban areas and 24.4 percent
of live births in the rural areas.
By the year 2000, many international organisations changed their approach on TBAs and
their role was redefined to, amongst others, assist in the birth planning, by arranging
transport, by providing social and emotional support for the woman and to sensitise the
community on national strategies for skilled birth. The change in approach was as a result of
research findings based mostly on developed countries, which demonstrated lower maternal
death rates for countries that did not use TBAs (Stanton, 2008). At the same time, a growing
body of research showed that for developing countries the TBAs should be retained in birth
delivery especially where there was inadequate health infrastructure and shortage of skilled
birth delivery personnel.
18
Malawi, through the Ministry of Health, quickly signed up to the new international position
on TBAs and abandoned the support that had hitherto been offered to the development of
TBAs. The Muluzi Administration called government withdrawal from supporting TBAs a
suspension of policy and never bothered to publicise it. In contrast to the international
studies, local data showed that the maternal mortality doubled, from 620 to 1120 per
100000 live births between 1992 and 2000. At pains to explain the situation, the Ministry of
Health’s Reproductive Health Unit commissioned a study to investigate the impact of the
suspended TBA support programme. The study was conducted by Centre for Social Research
(CSR) in 2003 but the study report was rejected by officials on the bases of methodologies
utilised. Encouraged somewhat by donors, government continued with its policy and
adopted laws that stopped TBAs from delivering babies and also prohibited women from
giving birth at a TBA’s house.
In 2010 President Mutharika lifted the ban on the TBAs, but there was a struggle to reimplement
TBA support efforts where most of the funds were provided by donors. Donors
could not see the basis for this new expenditure and government could not entirely afford to
shoulder it. So confusion on TBAs’ role resurfaced at the national level from 2010 until last
year when there was a change of regime and the new President has reinstated the ban on
TBAs. The struggle to act on the basis of local evidence that contradicts wider studies reflect
the power that is exercised by donors and the reality that evidence based policy making is
not only a challenge for the Malawi government, but also has bearing on DP who function on
the basis of international policy templates.
59. Given the centrality of the President in the policy process, during the Mutharika period,
a Policy Research Unit (PRU), fully staffed and paid for by government was established in
the Office of the President and Cabinet (OPC). Its overall mission was to improve and
standardise the formulation of policies across government institutions. There was no
direct involvement of donors in setting up the Unit in 2005 and it was staffed by
competent professional staff from a number of fields: economics, health, education and
statistics. Later, some limited donor support was identified for training and assistance
with systems and procedures for evaluating policies before they are submitted to
Cabinet.
60. The PRU had immediate impact on the policy formulation process as a number of policy
documents were referred back that did not meet set criteria on quality, content and
implementation plans. The officials of the Unit were also assigned to attend numerous
Cabinet committees that deliberated on policy. However, within a short period of time
after establishing PRU, the then President shifted towards a more dictatorial policy
approach where the role of evidence was minimal. This was worsened by general budget
constraints from 2010 and as a result the PRU lost its intended focus.
61. High level consultations during this study indicate that the President and Ministers are
beginning to place added emphasis on driving implementation and policy as a collective
and hence are looking forward to more effective reporting on government’s
performance. One purported indicator of this movement towards better information for
policy is the establishment of a Service Delivery Unit (SDU) with the mandate to monitor
and evaluate the delivery of public services. Indications from within OPC are that the
SDU is being established with external international support.
62. The President has reportedly placed added emphasis on the importance of results-based
monitoring. As a result of this, the SDU has conducted a number of service delivery
19
assessments where government departments and public institutions explained the
reasons for their performance. The growing and recognised challenge, however, is the
absence of clear indicators for performance at the Ministry or department level and the
lack of baseline data. Within the current system, with restrictive sector resource
allocations and the continuing pressures for patronage, it is difficult to envisage how the
Cabinet or the President could become a source of substantive demand for evaluation or
independent research.
National Statistical Office
63. The National Statistical Office is the government department that is charged with the
responsibility of collecting national data under the Statistical Act. The NSO was identified
as a key player in the poverty monitoring system with respect to the generation of
outcome and impact indicators through periodic surveys. For example, within the MPRS,
it was envisaged that the Core Welfare Indicators Survey (CWIS) would be carried out
annually and that the Integrated Household Survey (IHS) and Demographic Health
Survey (DHS) would be carried out every five years.
64. Notable surveys that have been carried out by NSO since the launch of the first medium
term national development strategy (MPRS) in 2002 include the Core CWIS and two IHS.
The initial surveys were conducted on schedule but thereafter it has not been possible
to maintain consistency in timing. The IHS, for example, has been carried out in 1998,
2004 and 2010 which is not in line with the expected five year time periods in the
NDMES. In order to improve the capacity of NSO in the collection of data, there are
plans to have a statistical master plan. A needs assessment funded by DFID has been
carried, but generally progress has been slow.
65. NSO is also responsible for the Malawi Socio-Economic Database (MASEDA), a computer
based data base at District Assembly (DA) level that was designed to be key feature of
the national poverty monitoring system. Under the system, output and impact indicators
are expected to be captured at district level. The DAs are expected to send the up-dated
information to NSO which in turn will integrate the district and line ministries
information into one data base that will be provided to the M&E Division of MEPD (NSO,
2004). Many officers in the DAs were trained in the operation and use of MASEDA, but
not all the DAs have the application and requisite hardware. As a consequence, the
system is not fully institutionalised (GOM, 2004c).In addition, MASEDA has been
confronted with a range of administrative problems since its establishment and has not
really received the required support from line or sector Ministries.
66. The role of research institutions, such as universities and other key supply agents for
evaluative research and other studies, in the national poverty monitoring systems, have
been articulated both under the M&E MP and MASEDA but it has never been fully
operationalised. Research institutions often conduct studies, both commissioned and
non-commissioned, that may provide information on poverty outcomes and these
should find a proper channel to inform policy and decision makers on a regular basis.
However, in the national poverty monitoring system research institutions and
universities do not seem to play an active and dedicated role in poverty monitoring.
20
3.3. Supply of evidence and evaluation agents
67. Many DPs have historically commissioned evaluations that cover the spectrum of the
development assistance provided. These studies are largely directed at providing
information for external accountability purposes and are focused on the efficacy of
interventions from the perspective of DPs, rather than from the perspective of
Government. Recommendations in many of the reports are primarily focused on
changes that are needed within DP practices as they interface with Government. Supply
is spread across professional public and private suppliers, and embedded in the work of
independent think-tanks, universities and professional associations.
Professional suppliers
68. There were many reports of evaluations done by Malawian institutions but given the
time constraints for the study, it was not possible to access these, as are the reports
commissioned by external institutions. This has to do with record keeping: many
external institutions have websites hosting their work but local ones do not. Thus many
of the evaluations reviewed during the study process were contracted to agents based in
the North but completed in partnership with local consultants. The general view in
Malawi, confirmed by UNDP and UNICEF during the study, is that there is substantive
capacity in the country for the management and delivery of evaluation studies. The
views of UNDP and UNICEF are very important on this matter, given that UNDP run the
programme that recruited 40 M&E officers at the district level and UNICEF was behind
initial efforts to establish an evaluation association in Malawi plus many other activities
related to M&E. There is, however, no real evidence of substantive capacity for delivery.
Stakeholders point to the existence of specific individuals and institutions outside of
government carrying out evaluative research.
69. It has not been possible to make any authoritative judgment in this study on the wider
capacity for supply of evaluative research within the country. Many indications incline
towards the extensive use of foreign consulting firms who work in partnership with local
providers. On the surface, and based on studies generated by local academics and
consultants, there appears to be a high level of available capacity. The challenge is that
there is no source of information on the number of professionally trained evaluation
specialists and it appears that many who enter the space do so on the basis of skills
obtained in other areas of analytical work.
Independent think tanks and research institutions
70. The most prominently identified research institute established, as a Malawi-owned
specific think-tank, is the Centre for Social Research (CSR). CSR was established in 1992,
initially as an exclusive Project Unit within the University of Malawi to evaluate the
implementation of UNICEF’s nationwide water, sanitation and health programmes.
UNICEF noticed a gap in local capacity to conduct technically grounded evaluations of its
programs that required survey based methodologies, and teamed up with the University
of Malawi to establish CSR for this purpose.
71. CSR initially relied on staff in the Faculty of Social Science at Chancellor College, but the
volume of work was such that it had to recruit its own fulltime staff without necessarily
cutting off the need to draw on relevant additional expertise, when needed, from the
University. Over the years, with a change in approach to monitoring and evaluation,
21
UNICEF facilitated the evolution of CSR into an independent research institution that
could stand on its own and serve other clients. CSR has grown and now has a staff
compliment of 12, five of whom have PhDs and it still draws on staff in the University
when needed.
72. Malawi does not appear to have other well established independent research
institutions. At a minimum, professional bodies, such as the Law Society, Accountants
Society, Economics Association, do position themselves as think tanks on occasion
depending on the topical issues at hand and the capacity of the institution to offer
advice. Capacity, funding or/and management problems appear to have crippled the
performance of a number of locally-owned policy advocacy CSOs. The study did not find
a single CSO with a fully-fledged research or monitoring and evaluation unit.Many
international CSOs work with the government and invest into areas identified by
government. Plan Malawi and Water Aid engage in analysis in their areas of interest and
do utilise statistical data on poverty and other indicators provided by NSO. Where things
are not working well, they will share their findings with government in a nonconfrontational
manner to seek resolution.
73. The few available suppliers of analytical studies, work as individuals, and are generally
based at the University of Malawi. It is held by some stakeholders that the capacity may
not really exist in the country to conduct large-scale multi-disciplinary research or
evaluation given the fragmented nature of its suppliers. It is difficult to judge whether
there is inadequate research or evaluation capacity in the country, and the use of
external (to the country) consultants is perhaps more due to lack of corporatisation than
availability. It is held by some of the stakeholders that the fragmentation might also be a
manifestation of high demand for research and evaluation services. Capable people do
not see the need to work under employment terms as is often the case with corporate
entities.
Universities
74. Malawi has three publicly funded universities and seven privately funded Universities
and colleges. The most prominent is the University of Malawi which started in 1965 with
90 students. A study conducted by the Southern African Regional University Association
reveals that there are approximately 890 academic and research staff at universities in
Malawi. Of this number, approximately 90 percent are employed by the University of
Malawi. The majority of qualifications are awarded at the undergraduate level. It is
furthermore reported that the University of Malawi only produced one doctoral
graduate in the 2009/2010 academic year (SARUA: 2012), the highest being seven in
2008/09. Whilst there are plans underway to expand post graduate studies and numbers
of students, available data paints a bleak picture on the production of the required
research capabilities that the country would need for the future. Outside of the work
undertaken by the Department of Economics at the University, there is very little to
suggest that the universities are or will be, in the short-run, a source of adequate
evaluation expertise.
75. The Department of Economics has a professional staff compliment of 11, with 9at the
PhD level. Compared to other University Departments, it currently has the highest
number of PhD holders. Given this reality, the academics are generally sought after for
analytical and research work commissioned by donors. The work should be channelled
22
through the Consultancy Bureau of the University, but this strategy has not worked in
practice and lecturers tend to engage in their own consultancy work. The demand is
seen to be high and lecturers have the opportunity to choose who to work for.
76. The University of Malawi has a Centre for Agriculture Research and Development (CARD)
based at the Bunda College of Agriculture. This research based institute has full time
professional staff members who engage in monitoring and evaluation of agriculture and
natural resource based projects. CARD has a smaller staff compliment than CSR: five
professional staff of whom three have PhDs. CARD provides training on short courses on
monitoring and evaluation, unlike the Department of Economics. Two relatively new
private Universities, the Catholic University and the University of St John the Baptist
have introduced diploma level courses in monitoring and evaluation in 2013.
Evaluators and evaluation associations
77. It is not possible to establish the actual number of skilled evaluators within Malawi.
However, during interactions with stakeholders, the individuals responsible for M&E
within DP, government and local institutions characterised themselves as evaluators and
generally held related titles. Given the general global growth in the area of M&E, a
number of Malawian institutions have indicated that they intend appointing skilled
individuals for M&E purposes for in the future.
78. Given that evaluation has not really taken root as an area of professional specialisation,
it is hence not surprising that Malawi does not have a functional Evaluation Association.
An attempt was made to launch such an association in 2012, but this initial momentum
fizzled out. There are current efforts underway to re-establish a momentum for such an
association and that it is likely that such a body will be established in the course of 2013
or 2014.
79. Besides the possibility of membership to the African Evaluation Association, individuals
engaged with during the study process expressed an eagerness for the establishment of
the association in order to drive awareness on evaluation and research to guide policy
making in the country. Many also asserted that the association would provide a basis for
enhancing local participation in evaluations undertaken and could be instrumental in
promoting the development of relevant training and educational programmes.
80. A common observation from the key informants is that there is demand for evaluation
services in the country and that existing suppliers, such as CSR, are unable to respond to
the many requests. Whilst the sources of demand is still largely from within DP driven
processes, the supply availability, at the level of corporate structures continues to be
low, with the consequence that foreign organisations commission and contract out to
individual consultants. According to the Department of Economics at the University of
Malawi, the general perspective is that many of the evaluations conducted tend to be
descriptive analyses that do not provide adequate sense or lessons on the achievements
of particular interventions, nor can the reports themselves meet the standards of quality
for publication in peer reviewed publications or journals.
81. Many evaluation interventions have no baseline data for identified indicators, rendering
the exercise difficult to complete and of no real value. Data available from official
statistical sources tends to be too general to be used in the context of narrowly defined
interventions. Stakeholders have expressed that there is need to increase the uptake of
23
evidence in both policy and practice. There is also need for research results and findings
to be robust, hence, raising the importance of triangulation especially on sensitive or
contentious issues. These challenges and the opportunities for enhancing demand and
supply are taken up in more detail In Box 4.
Box 4: Starter Packs/Targeted Input Programme
The proliferation of SWGs may not be adequate to resolve the perennial problem that both
government and donors can look at recommendations from studies differently. For example,
Malawi has become famous for the current Farm Inputs Subsidy Program (FISP) and the
programme has been the subject of wider contestation and numerous studies by both
government and donors. This case largely demonstrates that both government and donors
cherry pick recommendations from research findings, for different reasons.
The Starter Pack (SP) program and its successor, the Targeted Inputs Program (TIP), were
implemented by the Ministry of Agriculture and Irrigation (MoAI) for nearly five years. The
main objective of the programme was to increase food security by giving smallholders in
rural areas packs of fertiliser to grow adequate food to feed their families. Both Programs
were funded by government and international donors - the World Bank, DFID and EU.
A large-scale M&E exercise was initiated between 1999 and 2002. This included an initiative
to assess the impact of the programme. The research was commissioned by the Ministry of
Agriculture and Food Security and funded by DFID at a cost of USD 1.3 million. It involved 13
integrated modules – comprising nationwide surveys, participatory research exercises and a
set of case studies. Technical support and supervision was provided by the Statistical
Services Centre, University of Reading and Calibre Consultants.
The research found that provision of free inputs had positive impact on food security in
Malawi, in particular during SP1 and SP2 when there was universal coverage. The
Programme was capable of contributing an average of two to three 50 kg bags of maize per
beneficiary household. In 2000, the additional maize produced (around 350,000 tonnes) was
adequate to create a surplus at macro-level and keep maize prices low until the following
harvest. The replacement of Starter Pack by TIP – with fewer beneficiaries and lower output
per beneficiary – was a major contributing factor to the 2002 hunger crisis, when maize
prices rose to roughly six times as high as the previous year.
In 2002, Government adopted the research recommendation to return to a universal Starter
Pack, hence the TIP (a precursor of the current FISP), providing a pack of free inputs for
every rural smallholder. In the 2002-03 season Government, with support from DFID and
Norway, distributed two million packs in time for the rains. Government tried to obtain
funding for a further one million packs, but progress was slow and distribution of the
additional packs was too late for the planting season. Although government was keen to
extend the programme on the basis of the positive evaluations, it generally ignored areas that
did not contribute to building its wider popularity, such as crop diversification and the use of
better quality seeds. On the donor side, including the World Bank, the UN agencies, the EU,
DFID and USAID, indications were that there was an element of reluctance to discuss the
research findings and their policy implications on aid flows. In particular, there was
reluctance to appreciate, amongst others, the evidence on the effectiveness of free inputs as a
transitional (medium-term) food security measure in the context of agricultural
liberalisation in an extremely poor country.
Government easily adopted the recommendation to return to a universal Starter Pack and
the reason was simply political imperative. The Starter Pack was highly popular in rural
areas and probably contributed to the re-election of Muluzi in 1999, as his government was
seen to be doing something about hunger. On the other hand, TIP – which involved scaling
24
down and targeting – was highly unpopular. Even those who received a pack under TIP
thought that the programme was unfair and socially divisive. After the 2002 hunger crisis,
continued targeting would have been political suicide. The reasons why government ignored
the findings on sustainable agriculture are more complex and perhaps related to a wellfunded
international lobby organisation promoting intensive cropping of hybrid maize -
Saskawa.
The donor community had its own share in ignoring research findings. Amongst others,
donors stuck to their established perspectives on social interventions as key drivers for
development and the general agendas developed by donors themselves on what
interventions were appropriate to addresses deepening poverty in the country. This
experience points to the importance of securing commitment to engage in the results of
evaluations in an open and transparent manner.
4. Pathways, opportunities and challenges
82. This report sought to understand: (i) the conditions under which demand is generated
for evidence; and (ii) the areas in which supply can be strengthened to meet and foster
this demand. This report has shown that there are currently mainly latent and potential
demands for evaluation in Malawi. The latent and potential demands are nested within
the demands for evidence from principals and civil society in Malawi. At present, the
demand is largely conditioned and driven by development partners. Supply could in the
short-term be strengthened through existing civil society based organisations and the
University of Malawi. A range of pathways relating to these twin challenges is identified
in response to the main research questions below.
83. This study set out to understand the conditions under which demand is generated for
evidence in policy and decision making and the areas in which supply can be
strengthened to meet and foster this demand in Malawi. Demand has been weak and
inconsistent largely due to a deeply rooted historical culture of patronage and fluid
policies. On a positive note, the current government has taken positive steps to
introduce a system of performance assessments where the definition and collection of
indicators for monitoring will take a prominent role. Donors have become even more
supportive in recent years, if the level of recent financial support the country has been
receiving is anything to go by. Coupled with the positive space government is promoting
on the use of evidence in management, the future for making use of research findings
from evaluation seems promising. The main limitation that an absence of collated
information makes reasonable estimates on the capacity for evaluative research difficult.
84. The drive into the future has to be predicated on the reality that patrimonialism will
continue to shape policy discourse, implementation and indeed the nature and
responsiveness to evaluations and analytical studies. National systems designed nearly
ten years ago are still nascent. Key stakeholder roles are not well defined, nor are
linkages across key public sector institutions strong. With a supportive policy
environment, there is room in the future for the use of evaluative studies. Where
evidence is provided from objective and undisputable scientific research, especially in
fields such as health, there is usually willingness to make use of them. The current
government is rolling out a results-based management system and this has opened up
more space and opportunities for the use of evidence. The senior bureaucrats are likely
to seize the opportunity to encourage monitoring and the collection of credible
25
administrative data in their institutions so as to provide quantitative data for
assessments.
85. The clear areas for investment in order to promote evaluation capacity development in
Malawi would have to involve investment in training and capacity building at the
individual and institutional levels. Donors could, for example, promote the introduction
of University level training in this area, support institutional development for evaluative
research outside the public sector like the establishment of an independent evaluators
association or its equivalent, strengthen linkages and the role of civil society to
contribute to uptake of evaluative research.
4.1 Building the demand for evaluation
86. Even though the demand for evidence primarily centres on the collation of evidence to
shape the distribution of patronage, to contest elections and for constituency
distribution purposes, it provides a latent opportunity for enhancing the overall
evaluation system. In addition, civil society and development partners have active
spaces for articulating their perspectives, thus opening many for enhancing demand for
evaluations. These opportunities and the challenges they embody for the future are
elaborated below.
87. It is important that Malawian policy-makers and implementers own the research
evidence needed for effective support and implementation. This is in contrast to the
position where evidence is largely the property and domain of research providers, or
sometimes lower level technocrats who try to introduce research findings in the system
without much success. Ownership of the best available evidence can enhance its use
and facilitate well informed decisions. In order to improve ownership and uptake of
evidence, in both policy and practice, developing better, on-going interaction between
evidence providers and evidence users is necessary and can be constructed on the basis
of added engagement on establishing a common coordinative mechanism for
evaluations that unfold on government programmes.
88. Malawi does not have regular and vibrant policy dialogue forums. The M&E Master Plan
envisaged engagements with stakeholders, but efforts in this direction have been slow.
Key informants observed that government has historically avoided such forums on the
ground that they tend towards being politicised and are used as opportunities to criticise
the ruling government. It may well be possible to introduce dialogue on the basis of
substantive research and to establish such high level forums for simply sharing results of
completed research of national value. There generally appears to be a sense of
openness to wider engagements and learning, but limited investments are being made
in efforts to foster dialogue on the basis of active research.
89. A further challenge for the research community in Malawi is to make research findings
usable for the policy-making community. Research studies are not easily accessible
documents to the busy bureaucrats and policy makers. Evaluation studies tend to be
generated as complex documents that few are able to engage with. Beyond the
production of Executive Summaries, there needs to be engagement amongst all parties
to facilitate greater capacity for the communication of findings in a manner that would
facilitate media and policy attention. Given the absence of active coordinating efforts
26
and leadership in this terrain this has not been an element of expressed concern but a
reality observed by many commentators.
90. Malawi does face a key challenge on how to communicate evaluation research findings
in order to influence policy and program design changes, despite the fact that some
studies are commissioned by the institutions themselves – government, donors and civil
society organisations alike. The strategies used to get research findings to their point of
use involve both dissemination (providing information from the supplier outwards –
using media, seminars) and the provision of access (libraries, websites, and other
repositories of information which research users can get hold of). In some countries,
institutional arrangements in the form of evaluation associations do help to serve as
quick reference for locating suppliers and users of evaluation research. The
apprehension at the higher levels to engage with complex research studies can be
overcome if the studies are disseminated to fit in with the reality of existing capacity
constraints. The fact that most policy makers will not engage with complex documents
may not be unique to Malawi but what is important is to arrange appropriate packaging
to reach the intended audience.
4.2 Strengthening evaluation through government agents
91. Government agents are central to the establishment of a workable and sustainable
system for the evaluation of development interventions. The current system reflects a
level of diffusion and unevenness in capability and commitment. Building the required
capability for an integrated orientation takes time. Outside of the numerous challenges
for enhanced capacity, there is system wide commitment for coordination, reflected in
part by a growing interest with the establishment of centralised coordinative capacity
and with the systems that have unfolded historically. The opportunities and challenges
embodied in this space are captured in the paragraphs that follow.
92. There is need to distinguish people who are users of research-based evidence and those
who are providers of research. It is not necessary for professional decision-makers and
practitioners to be competent executors of research, but it is both reasonable and
necessary for such people to be able to understand and use research results in their
professional practice. An increasingly necessary skill for professional policy-makers and
practitioners is to know about the different kinds of relevant research studies available
in their area of work; how to gain access to them and how to critically appraise them.
Without such knowledge and understanding it is difficult to see how a strong demand
for research, let alone evaluative research, can be established and, hence, how to
enhance its practical application. Joint training and professional development
opportunities for policy-makers and analysts may be one way of taking this forward and
for matching strong demand with a good supply of appropriate evidence.
93. In light of the fact that most studies are commissioned or funded by DPs, Malawi needs
to establish an active forum or institution base for identifying research areas that would
be important for the development of the country. This was somewhat envisaged within
the M&E Master Plan but seems to not have been taken forward. Robust research into
this area would be of prime importance and needs to be established by Malawians
themselves. Good research will also require improving statistics at the National
Statistical Office that are often seen as not regular, comprehensive and consistent. The
NSO is aware of its challenges in this regard, but it does believe it is getting adequate
27
support from government and stakeholders to move in the right direction. The only real
autonomy the NSO has is when to release survey results, but even that can sometimes
be compromised with a strong interventionist government. The NSO also constantly
confronted capacity problems as recruitments for replacements are beyond its control.
A comprehensive up to date national statistical and administrative databank would
reduce the costs of research in the country as time and resources for primary data
collection would be reduced.
4.3 Building the wider supply system
94. Whilst there is some disconnect between supply and demand, there is general
appreciation of capacities within civil society and on the need to enhance research
capability across the country. Existing institutions and individual are overwhelmed with
immediate demand, hence living little time for the building of institutions that could
serve to enhance capacity at a wider level. Given the potential level of demand for the
limited supply, the opportunities for enhancing capacity in this space is fairly wide and
explored in the paragraphs that follow:
95. Malawi has a clear shortage of certified monitoring and evaluation training programmes.
The public sector is known for high vacancy for professional staff, and is often as high as
50 percent due to low wages. The lack of quality training in M&E would also affect the
quality of administrative data collected for implementing program and project
interventions, and it is not surprising that some observers indicated that it is nearly
impossible in the public sector to carry out impact evaluations of specific projects due to
lack of data or its poor quality.
96. Although the MAMP would capture some of the information on completed studies,
Malawi would benefit from a central repository of all types of research in the country, as
long as it is of good quality, and these would be grouped into various fields of study in a
databank. Depending on the costs and demand, it can be explored whether access
would be free or not or a combination. There might also be incentives provided for
individual researchers to deposit their research studies in such an arrangement. National
or educational institutions libraries can be linked to such a repository, or they can be
brought together in an arrangement to oversee this
97. There is a lack of professional or peer reviewed publications in the country. This makes
it difficult to follow through developments in the local research community. The Malawi
Journal of Social Sciences, the only peer reviewed journal for the University of Malawi,
has of late been irregular, dependant often on the person who is the editor at the time.
There is need to invest in such publications and resolve challenges that limit their
availability.
28
References
Arcadis Euroconsult (2004) Developing Management and Information Systems for the
Poverty Reduction Strategy: An Initial Assessment, Technical Report 21, GOM/EU
Capacity Building Project for Economic Management and Policy Co-ordination, Ministry
of economic Planning and Development
Bergström, S and Goodburn, E (undated) The Role of Traditional Birth Attendants in the
Reduction of Maternal Mortality.Staffan.Bergstrom@phs.ki.se.
Booth, D, Cammack, D, Harrigan, J, Kanyongolo, E, Mataure and Ngwira, N (2006) Drivers of
Change and Development in Malawi. Working Paper 261, Overseas Development
Institute
Chirwa, E (2004) Poverty Monitoring Systems in Malawi: An Analysis of Institutional
Arrangements. University of Malawi: Zomba.
Chirwa, E, Kumwenda, I, Jumbe, C, Chilonda, and Minde, I (2008). Agricultural Growth and
Poverty Reduction in Malawi: Past Performance and Recent Trends, ReSAKSS Working
Paper No.8, South Africa.
Fölscher, Alta and Mkandawire, A and Faragher, R (2012) Evaluation of Public Financial
Management Reform in Malawi 2001–2010: Final Country Case Study
Report.www.sida.se/publications
Harrigan, J (2003) U-Turns and Full Circles: Two Decades of Agricultural Reform in Malawi
1981-2000. World Development, 31 (5), 847-863
International Monetary Fund (2012) IMF Country Report No. 12/221, www.imf.org
Joala,R, ( 2012), The super-inflasion of the Presidency in Malawi, African Security Review, 21
(4), pp.57-63.
Kadzamira, E, Moran, D, Mulligan, J, Ndirenda, N, Nthara, K, Reed, B, Lea, N and Lucia (2009)
Constraints to Growth in Malawi. World Bank Policy Research Working Paper, 5097
Levy, S. (2002) Bridging Research and Policy: Starter Packs in Malawi.www.reading.ac.uk
Malawi Government: Ministry of Economic Planning and Development (2012) Economic
Report.
Malawi Government: Ministry of Economic Planning and Development (2008, 2009) Malawi
Growth and Development Strategy Annual Review.
Malawi Government (2006) National Monitoring and Evaluation) Master Plan.
Miller, C., Tsoka, M and Reichert, K. (2008) Impact Evaluation Report: External Evaluation of
the Mchinji Social Cash Transfer Pilot. Center for International Health and Development
(Boston University) and Centre for Social Research.
Svåsand, Lars and Mpesi, Andrew, The Development and Functions of Electoral Manifestos
in Malawi: The Impact of Weak Parties and Excessive Presidentialism (2012). APSA 2012
Annual Meeting Paper. Available at SSRN: http://ssrn.com/abstract=2108047
National Statistical Office (2004) Integrated household Survey, www.nsomalawi.mw
National Statistical Office (2010) Integrated Household Survey, www.nsomalawi.mw
National Statistical Office (2008) Population Projections Malawi, www.nsomalawi.mw
Oduro, K, Quist, R and Steen, G (2008) Public Finance Management Assessment for Malawi
based on PEFA. Consortium Pohl Consulting & Associates: Final PEFA Report
Rakner, L and Mukubvu, L Ngwira, N and Smiddy, K. (undated, approx 2004) The budget as
theatre – the formal and informal institutional makings of the budget process in Malawi.
Reserve Bank of Malawi (2012) 4
th Quarterly Economic Review, www.rbm.mw
Resnick, D (2012), Two steps Foreward, One Step Back: the limits of Foreign Aid on Malawi’s
Democratic Transition, Working paper. Helsinki: United Nations University.
29
Rose, P (2004) Malawi: Study of Non-State Providers of Basic Services.www.idd.bham.ac.uk
Saasa, O and Atema, J (2010) Final Report: Review of the Malawi Growth and Development
Strategy, UNDP Malawi.
Southern African Regional Universities Association (SARUA( (2012), A Profile of Higher
Education in Southern Africa, SARUA: South Africa.
Savage, K and Umar, E (2006) Independent Evaluation of Oxfam GB Malawi’s Cash-Transfer
Programme, Overseas Development Institute.
Scott, C (2005) Measuring Up to the Measurement Problem: The role of statistics in
evidence-based policy-making. London: London School of Economics.
Stanton. C (2008) Steps towards achieving skilled attendance at birth, Bulletin of the World
Health Organization, 86 (4) pages.
United Nations Development Programme (2012) Human Development Report.
30
Appendix A: Interview list
No Name Position Organisation
Mr Williams Samute Deputy Chief Secretary Office of the President and
Cabinet
2 MrLuckieSikwese Principal Secretary,
Public Sector Reforms
Office of the President and
Cabinet
3 Mr Cliff Chiunda Director, Public Sector Monitoring
and Evaluation
Office of the President and
Cabinet
4 Mr Francis Zhuwao Acting Director, Economic Affairs Office of the President and
Cabinet
5 MrTwaibu Ali Deputy Director, Debt and Aid
Management
Ministry of Finance
6 Ms Kate Msukwa Principal Economist Ministry of Finance
7 MsTiyamikeKanthambe Debt and Aid Officer Ministry of Finance
8 Mr Jimmy Kawaye Program Manager, Joint Program
Support for Monitoring and
Evaluation
Ministry of Economic Planning
and Development
9 Hon. Abbie Shaba M.P., Budget and Finance
Committee
Malawi Parliament
10 Ms Naomi Kitahara Deputy Resident Representative United Nations Development
Programme
14 Mr Ernest Misomali Assistant Resident Representative United Nations Development
Programme
16 Mr Peter Kulemeka M&E Trust Fund Manager United Nations Development
Programme
17 Mr Patrick Kamwendo MDGs Advisor United Nations Development
Programme
18 Dr A. Mwaba Country Representative African Development Bank
19 Mr Benson Nkhoma Principal Water and Sanitation
Specialist
African Development Bank
21 MrNysonChizani M & E Specialist UNICEF
22 MrAshish Shah Evaluation and Results Advisor DFID
23 Mr Stein TerjeVikan Program Manager, Support to
National Statistical Office
Norwegian Embassy
24 MrDalisoKubalasa Executive Director Malawi Economic Justice
Network
25 Mrs Matilda Palamuleni Director of Programmes Ministry of Health
26 MrHeatherwickNjati Director of Planning National Aids Commission
27 Mr Davie Kalomba Director of Planning WaterAid
28 Mr Boyce Nyirenda Acting Country Program Manager Plan International
(KalondolondoPrg)
29 MrSiphoJale Quality Effectiveness and
Learning Specialist
Plan International
(KalondolondoPrg)
30 Mr C. Machinjili Commissioner National Statistical Office
31 Mrs Mercy Kanyuka Deputy Commissioner National Statistical Office
32 Dr A. Munthali Director Centre for Social Research,
University of Malawi
33 DrMussa The Head Economics Department,
University of Malawi
34 Prof Ephraim Chirwa Managing Consultant WADONDA Consult
35 Mr Allan Chimtedza Program Manager Tilitonse Program
Date
Resource Type Tag
Language Type Tag
Image Thumb
a