There is growing interest in the potential of incentive mechanisms, such as
conditional cash transfers and performance based funding, for increasing the
quality, coverage and use of priority health interventions. Performance-based
funding mechanisms can only work with better data. Sound data are needed to
identify appropriate performance indicators, to track progress towards benchmarks
and targets, to monitor both intended and unintended effects, and to manage
potential conflicts of interest which financial incentives may aggravate. The
implementation of performance based mechanisms requires attention to the selection
of performance metrics and identification of related measurement strategies,
particularly in terms of health system functioning and performance. Yet currently,
health information and statistical systems in many developing countries are
not able to generate the kind of timely and reliable statistics needed for monitoring
health systems. For example, there are multiple problems in clinic and programme-based
reporting systems which often form the basis for the assessment of performance
against benchmarks.
The need to develop better metrics of health system performance was first addressed
by the World Health Organization in its 2000 World Health Report (World Health
Report 2000). The framework for health system performance covered the range
of indicators from health system inputs and processes (human resources, infrastructure,
essential medicines etc.), through programme output measures (coverage of interventions)
to health outcome or impact (mortality, morbidity, financial contribution, and
responsiveness). The report did not attempt to quantify the full range of indicators,
but ranked country health system performance using five outcome level indicators:
population health, inequalities in health status, level and distribution of
responsiveness and distribution of financial burden. The country rankings led
to considerable debate and distracted attention from the value of the unifying
framework for monitoring health systems.
Not until 2007 did WHO returned to the issue of monitoring health systems with
the publication Everybody's Business (WHO 2007). Its health system framework
was similar in terms of overall components to the earlier framework but paid
more attention to the health system inputs and processes, summarized as six
core building blocks of health systems: service delivery, health workforce,
information, medical products, vaccines and technologies, financing, and leadership
and governance.
The relevance of this effort has become ever more apparent with the growing
attention of the health and development community on the health-system constraints
that impede the attainment of disease- or programme-focused goals. Health-system
strengthening is now recognized as central to strategies for accelerating the
achievement of the MDGs – as noted by, for example, the Toyako Framework for
Action at the G8 Summit. The International Health Partnership Plus (IHP+) promotes
support to national health strategies and plans (including strengthened information
systems for monitoring and evaluation) as a means of increasing alignment on
national priorities. At the same time, virtually all health initiatives and
funds call for better information in order to effectively steer and track health-systems
strengthening efforts.
Feasible and realistic measurement methods
These developments provide a welcome rebalancing in terms of the contribution
of "vertical" and "horizontal" approaches to health improvement.
But from a monitoring perspective, they present a challenge in terms of the
kinds of measurement strategies for health systems that would be commensurate
with those used by categorical, or disease programmes. Health systems are complex;
measuring and monitoring their performance will need to take this complexity
into account.
Initial responses to these challenges started as long ago as 2004, when several
global health initiatives, notably the GAVI Alliance and the Global Fund to
fight AIDS, TB and Malaria, realized the need to tackle the constraints to the
achievement of their own goals presented by weak country health systems. In
2004, WHO and the World Bank convened a series of meetings, bringing together
country experts, academics, donors, funds and development partners, around a
joint effort to improve health systems monitoring. The focus of the effort was
on low and lower-middle income countries. The aim was not simply to define long
lists of health system indicators and leave it at that. Rather, the strategy
was to reach consensus on a parsimonious set of indicators for each health system
building block and to identify feasible and realistic measurement methods for
each indicator. It was anticipated that the effort would help countries establish
their own benchmarks and targets, and that progress would be reflected in country
progress reports or "dashboard". The dashboard could serve as a powerful
advocacy tool for health system investments.
Arising out of this work is a draft Toolkit on monitoring health systems strengthening
(WHO 2008).
The Toolkit describes a set of indicators, measurement approaches and strategies
that
permit establishment of country health system statistical profiles;- permit monitoring of health systems and guide country and partner investments;
- highlight gaps in terms of data availability and quality and point to needed
investments in measurement strategies;
- contribute to a global consensus around how to monitor and benchmark health
systems strengthening. (see Table 1).
| Health system building block |
Indicators |
Data sources |
| Governance |
Policy index (index to score whether certain policies are
in place)
Implementation indicators for building blocks
CPIA and CPIA score for health |
Expert review
Key informant surveys
Uses all data sources from other building blocks |
| Financing |
Total health expenditure per capita
% General Government expenditure on health
Out of pocket as % of total health expenditure
Total health expenditure as % of GDP |
National Health Account
Public expenditure reviews
Expenditure reporting systems
Household surveys |
| Workforce |
Health care professionals per 10,000 population with distribution within
country
Graduates of health training institutions per 100,000 population |
Facility assessments
Administrative data
Professional society databases |
| Information |
Health Information System Performance Index (HISPIX) |
Expert review |
| Medical products |
Tracer drug availability in health facilities
Median drug price ratio for tracer drugs |
Facility assessments
Facility reporting systems |
| Service delivery |
Health facilities per 10,000 population
Inpatient beds per 10,000 population
OPD visits per 100 population per year
General service capacity – proportion of facilities with basics such as
clean water, basic equipment
Specific service capacity – proportion of facilities with key components
for IMCI, FP, AIDS etc. |
Facility assessments
Facility reporting systems |
In working with countries to measure and compare health systems functioning,
it is important to strike a good balance between avoiding blueprints that do
not allow for country contexts and specificities, while simultaneously encouraging
a degree of standardization that enables comparisons within and between countries
and over time. Standardized indicators allow comparisons between countries and
can help mutual learning, including identification of bottlenecks and sharing
lessons learnt. However, measurement should be attuned to a country's health
strategy objectives. Thus, each toolkit section has proposed "core indicators"
that all countries are encouraged to collect and a wider set of indicators that
users can chose or modify as needed. Countries should integrate new indicators
with existing indicators of their health sector and statistical strategies and
plans.
Poor availability and quality of health systems data
Following the development of the Toolkit, WHO and other partners collaborated
with country partners to populate the indicators and generate the country dashboard
with regular statistics for a minimum set of country health system indicators.
The aim was to bring together data that would be comparable over time and between
countries, and to include both national and sub-national or distributional data.
In practice, the country dashboard proved to be more difficult than anticipated
because of the poor availability and quality of health systems data.
A recent review of current health system monitoring practices in selected countries
– Burkina Faso, Cambodia, Ghana, Kenya, Tanzania and Zambia – identified opportunities,
challenges and best practices relevant to each building block (Monitoring health
systems strengthening) Major concerns are evident regarding the availability
and quality of data. Growing demand for results in the context of the MDGs has
led to substantial investments in population-based health surveys to improve
the availability and quality of health statistics on coverage of interventions,
risk factors, nutrition, disease prevalence and mortality. By contrast, only
fragmented efforts have been made to improve data on the health system building
blocks. Yet both countries and development partners have a common interest in
generating international consensus on comparable methods and measures for monitoring
health systems strengthening.
In fact, countries are themselves taking the lead in implementing strategies
to monitor health systems performance. Several high income countries are conducting
various forms of performance assessment, often focusing on benchmarking and
comparisons with other countries. For example, the Organisation for Economic
Cooperation and Development (OECD) has been studying ways of measuring the performance
of different health systems and of using such information to support on-going
health systems improvement since 2002 (OECD 2002). The 2008 Dutch Health Care
Performance Report presents a broad picture based on 110 indicators and includes
comparisons over time and between countries where possible (Westert et al, 2008).
In the USA, the Commission on a High Performance Health System has developed
a National Scorecard, summarizing U.S. health system performance on 37 core
indicators and providing comparisons with other countries (The Commonwealth
Fund 2008). Authorship of such reviews varies, from multilateral organizations
such as OECD, to public health bodies (for example in the Netherlands) and to
independent bodies (as in the USA).
Investing in appropriate measurement strategies
Cross country comparisons are possible when data availability and quality are
relatively complete. However, this is not the case for many developing countries.
Here, there is much to be gained from an examination of sub national variations
in health system indicators. For example, the South Africa District Health Barometer
monitors a selected set of socio-economic and health care indicators in order
to highlight inequities in health resource allocation, inputs, outputs and outcomes
as well as the efficiency of health processes between provinces and between
districts (Health Systems Trust 2007).
These country experiences demonstrate that progress towards more effective
and efficient monitoring of health systems strengthening and health systems
performance is possible, given the necessary political commitment and resources.
Countries have much to learn from each other. It is clear that the main impediment
towards better monitoring of health systems is not the lack of indicators but
the failure to invest in appropriate and sustainable measurement strategies,
and the generally poor quality of available data.
What is now needed to support countries is to further refine and promote standardized
approaches, tools and methods for generating and analyzing health systems data
and to build institutional and individual capacities for the collection, analysis,
sharing and use of such data. Progress in these areas will require the mobilization
of technical and financial support for countries and other stakeholders to support
the process of improved health-system monitoring. Any approach will also need
to find the right balance between standardization and country specificity, and
will require a set of specific outputs that will improve the monitoring and
strengthening of health systems. WHO and other partners are now working closely
with funds, foundations, development agencies and countries, especially those
involved in the International Health Partnership, to implement such approaches
and to share lessons learnt.
*Carla AbouZahr is a statistician and public health expert working at
the World Health Organization, Geneva, in the Department of Health Statistics
and Informatics. She is currently working with a number of countries and partners
on strategies for monitoring health systems strengthening. She is responsible
for the views in this paper which do not necessarily represent the decisions
or the stated policy of WHO.
References
- World Health Organization 2000. The World Health Report. Health systems:
improving performance. The work of WHO. ISBN-13 9789241561983
- World Health Organization (2007) Everybody's business - Strengthening health
systems to improve health outcomes - WHO's framework for action. ISBN 978
92 4 1596'77
- World Health Organization (2008) Toolkit on monitoring health systems strengthening.
http://www.who.int/healthinfo/statistics/toolkit_hss/en/index.html
accessed 20 January 2009
- Monitoring health systems strengthening: country practices and impact assessment.
Unpublished paper
- OECD (2002) Measuring Up. Improving Health System Performance in Developing
Countries.
- Westert GP, Berg MJ van den, Koolman X, Verkleij H (editors) (2008) Dutch
health care performance report 2008 http://www.rivm.nl/vtv/root/o33.html
accessed 20 January 2009
- The Commonwealth Fund (2008) Why Not the Best? Results from the National
Scorecard on U.S. Health System Performance
- Health Systems Trust (2007) The District Health Barometer 2006-7
http://www.hst.org.za/publications/717 accessed 20 January 2009
