Making Decisions on Data Quality

The landscape has become somewhat crowded with tools for assessing health data quality in low- and medium-resource countries. Just recently, the United States President’s Emergency Plan for AIDS Relief (PEPFAR) and the Centers for Disease Prevention and Control (CDC) published excellent tools to strengthen data for measuring outcomes for people on treatment for HIV. How can countries and health planners know which tools to use to assess their data when so many good options are available?

A new publication, Comparative Analysis of Data Quality Assessment Tools, published by MEASURE Evaluation, which is funded by the USAID and PEPFAR, outlines the similarities and differences among these new tools—comparing their objectives and the scope of their methods. The publication is intended to help countries understand the purpose of these tools and select those appropriate for their context.

“The comparison of methods was in response to these new tools,” says David Boone, PhD, an epidemiologist and data quality activity manager for MEASURE Evaluation and senior technical advisor for JSI. “USAID wanted to know which one to use and when—and this helps to answer that question.”

Boone is a proponent of Country-Led, Holistic Data Quality Assurance, an approach embodied by a suite of tools and guidelines that contributes to USAID’s vision of improving the evidence base for public health monitoring, evaluation, and planning by improving the quality of routine health data. The holistic approach is centered on the Data Quality Review (DQR) methodology and toolkit, which provides countries a systematic and routine means of ensuring a minimum standard of quality for health data and identifies areas in need of strengthening. This DQR improves efficiency and reduces the burden on the health workforce by meeting the needs of stakeholders in one routine, carefully planned and coordinated assessment which obviates the need for ad hoc and overlapping program-specific assessments.

The guidance is the product of collaboration with international health program experts from leading donors and technical assistance agencies, such as the World Health Organization, USAID, Gavi, the Vaccine Alliance, and the Global Fund to Fight AIDS, Tuberculosis and Malaria. MEASURE Evaluation assisted in the development of the guidance and tested approaches to improve country ownership and leadership of data quality assurance.

MEASURE Evaluation recently published guidance on instituting a multi-stakeholder central-level coordinating mechanism for data quality assurance centered on the DQR methodology, Country-Led, Holistic Data Quality Assurance: Institutionalizing Data Quality through a National Technical Working Group and the Data Quality Review. The guidance provides instructions for establishing a technical working group (TWG); using the DQR for collecting and preparing data for analysis, conducting data verifications, analyzing and interpreting results; and best practices for countries on how and when to apply the methods.

“We feel the holistic approach is the best way to improve quality, coordination, and efficiency in data quality assurance,” says Boone. “Organizations or programs needing more in-depth information on data quality can still conduct program-specific assessments using their choice of tool, but this should be in collaboration with and coordinated by the central TWG to ensure country ownership and capacity to assess and improve data quality. We have made progress in reducing fragmentation in routine health information systems, thereby improving coherence and quality in the data. We want to avoid vertical approaches if possible so as not to lose these gains.”

“The DQR for country-led data quality assurance sets out a way for countries—not outside donors—to lead and manage data quality with their own partners and ministries calling the shots,” continues Boone. “We were trying to show if you could put a TWG in place and work collaboratively, this approach would help countries improve data quality and meet the needs of various stakeholders. The DQR is the centerpiece of this approach.”

Boone says that WHO deserves most of the credit for developing and promoting the holistic approach. “With good information up front and an understanding of the strengths and limitations of the data, health planners can make good decisions about service provision and resource allocation,” Boone says. “This is important—you wouldn’t bet big on something you weren’t sure of.”

This article is cross-posted from the MEASURE Evaluation project.