Optimizing Performance Monitoring Teams to Improve Data Quality and Use in Ethiopia Hospitals

In early 2019, the health data management team at Zewditu Memorial Hospital in Ethiopia spotted a startling data inconsistency. The total number of recorded deaths in an inpatient ward did not match the morgue’s register. Upon examination, the team revealed a pervasive problem that spanned from staff capacity to larger organizational dilemmas. While the discovery was worrisome, it ultimately allowed Zewditu to re-examine its data practices and reinvigorate old systems with the help of the Ethiopia Data Use Partnership (DUP), a joint initiative of JSI Research & Training Institute, Inc. and the Ministry of Health (MOH).

Uncovering Hard Truths

Seeking to understand the extent of the hospital’s data discrepancies, Zewditu’s Planning, Budgeting, Monitoring and Evaluation Directorate (PBMED) conducted an in-depth analysis that found the hospital’s data quality below acceptable. Moreover, only 4 percent of the departments were reviewing their data and using it for planning and decision-making.

Behind the poor data quality, PBMED discovered a large knowledge gap among staff. Around 60 percent of those who collected data lacked proper training on its management, and only 4 percent of the workforce had the required data skills and knowledge. Consequently, staff made frequent recording and reporting errors, and PBMED found that 66 percent of submitted reports were incomplete and late.

Besides ill-equipped staff, each department lacked an internal review and verification process. Staff submitted monthly reports, often incomplete and inaccurate, directly to the PBMED without prior review or approval from the department or directorate head.

PBMED would then grapple to gauge data quality from the reports and present an aggregated report to the performance monitoring team (PMT), which meets monthly and is chaired by the hospital’s chief executive officer and medical director. The primary goal of the PMT is to review and analyze hospital performance and develop action plans for course correction.

For most PMT members, the monthly meeting would be where they first learned about their respective units’ reports. They were “caught off guard,” says Ahlam Mohmound, director of PBMED. Some members reacted with denial and defensiveness, which according to Ahlam contributed to the ineffective implementation of the PMT’s data-driven and evidence-based actions. Outside the monthly meetings, there was no platform for departments and directorates to exchange information or collaborate on PMT action items. “PMT was being held only for the sake of holding it. It had no major result,” recalls Ahlam.

Prioritizing Root Causes

Following its analysis, PBMED created a quality improvement (QI) team—a task force drawn from PMT and the health management information system (HMIS) section—to uncover the reasons for the hospital’s data issues and devise interventions. The QI team conducted several rounds of brainstorming sessions with relevant staff, and using a Fishbone Diagram to determine causal factors, identified several root causes. One was poor recording because registers and summary sheets were in short supply. “To make up for the shortages, we used to prepare [our own] register, which usually missed out some data elements. That played big part in undercutting the data quality,” says Hana Obse, head of Minor OR Surgery Department. Other problems included lack of training for staff and their inexperience in data elements; absence of feedback mechanisms; disorganization at various levels; and lack of accountability. The QI team organized the root causes it discovered into six categories: timeliness, accuracy, integrity, reliability, completeness, and data use. Because of limited resources, the QI team prioritized poor data use, lack of training, poor organization, and lack of accountability for immediate interventions.

Implementing Change

Intervention activities started in April 2019 with the help of DUP. To standardize data collection and prevent makeshift registers, standard data collection tools, like registers and tally sheets, were distributed as early as possible to allow staff to enter data daily. “It reduced the practice of entering several data later, which contributed significantly to improving data quality,” says Hana. Data recording and reporting procedures were restructured and streamlined. Job descriptions were also revised to better represent and clearly articulate staff responsibilities for data within their department and to increase accountability.

Management and staff were trained on health data management. As a means to encourage further skill development and knowledge sharing, each department established a learning platform called Peer Discussion Group, on which staff attend twice-monthly learning sessions that focus on various HMIS policies, guidelines, and best practices.

Improving data flow in reporting was also a top priority for the QI team. The intervention introduced revised monthly summary sheets to reflect who was responsible at each step of data collection, approval, and confirmation. Departments began reviewing and analyzing their data every week for decision-making before sending it the next level. Zewditu also instituted a directors forum, an additional monthly checkpoint to measure the veracity of submitted reports and data quality and use before they are sent to PBMED. Currently, PBMED cannot accept reports that have not been approved and confirmed at the department and directorate level.

The new data flow reorganized the previously chaotic process and ensured active engagement of all stakeholders. It also created more capacity for PBMED, which had the opportunity to repurpose its roles. The new process also interestingly, revitalized the PMT and reconnected it to its initial purpose of ensuring the hospital’s strong performance.

Achieving Results

PBMED deployed a customized a data quality assessment tool to measure the implementation outcomes. Just three months into the intervention, data quality at the hospital rose to 77 percent from the baseline of 41 percent. Another quarter later, it improve to 80 percent, and Zewditu is on track to achieve 90 percent data quality by the end of March 2020. Concurrently, data demand and use among departments surged from 4 to 75 percent.

The PMT meetings, previously symbolic, began to have a clear purpose. Its members, who are a part of their unit’s data review, began showing a greater sense of ownership and accountability to their respective data. This energy infused the team with a more collaborative and creative approach to overcoming challenges.

The swift cultural shift in data practice at all levels of the hospital is noticeable. According to Zerihun Ababo, head of Maternity Ward Department, all staff internalized the understanding that data is as important as “the job itself.” “Now everyone believes that work not recorded is work not done,” he says.

Zewditu has become a model for other facilities, and is helping five hospitals in Addis Ababa reboot their PMTs and improve their data quality. Data interventions that target the root causes of challenges at all levels not only improve data quality and use, they also have the ability to transform seemingly rigid fixtures, like staff culture and PMT forums, into valuable assets that advance performance.