Monitoring, Evaluation, Quality Improvement and Targeted Research to Support Site Goals
Elements of monitoring, evaluation, quality improvement, and research cut across every phase of the implementation matrix and value chain. This component offers guidance and useful insights for data collection for mental health programming and how to use this data to improve program processes. Below is a Monitoring and Evaluation (M&E) Process Map that outlines how site teams have used M&E techniques throughout implementation of mental health programming.
Mental Health Monitoring and Evaluation Process Map
1. Assess
Assess existing MH data collection processes and tools
2. Define
Define the population, indicators, targets, and analysis plan, including how results will be communicated with stakeholders
3. Designe & Set Up
Create a template or database for data collection, determine a plan for data collection, including a plan to supervise and train people using the data system
4. Implement
Train providers and data clerks, pilot data collection, and revise processes if needed for quality improvement
5. Evaluate & Follow Up
Measure indicators against targets, conduct data quality checks, implement quality improvement projects, and disseminate results
Key Performance Indicators
This component suggests useful indicators for integrated mental health program planning and assessing the quality of mental health care. The indicators highlighted here can assist implementers to assess mental health burdens during pre-implementation or monitor the impact of mental health programs or quality improvement projects.
Routine Health Information Systems
Determining essential data collection tools is a core component of M&E strategies. Below are examples of tools and forms PIH mental health teams use.
Mental Health Mastercards
The “Mastercard” approach has been designed and utilized by the Malawi Ministry of Health – NCD Unit, APZU, PIH Sierra Leone and PIH Liberia. This concise form provides a more consolidated approach for clinics looking to establish routine data collection for each visit.
Learn MoreMental Health Registers
Daily registration of patient visits is a critical first step in monitoring patient volume and demographics and in understanding what types of treatments are provided at health facilities and communities. Below is a sample register from Sierra Leone's outpatient mental health clinic.
Learn MoreElectronic Information Systems
The establishment of Electronic Medical Records Systems (EMRs) and digital technologies can improve the data collection capacity and its value in applications to patient care, M&E, and research. OpenMRS is a collaborative open-source project to develop software to support the delivery of health care in developing countries and is used across PIH care delivery sites.
Quality Improvement
Quality improvement is a structured approach to evaluating the performance of systems and processes, and then determining needed improvements in both functional and operational areas. Mental health teams across care delivery sites have used quality improvement methods to improve mental health care delivery systems. Teams have done quality improvement projects focused on increasing case detection and referral of people with mental health conditions, decreasing lost to follow-up rates for patients in outpatient mental health clinics and reducing the need for referrals to the national psychiatric referral hospitals.
Below are quality improvement training materials and resources to help implementers assess program gaps or challenges and track QI projects.
PIH Resources
Global Resources
Targeted Research
Providing targeted research support to answer critical questions related to locally determined service delivery goals is a core pillar of PIH’s Cross-Site Mental Health Program. We have aligned mental health service delivery, training and research, and optimizing academic opportunities with partner institutions. A primary example is the Harvard University Global Health Delivery Partnership, linked through the Program in Global Mental Health and Social Change at Harvard Medical School (HMS) and now through GlobalMentalHealth@Harvard. The linkage of high-quality services for the poor with an academic mission is what makes PIH unique.
PIH Research Articles
Adapting Problem Management Plus for Implementation
Problem Management Plus (PM+) is a low-intensity psychological intervention that can be delivered by non-specialists to address common mental health conditions. This paper describes the process of contextually adapting PM+ for implementation in Rwanda, Peru, Mexico, and Malawi undertaken by PIH.
Read More
Development of a Comprehensive, Sustained Community Mental Health System in Haiti
Between 2010 and 2019 PIH and its sister organization Zanmi Lasante (ZL) mounted a long-term response to the 2010 Haiti earthquake, focused on mental health. Over that time, the organization successfully developed a comprehensive, sustained community mental health system in Haiti’s Central Plateau and Artibonite departments.
Read More
Outcomes of a Primary Care Mental Health Implementation Program in Rural Rwanda
To address the know-do gap in the integration of mental health care into primary care in resource-limited settings, a multi-faceted implementation program initially designed to integrate HIV/AIDS care into primary care was adapted for severe mental disorders and epilepsy in Burera District, Rwanda.
Read More
-
Additional Key Research Articles
- Adapting Problem Management Plus for Implementation: Lessons Learned from Public Sector Settings Across Rwanda, Peru, Mexico and Malawi (2021)
- Living Six Hours Away from Mental Health Specialists: Enabling Access to Psychosocial Mental Health Services Through the Implementation of Problem Management Plus Delivered by Community Health Workers in Rural Chiapas, Mexico (2021)
- 'Jack be nimble, Jack be quick...': mental health and psychosocial response in the time of the coronavirus (2020)
- Mental Health of Communities during the COVID-19 Pandemic (2020)
- Development of a Comprehensive, Sustained Community Mental Health System in Post-Earthquake Haiti, 2010–2019 (2020)
- Outcomes of a Primary Care Mental Health Implementation Program in Rural Rwanda: A Quasi-Experimental Implementation-Effectiveness Study (2020)
- Innovative Models in Mental Health Delivery Systems: Task Sharing Care with Non-specialist Providers to Close the Mental Health Treatment Gap (2019)
- Providing Mental Health Care in Primary Care Centers in LMICs Addressing Complex Health-Care Gaps in Chiapas, Mexico
- Integrating Mental Health Care Into Primary Care: The Case of One Rural District in Rwanda (2017)
- Validity and Utility of the Patient Health Questionnaire (PHQ)-2 and PHQ-9 for Screening and Diagnosis of Depression in Rural Chiapas, Mexico: A Cross-Sectional Study (2017)
- Mental Disorders and Drug/Alcohol Use in Patients Commencing Extensively Drug-Resistant Tuberculosis Treatment (2017)
- Evaluating Process and Clinical Outcomes of a Primary Care Mental Health Integration Project in Rural Rwanda: A Prospective Mixed-Methods Protocol (2017)
- Peruvian Mental Health Reform: A Framework for Scaling-Up Mental Health Services (2017)
- Development and Validation of Haitian Creole Screening Instrument for Depression (2015)
- The 2010 Haiti Earthquake Response (2013)
- Scaling Up for the “Bottom Billion”: “5×5” Implementation of Community Mental Health Care in Low-Income Regions (2012)
- Mental Health Response in Haiti in the Aftermath of the 2010 Earthquake: A Case Study for Building Long-Term Solutions (2012)
- Adapting Problem Management Plus for Implementation: Lessons Learned from Public Sector Settings Across Rwanda, Peru, Mexico and Malawi (2021)
- Living Six Hours Away from Mental Health Specialists: Enabling Access to Psychosocial Mental Health Services Through the Implementation of Problem Management Plus Delivered by Community Health Workers in Rural Chiapas, Mexico (2021)
- 'Jack be nimble, Jack be quick...': mental health and psychosocial response in the time of the coronavirus (2020)
- Mental Health of Communities during the COVID-19 Pandemic (2020)
- Development of a Comprehensive, Sustained Community Mental Health System in Post-Earthquake Haiti, 2010–2019 (2020)
- Outcomes of a Primary Care Mental Health Implementation Program in Rural Rwanda: A Quasi-Experimental Implementation-Effectiveness Study (2020)
- Innovative Models in Mental Health Delivery Systems: Task Sharing Care with Non-specialist Providers to Close the Mental Health Treatment Gap (2019)
- Providing Mental Health Care in Primary Care Centers in LMICs Addressing Complex Health-Care Gaps in Chiapas, Mexico
- Integrating Mental Health Care Into Primary Care: The Case of One Rural District in Rwanda (2017)
- Validity and Utility of the Patient Health Questionnaire (PHQ)-2 and PHQ-9 for Screening and Diagnosis of Depression in Rural Chiapas, Mexico: A Cross-Sectional Study (2017)
- Mental Disorders and Drug/Alcohol Use in Patients Commencing Extensively Drug-Resistant Tuberculosis Treatment (2017)
- Evaluating Process and Clinical Outcomes of a Primary Care Mental Health Integration Project in Rural Rwanda: A Prospective Mixed-Methods Protocol (2017)
- Peruvian Mental Health Reform: A Framework for Scaling-Up Mental Health Services (2017)
- Development and Validation of Haitian Creole Screening Instrument for Depression (2015)
- The 2010 Haiti Earthquake Response (2013)
- Scaling Up for the “Bottom Billion”: “5×5” Implementation of Community Mental Health Care in Low-Income Regions (2012)
- Mental Health Response in Haiti in the Aftermath of the 2010 Earthquake: A Case Study for Building Long-Term Solutions (2012)