Assessing the Capacity of Massachusetts' Local Public Health System

Client Office of Local and Regional Health – Performance Standards and Data Systems Unit

Practice Area Strategic Planning and Design

The Story of the Massachusetts Department of Public Health’s Office of Local and Regional Health

Through state-local coordination, the Office of Local and Regional Health (OLRH) serves as a focal point for connecting local public health partners with the diverse programs, services, and resources offered by the Massachusetts Department of Public Health (DPH). OLRH dedicates its efforts to bolstering local public health through collaboration with professional associations, academic partners, and other external partners. Its primary priority is addressing the needs of local public health, providing resources, tools, and support that underscore the crucial role of public health at the community level. This encompasses a spectrum of services, ranging from professional development training to hands-on technical assistance, ensuring that local health departments can serve their constituents effectively.  

OLRH Vision Statement: Every person in Massachusetts can equitably access excellent local public health services to be their healthiest self.  

OLRH Mission Statement: We work with partners to build capacity towards achieving efficient, effective, and equitable local public health services for all people in Massachusetts.  

One of the main initiatives OLRH employs to advance public health is sponsoring a grant program for local public health. The goal is to improve the efficiency, effectiveness and equity of local public health by establishing Shared Services Arrangements. These arrangements involve groups of municipalities committed to pooling resources, functions, and expertise to offer more comprehensive public health services to their populations. OLRH focuses on ensuring communication and information dissemination, supporting community health improvement planning, and championing the establishment of the first-ever Massachusetts Performance Standards for Local Public Health to set a baseline for service delivery. OLRH values continuous partner engagement within its initiatives and aims to promote a culture of collaboration and communication throughout its programs.

The Challenge of Assessing the Capacity of Massachusetts' Local Public Health System

In response to the recommendations outlined in the Blueprint for Public Health Excellence, OLRH introduced the first-ever Massachusetts Performance Standards for Local Public Health, delineating all mandated services local public health must deliver. These standards were pivotal in establishing fundamental service levels, defining clear parameters for workforce credentials, and outlining training requirements. 

A significant challenge for OLRH was the lack of a structured framework to evaluate how comprehensively local public health entities were meeting the newly established Performance Standards. The absence of a robust evaluation system hindered OLRH’s ability to gauge the capabilities of local health departments, identify areas needing improvement, strategically allocate resources, set measurable goals, and support a more equitable delivery of public health services across the State. 

Recognizing the urgency of addressing this gap, OLRH collaborated with BME Strategies to develop an evaluation framework tailored to the Massachusetts Performance Standards for Local Public Health. This collaborative initiative aimed to establish a systematic and data-driven approach for assessing the ability of local public health to achieve the Performance Standards. It is important to note that the intent of the assessment was to serve as a capacity assessment for local public health and not a performance assessment. The overarching goal was to transform the Performance Standards from static guidelines into a dynamic tool, enabling assessment and continuous improvement. 

OLRH also faced the complex task of developing a comprehensive assessment that could gather essential information while being mindful of the burden on Massachusetts’ local public health entities. The development process involved creating surveys and defining procedures for collecting documentation from municipalities, such as food inspection reports. This approach underscored OLRH’s commitment to setting standards and creating a robust system that could effectively monitor the progress and enhance the implementation of those standards.

How BME Responded

In partnership with OLRH and the Massachusetts Health Officers Association (MHOA), BME developed and implemented an evaluation framework to assess the capacity of local health departments (305 out of 351 municipalities as part of the Public Health Excellence Grant Program) via a three-phase approach—Baseline Capacity Assessment (BCA), Workforce Assessment, and Backup Documentation Submission. The BCA was the first phase of the 2022-2023 Massachusetts Public Health Excellence Shared Services Grant Program Capacity Assessment. The BCA involved developing and implementing a 179-question municipal self-report survey that evaluated local public health’s capacity to meet the Performance Standards. OLRH and BME collaborated extensively with diverse partners including subject matter experts coming from the local public health field to create, distribute, and analyze the assessment. BME designed a comprehensive data analysis plan that served as a roadmap, guiding the extraction of meaningful insights from the collected data and facilitating informed decision-making based on the outcomes of the BCA.  

The second phase of the Capacity Assessment was the Workforce Assessment. In partnership with OLRH and MHOA, BME developed an individual self-report survey to evaluate the local public health workforce’s education, credentialing, and training. 1,020 individuals responded to the Workforce Assessment, representing a wide range of positions across the local public health workforce and geographic diversity. The Workforce Assessment helped to shed light on the disparities in education, training, and credentialing within the local public health workforce. Furthermore, this assessment phase obtained demographic data about the local public health workforce. This data will guide the development of recruitment and retention strategies aimed at increasing diversity within the workforce that is representative of the communities OLRH serves. 

The third phase of the Capacity Assessment included unique backup documentation submissions for each municipality, based on municipalities’ responses to the BCA, for a qualitative look at the existing practices of local public health in delivering services. In this phase, over 5,506 documents were reviewed for proficiency in meeting the standards. BME configured a comprehensive document collection protocol and scoring criteria for various categories of documents (e.g., Food Protection, Swimming Pools, Title 5, etc.).  

Through the careful implementation and execution of these three phases, OLRH and BME addressed the challenges, enabling a comprehensive assessment of local public health capacities, workforce capabilities, and documentation.

The Results

Partnering with BME, OLRH achieved an outstanding 98% response rate on the BCA (Phase 1), enhancing its comprehension of local public health departments’ capacities throughout Massachusetts and the resources needed to meet Performance Standards on the SSA level. Delivering this at the SSA level, while considering the current capacity and areas of opportunity within all member municipalities, helped to pinpoint priorities, foster areas for resource sharing, and advocate for the SSA as a whole. Additionally, the Workforce Assessment (Phase 2) revealed gaps in education, training, and credentialing across all position categories, with 1,020 respondents contributing to this critical evaluation. Furthermore, the backup documentation collection (Phase 3) engaged over 75% of participating municipalities, allowing OLRH to identify improvement areas, notably the lack of standardized inspection and report forms and consistent lapses in required inspections. Complementing the evaluation framework, BME supported OLRH in the development and publication of an interactive Tableau dashboard, empowering SSAs to visualize their data and utilize it to work towards reaching more Performance Standards. SSA-specific Capacity Assessment results and their data-driven recommendations are currently used to inform work plans for Public Health Excellence for Shared Services Grant Program grantees. Beginning in February of 2023, OLRH and BME Strategies hosted individual Capacity Assessment Results Toolkit (CART) meetings with all 50 SSAs over eight weeks to discuss each SSA’s assessment results. This resulted in approximately 75 hours of dynamic and complex conversations. 

OLRH, MHOA, and BME’s creation and analysis of the evaluation framework enabled the establishment of a baseline for assessing municipalities, laying the groundwork for a second round of the Capacity Assessment set to commence in the future. This initiative is pivotal in advancing Massachusetts’ public health sector and propelling the modernization of the local public health system by working toward adopting the Massachusetts Foundational Public Health Services as the final iteration of the Performance Standards. OLRH will use these baseline measurements and quality improvement best practices to identify areas of opportunity and create measurable pathways of achievement. The results of this comprehensive evaluation will not only assist in enhancing public health capacities, but also support SSA’s in addressing the inequities that exist with their communities, working towards a more equitable distribution of resources and services.

Deliverables  

  • Utilized findings to inform SSA’s workplans for FY 24 in the PHE Grant Program.  
  • Capacity Assessment Interactive Tableau Data Dashboard  
  • Best Practices Guide Sheets tailored to each subject area, intended for training purposes. 

Overall Impact Statement

Through the comprehensive development and execution of a three-phase evaluation framework, BME addressed the challenges faced by OLRH, enabling a comprehensive assessment of the Performance Standards, paving the way for ongoing improvements and the modernization of Massachusetts’ local public health system.