Evaluating the impact of community health worker certification in Massachusetts: Design, methods, and anticipated results of the Massachusetts community health worker workforce survey

Abstract

Background: Professional certification of community health workers (CHWs) is a debated topic. Although intended to promote CHWs, certification may have unintended impacts given the grassroots nature of the workforce. As such, both intended effects and unintended adverse effects should be carefully evaluated. However, there is a lack of published literature describing such effective evaluations with a robust methodology. In this methods paper, we describe a key component of evaluating CHW certification in Massachusetts-the Massachusetts CHW Workforce Survey.

Methods: Design of the surveys was informed by a program theory framework that delineated both positive and negative potential impacts of Massachusetts CHW certification on CHWs and CHW employers. Using this framework, we developed measures of interest and preliminary CHW and CHW employer surveys. To validate and refine the surveys, we conducted cognitive interviews with CHWs and CHW employers. We then finalized survey tools with input from state and national stakeholders, CHWs, and CHW employers. Our sample consisted of three frames based on where CHWs are most likely to be employed in Massachusetts: acute care hospitals, community-based organizations, and ambulatory care health centers, primarily community health centers and federally qualified health centers. We then undertook extensive outreach efforts to determine whether each organization employed CHWs and to obtain CHW and CHW employer contact information. Our statistical analysis of the data utilized inverse probability score weighting accounting for organizational, site, and individual response.

Anticipated results: Wave one of the survey was administered in 2016 prior to launch of Massachusetts CHW certification and wave two in 2021. We report descriptive statistics of the three sample frames and response rates of each survey for each wave. Further, we describe select anticipated results related to certification, including outcomes of the program theory framework.

Conclusions: The Massachusetts CHW Workforce Survey is the culmination of 5 years of effort to evaluate the impact of CHW certification in Massachusetts. Our comprehensive description of our methodology addresses an important gap in CHW research literature. The rigorous design, administration, and analysis of our surveys ensure our findings are robust, valid, and replicable, which can be leveraged by others evaluating the CHW workforce.