ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation
UMass Chan AffiliationsDepartment of Medicine
Meyers Primary Care Institute
Document TypeJournal Article
electronic health records
Health Information Technology
Health Services Administration
Health Services Research
Public Health Education and Promotion
MetadataShow full item record
AbstractBACKGROUND: Patient portals are widely adopted in the United States and allow millions of patients access to their electronic health records (EHRs), including their EHR clinical notes. A patient's ability to understand the information in the EHR is dependent on their overall health literacy. Although many tests of health literacy exist, none specifically focuses on EHR note comprehension. OBJECTIVE: The aim of this paper was to develop an instrument to assess patients' EHR note comprehension. METHODS: We identified 6 common diseases or conditions (heart failure, diabetes, cancer, hypertension, chronic obstructive pulmonary disease, and liver failure) and selected 5 representative EHR notes for each disease or condition. One note that did not contain natural language text was removed. Questions were generated from these notes using Sentence Verification Technique and were analyzed using item response theory (IRT) to identify a set of questions that represent a good test of ability for EHR note comprehension. RESULTS: Using Sentence Verification Technique, 154 questions were generated from the 29 EHR notes initially obtained. Of these, 83 were manually selected for inclusion in the Amazon Mechanical Turk crowdsourcing tasks and 55 were ultimately retained following IRT analysis. A follow-up validation with a second Amazon Mechanical Turk task and IRT analysis confirmed that the 55 questions test a latent ability dimension for EHR note comprehension. A short test of 14 items was created along with the 55-item test. CONCLUSIONS: We developed ComprehENotes, an instrument for assessing EHR note comprehension from existing EHR notes, gathered responses using crowdsourcing, and used IRT to analyze those responses, thus resulting in a set of questions to measure EHR note comprehension. Crowdsourced responses from Amazon Mechanical Turk can be used to estimate item parameters and select a subset of items for inclusion in the test set using IRT. The final set of questions is the first test of EHR note comprehension.
J Med Internet Res. 2018 Apr 25;20(4):e139. doi: 10.2196/jmir.9380. Link to article on publisher's site
Permanent Link to this Itemhttp://hdl.handle.net/20.500.14038/40634
RightsCopyright © John P Lalor, Hao Wu, Li Chen, Kathleen M Mazor, Hong Yu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 25.04.2018. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Except where otherwise noted, this item's license is described as Copyright © John P Lalor, Hao Wu, Li Chen, Kathleen M Mazor, Hong Yu. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 25.04.2018. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Showing items related by title, author, creator and subject.
Policy Brief: Addressing Social Determinants of Health through Community Health Workers: A Call to ActionLondon, Katharine; Damio, Grace; Ferrazo, Meredith; Perez-Escamalla, Rafael; Wiggins, Noelle (2018-01-30)This technical report was compiled by the Hispanic Health Council in partnership with Southwestern AHEC and a panel of Community Health Worker Policy Research Experts which included our Katharine London from the Center for Health Law and Economics. The report offers a number of policy recommendations for community health workers for communities that might benefit from community-based services. The report offers recommendations on; payment of community health workers; community health worker caseloads; community health worker recruitment; community health worker training; reflective and trauma-informed mentoring and supportive supervision of community health workers; integration of community health workers into care teams; documenting the effect of community heal worker services on social determination of health. The Hispanic Health Council believes a service design that effectively supports community health workers would incorporate the seven areas of policy recommendation included in this report.
A Public Health Framework for the State Mental Health Authority: A Call for Action by Massachusetts Consumers and Family MembersDelman, Jonathan (2006-01-01)During the Spring of 2006, Consumer Quality Initiatives (CQI) conducted 20 focus groups across the state, 12 with adults with mental illness, 3 with parents of youth with serious emotional disorder, 2 with youth with SED, 1 with family members of adult consumers, and 2 with youth in transition. Supported by a contract with Massachusetts Department of Mental Health (DMH), the goal was to assist DMH in framing the criteria for its upcoming reprocurement. Our findings reveal a frustration with an approach to health care delivery that focuses primarily on the provision of psychiatric care (egs, medication, therapy, hospitalization). We reviewed the focus group reports to identify the most significant themes, which clustered within eight broad categories.
Making the Case for Sustainable Funding for Community Health Worker Services: Talking to Payers and ProvidersLondon, Katharine (2018-01-27)In this presentation, Katharine London of the Center for Health Law and Economics makes her case for offering sustainable funding for community health worker services. Research has shown community health workers can have a distinct impact on health systems, helping them improve population health and contain costs, while also promoting health equity and community engagement. This presentation was designed to assist CHWs and other advocates in engaging with policymakers and payers to support CHW sustainability and develop a financial plan for their CHW work. It was presented as part of a CHW Sustainability event held at the Families USA’s annual conference, Health Action 2018: Staying Strong for America’s Families, in Washington, DC. See Katharine London's blog post on payment delivery methods for community health workers here.