Show simple item record

dc.contributor.authorLiu, Feifan
dc.contributor.authorPradhan, Richeek
dc.contributor.authorDruhl, Emily
dc.contributor.authorFreund, Elaine
dc.contributor.authorLiu, Weisong
dc.contributor.authorSauer, Brian C.
dc.contributor.authorCunningham, Fran
dc.contributor.authorGordon, Adam J.
dc.contributor.authorPeters, Celena B.
dc.contributor.authorYu, Hong
dc.date2022-08-11T08:10:35.000
dc.date.accessioned2022-08-23T17:13:44Z
dc.date.available2022-08-23T17:13:44Z
dc.date.issued2019-04-29
dc.date.submitted2019-06-07
dc.identifier.citation<p>J Am Med Inform Assoc. 2019 Apr 29. pii: ocz048. doi: 10.1093/jamia/ocz048. [Epub ahead of print] <a href="https://doi.org/10.1093/jamia/ocz048">Link to article on publisher's site</a></p>
dc.identifier.issn1067-5027 (Linking)
dc.identifier.doi10.1093/jamia/ocz048
dc.identifier.pmid31034028
dc.identifier.urihttp://hdl.handle.net/20.500.14038/46797
dc.description.abstractOBJECTIVE: Identifying drug discontinuation (DDC) events and understanding their reasons are important for medication management and drug safety surveillance. Structured data resources are often incomplete and lack reason information. In this article, we assessed the ability of natural language processing (NLP) systems to unlock DDC information from clinical narratives automatically. MATERIALS AND METHODS: We collected 1867 de-identified providers' notes from the University of Massachusetts Medical School hospital electronic health record system. Then 2 human experts chart reviewed those clinical notes to annotate DDC events and their reasons. Using the annotated data, we developed and evaluated NLP systems to automatically identify drug discontinuations and reasons at the sentence level using a novel semantic enrichment-based vector representation (SEVR) method for enhanced feature representation. RESULTS: Our SEVR-based NLP system achieved the best performance of 0.785 (AUC-ROC) for detecting discontinuation events and 0.745 (AUC-ROC) for identifying reasons when testing this highly imbalanced data, outperforming 2 state-of-the-art non-SEVR-based models. Compared with a rule-based baseline system for discontinuation detection, our system improved the sensitivity significantly (57.75% vs 18.31%, absolute value) while retaining a high specificity of 99.25%, leading to a significant improvement in AUC-ROC by 32.83% (absolute value). CONCLUSION: Experiments have shown that a high-performance NLP system can be developed to automatically identify DDCs and their reasons from providers' notes. The SEVR model effectively improved the system performance showing better generalization and robustness on unseen test data. Our work is an important step toward identifying reasons for drug discontinuation that will inform drug safety surveillance and pharmacovigilance.
dc.language.isoen_US
dc.relation<p><a href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=pubmed&cmd=Retrieve&list_uids=31034028&dopt=Abstract">Link to Article in PubMed</a></p>
dc.rightsThis work is written by US Government employees and is in the public domain in the US.
dc.subjectdrug surveillance
dc.subjectelectronic health records
dc.subjectknowledge representation
dc.subjectnatural language processing
dc.subjectsupervised machine learning
dc.subjectArtificial Intelligence and Robotics
dc.subjectBioinformatics
dc.subjectDatabases and Information Systems
dc.subjectDiagnosis
dc.subjectHealth Information Technology
dc.subjectHealth Services Administration
dc.subjectHealth Services Research
dc.titleLearning to detect and understand drug discontinuation events from clinical narratives
dc.typeJournal Article
dc.source.journaltitleJournal of the American Medical Informatics Association : JAMIA
dc.identifier.legacyfulltexthttps://escholarship.umassmed.edu/cgi/viewcontent.cgi?article=2266&amp;context=qhs_pp&amp;unstamped=1
dc.identifier.legacycoverpagehttps://escholarship.umassmed.edu/qhs_pp/1264
dc.identifier.contextkey14691255
refterms.dateFOA2022-08-23T17:13:44Z
html.description.abstract<p>OBJECTIVE: Identifying drug discontinuation (DDC) events and understanding their reasons are important for medication management and drug safety surveillance. Structured data resources are often incomplete and lack reason information. In this article, we assessed the ability of natural language processing (NLP) systems to unlock DDC information from clinical narratives automatically.</p> <p>MATERIALS AND METHODS: We collected 1867 de-identified providers' notes from the University of Massachusetts Medical School hospital electronic health record system. Then 2 human experts chart reviewed those clinical notes to annotate DDC events and their reasons. Using the annotated data, we developed and evaluated NLP systems to automatically identify drug discontinuations and reasons at the sentence level using a novel semantic enrichment-based vector representation (SEVR) method for enhanced feature representation.</p> <p>RESULTS: Our SEVR-based NLP system achieved the best performance of 0.785 (AUC-ROC) for detecting discontinuation events and 0.745 (AUC-ROC) for identifying reasons when testing this highly imbalanced data, outperforming 2 state-of-the-art non-SEVR-based models. Compared with a rule-based baseline system for discontinuation detection, our system improved the sensitivity significantly (57.75% vs 18.31%, absolute value) while retaining a high specificity of 99.25%, leading to a significant improvement in AUC-ROC by 32.83% (absolute value).</p> <p>CONCLUSION: Experiments have shown that a high-performance NLP system can be developed to automatically identify DDCs and their reasons from providers' notes. The SEVR model effectively improved the system performance showing better generalization and robustness on unseen test data. Our work is an important step toward identifying reasons for drug discontinuation that will inform drug safety surveillance and pharmacovigilance.</p>
dc.identifier.submissionpathqhs_pp/1264
dc.contributor.departmentDepartment of Medicine
dc.contributor.departmentDepartment of Population and Quantitative Health Sciences


Files in this item

Thumbnail
Name:
ocz048.pdf
Size:
377.3Kb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record