• Login
    View Item 
    •   Home
    • UMass Chan Departments, Programs and Centers
    • Population and Quantitative Health Sciences
    • Population and Quantitative Health Sciences Publications
    • View Item
    •   Home
    • UMass Chan Departments, Programs and Centers
    • Population and Quantitative Health Sciences
    • Population and Quantitative Health Sciences Publications
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Browse

    All of eScholarship@UMassChanCommunitiesPublication DateAuthorsUMass Chan AffiliationsTitlesDocument TypesKeywordsThis CollectionPublication DateAuthorsUMass Chan AffiliationsTitlesDocument TypesKeywordsProfilesView

    My Account

    LoginRegister

    Help

    AboutSubmission GuidelinesData Deposit PolicySearchingUsage StatisticsAccessibilityTerms of UseWebsite Migration FAQ

    Statistics

    Most Popular ItemsStatistics by CountryMost Popular Authors

    Qualifying Certainty in Radiology Reports through Deep Learning-Based Natural Language Processing

    • CSV
    • RefMan
    • EndNote
    • BibTex
    • RefWorks
    Authors
    Liu, Feifan
    Zhou, Peng
    Baccei, Steven J.
    Masciocchi, Mark J.
    Amornsiripanitch, Nita
    Kiefe, Catarina I.
    Rosen, Max P
    UMass Chan Affiliations
    Department of Radiology
    Departments of Population and Quantitative Health Sciences
    Document Type
    Journal Article
    Publication Date
    2021-10-01
    Keywords
    Artificial Intelligence and Robotics
    Diagnosis
    Radiology
    
    Metadata
    Show full item record
    Link to Full Text
    https://doi.org/10.3174/ajnr.a7241
    Abstract
    BACKGROUND AND PURPOSE: Communication gaps exist between radiologists and referring physicians in conveying diagnostic certainty. We aimed to explore deep learning-based bidirectional contextual language models for automatically assessing diagnostic certainty expressed in the radiology reports to facilitate the precision of communication. MATERIALS AND METHODS: We randomly sampled 594 head MR imaging reports from an academic medical center. We asked 3 board-certified radiologists to read sentences from the Impression section and assign each sentence 1 of the 4 certainty categories: "Non-Definitive," "Definitive-Mild," "Definitive-Strong," "Other." Using the annotated 2352 sentences, we developed and validated a natural language-processing system based on the start-of-the-art bidirectional encoder representations from transformers (BERT), which can capture contextual uncertainty semantics beyond the lexicon level. Finally, we evaluated 3 BERT variant models and reported standard metrics including sensitivity, specificity, and area under the curve. RESULTS: A kappa score of 0.74 was achieved for interannotator agreement on uncertainty interpretations among 3 radiologists. For the 3 BERT variant models, the biomedical variant (BioBERT) achieved the best macro-average area under the curve of 0.931 (compared with 0.928 for the BERT-base and 0.925 for the clinical variant [ClinicalBERT]) on the validation data. All 3 models yielded high macro-average specificity (93.13%-93.65%), while the BERT-base obtained the highest macro-average sensitivity of 79.46% (compared with 79.08% for BioBERT and 78.52% for ClinicalBERT). The BioBERT model showed great generalizability on the heldout test data with a macro-average sensitivity of 77.29%, specificity of 92.89%, and area under the curve of 0.93. CONCLUSIONS: A deep transfer learning model can be developed to reliably assess the level of uncertainty communicated in a radiology report.
    Source

    Liu F, Zhou P, Baccei SJ, Masciocchi MJ, Amornsiripanitch N, Kiefe CI, Rosen MP. Qualifying Certainty in Radiology Reports through Deep Learning-Based Natural Language Processing. AJNR Am J Neuroradiol. 2021 Oct;42(10):1755-1761. doi: 10.3174/ajnr.A7241. Epub 2021 Aug 19. PMID: 34413062. Link to article on publisher's site

    DOI
    10.3174/ajnr.A7241
    Permanent Link to this Item
    http://hdl.handle.net/20.500.14038/46928
    PubMed ID
    34413062
    Related Resources

    Link to Article in PubMed

    ae974a485f413a2113503eed53cd6c53
    10.3174/ajnr.A7241
    Scopus Count
    Collections
    Population and Quantitative Health Sciences Publications
    Radiology Publications

    entitlement

    DSpace software (copyright © 2002 - 2023)  DuraSpace
    Lamar Soutter Library, UMass Chan Medical School | 55 Lake Avenue North | Worcester, MA 01655 USA
    Quick Guide | escholarship@umassmed.edu
    Works found in eScholarship@UMassChan are protected by copyright unless otherwise indicated.
    Open Repository is a service operated by 
    Atmire NV
     

    Export search results

    The export option will allow you to export the current search results of the entered query to a file. Different formats are available for download. To export the items, click on the button corresponding with the preferred download format.

    By default, clicking on the export buttons will result in a download of the allowed maximum amount of items.

    To select a subset of the search results, click "Selective Export" button and make a selection of the items you want to export. The amount of items that can be exported at once is similarly restricted as the full export.

    After making a selection, click one of the export format buttons. The amount of items that will be exported is indicated in the bubble next to export format.