Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study
Kobeissi, Hassan ; Kallmes, David F ; Benson, John ; Nagelschneider, Alex ; Madhavan, Ajay ; Messina, Steven A ; Schwartz, Kara ; Campeau, Norbert ; Carr, Carrie M ; Nasr, Deena M ... show 10 more
Citations
Authors
Kallmes, David F
Benson, John
Nagelschneider, Alex
Madhavan, Ajay
Messina, Steven A
Schwartz, Kara
Campeau, Norbert
Carr, Carrie M
Nasr, Deena M
Braksick, Sherri
Scharf, Eugene L
Klaas, James
Woodhead, Zoe Victoria Joan
Harston, George
Briggs, James
Joly, Olivier
Gerry, Stephen
Kuhn, Anna Luisa
Kostas, Angelos A
Nael, Kambiz
AbdalKader, Mohamad
Kadirvel, Ramanathan
Brinjikji, Waleed
Student Authors
Faculty Advisor
Academic Program
UMass Chan Affiliations
Document Type
Publication Date
Keywords
Subject Area
Collections
Embargo Expiration Date
Link to Full Text
Abstract
Background: The Alberta Stroke Program Early CT Score (ASPECTS) is used to quantify the extent of injury to the brain following acute ischemic stroke (AIS) and to inform treatment decisions. The e-ASPECTS software uses artificial intelligence methods to automatically process non-contrast CT (NCCT) brain scans from patients with AIS affecting the middle cerebral artery (MCA) territory and generate an ASPECTS. This study aimed to evaluate the impact of e-ASPECTS (Brainomix, Oxford, UK) on the performance of US physicians compared to a consensus ground truth.
Methods: The study used a multi-reader, multi-case design. A total of 10 US board-certified physicians (neurologists and neuroradiologists) scored 54 NCCT brain scans of patients with AIS affecting the MCA territory. Each reader scored each scan on two occasions: once with and once without reference to the e-ASPECTS software, in random order. Agreement with a reference standard (expert consensus read with reference to follow-up imaging) was evaluated with and without software support.
Results: A comparison of the area under the curve (AUC) for each reader showed a significant improvement from 0.81 to 0.83 (p = 0.028) with the support of the e-ASPECTS tool. The agreement of reader ASPECTS scoring with the reference standard was improved with e-ASPECTS compared to unassisted reading of scans: Cohen's kappa improved from 0.60 to 0.65, and the case-based weighted Kappa improved from 0.70 to 0.81.
Conclusion: Decision support with the e-ASPECTS software significantly improves the accuracy of ASPECTS scoring, even by expert US neurologists and neuroradiologists.
Source
Kobeissi H, Kallmes DF, Benson J, Nagelschneider A, Madhavan A, Messina SA, Schwartz K, Campeau N, Carr CM, Nasr DM, Braksick S, Scharf EL, Klaas J, Woodhead ZVJ, Harston G, Briggs J, Joly O, Gerry S, Kuhn AL, Kostas AA, Nael K, AbdalKader M, Kadirvel R, Brinjikji W. Impact of e-ASPECTS software on the performance of physicians compared to a consensus ground truth: a multi-reader, multi-case study. Front Neurol. 2023 Sep 7;14:1221255. doi: 10.3389/fneur.2023.1221255. PMID: 37745671; PMCID: PMC10513025.