Loading...
Thumbnail Image
Publication

Improved Performance of ChatGPT-4 on the OKAP Examination: A Comparative Study with ChatGPT-3.5

Teebagy, Sean
Colwell, Lauren
Wood, Emma
Yaghy, Antonio
Faustina, Misha
Citations
Google Scholar:
Altmetric:
Student Authors
Faculty Advisor
Academic Program
UMass Chan Affiliations
Document Type
Journal Article
Publication Date
2023-09-11
Subject Area
Embargo Expiration Date
Abstract

This study aims to evaluate the performance of ChatGPT-4, an advanced artificial intelligence (AI) language model, on the Ophthalmology Knowledge Assessment Program (OKAP) examination compared to its predecessor, ChatGPT-3.5.  Both models were tested on 180 OKAP practice questions covering various ophthalmology subject categories.  ChatGPT-4 significantly outperformed ChatGPT-3.5 (81% vs. 57%; <0.001), indicating improvements in medical knowledge assessment.  The superior performance of ChatGPT-4 suggests potential applicability in ophthalmologic education and clinical decision support systems. Future research should focus on refining AI models, ensuring a balanced representation of fundamental and specialized knowledge, and determining the optimal method of integrating AI into medical education and practice.

Source

Teebagy S, Colwell L, Wood E, Yaghy A, Faustina M. Improved Performance of ChatGPT-4 on the OKAP Examination: A Comparative Study with ChatGPT-3.5. J Acad Ophthalmol (2017). 2023 Sep 11;15(2):e184-e187. doi: 10.1055/s-0043-1774399. PMID: 37701862; PMCID: PMC10495224.

Year of Medical School at Time of Visit
Sponsors
Dates of Travel
DOI
10.1055/s-0043-1774399
PubMed ID
37701862
Other Identifiers
Notes
Funding and Acknowledgements
Corresponding Author
Related Resources

This article is based on a previously available preprint in medRxiv, https://doi.org/10.1101/2023.04.03.23287957.

Related Resources
Repository Citation
Rights
Copyright The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. ( https://creativecommons.org/licenses/by-nc-nd/4.0/ ).