Evaluation of a Pre-Made Expanding-Retrieval Flashcard Tool and Medical Student Performance on Step 1

Abstract

BACKGROUND: In recent years, many U.S. medical schools have moved away from pre-clinical grades, and in their stead the United States Medical Licensing Examination (USMLE) Step 1 examination has become an increasingly important measure of student performance and residency qualification. With this several new and well-designed tools geared toward Step 1 preparation have emerged. These materials offer improvements to what students study, but not necessarily how they study. For many, the prevalent means of content engagement is reading, note-taking, and revisiting at some unspecified and largely arbitrary point in the future. As a method of studying and long-term knowledge retention, this process is, in the parlance of medical students, low yield. Electronic flashcards, however, offer a potential solution to both how and when content is studied. By their very nature, they utilize at least some measure of active recall, test-enhanced learning, and corrective feedback. Several electronic flashcard programs also utilize expanding-retrieval algorithms that methodically distribute subsequent reviews of flashcards based on a combination of user familiarity with the content and the Ebbinghaus forgetting curve. The latter describes the rate at which learned information is forgotten if there is not subsequent review, but also demonstrates how each instance of re-learning retards this rate of decay. Previous studies have demonstrated the benefits of expanding-interval retrieval in medical education, but only in narrow, subject-specific areas. As a learning tool whose benefit is theoretically greatest when the breadth of knowledge is large and the duration of study is long, expanding-retrieval flashcards may prove even more beneficial in the context of USMLE Step 1 exam preparation. We sought to test this using a pre-made, freely-distributed electronic flashcard deck covering the breadth of Step 1 material, evaluating the student engagement with and perception of this resource, as well as their scores on standardized exams, including Step 1. OBJECTIVES: Evaluate the efficacy and medical student opinion of pre-made, expanding-retrieval electronic flashcards designed for daily, longitudinal use in preparation for the USMLE Step 1 exam. METHODS: 3000 electronic flashcards covering the breadth Step 1 material were created using Anki, an expanding-retrieval electronic flashcard program. These flashcards – termed the Maimonideck – were distributed to interested students in the University of Massachusetts Medical School Class of 2019 preceding second-year coursework. Responding students were surveyed before the start of classes, following receipt of a National Board of Medical Examiners (NBME) exam score, and following receipt of Step 1 score. Surveys evaluated engagement with the flashcards, use of supplementary question banks, and performance on the associated exam. RESULTS: Of 125 available students, 109 requested access, with 88 (81%) completing the baseline survey, 44 (40%) completing the NBME survey, and 31 (28%) completing the Step 1 survey. The mean NBME score for all Maimonideck users was 79.4, compared to class average of 74.5 and calculated non-user average of 71.9. The mean Step 1 score for all Maimonideck users was 243.8, compared to a class average of 233 and calculated non-user average of 229.3. Nearly all (93%) respondents agreed that the Maimonideck was an effective learning tool; 91% would recommend it to a classmate. CONCLUSIONS: Pre-made, expanding-retrieval electronic flashcards are an effective means of longitudinal study for Step 1 for a large minority of students. Similar resources should be considered by individuals and institutions for Step 1 preparation and other stages of medical training.