A new measure of the cognitive, metacognitive, and experiential aspects of residents' learning

Abstract

PURPOSE: Psychometric data are presented for the Cognitive Behavior Survey: Residency Level (rCBS), a survey that profiles cognitive, metacognitive, and experiential aspects of residents' learning. METHOD: The authors asked 963 residents from seven medicine residencies of large academic medical centers to participate in their study and gathered data from the respondents during a three-year period, 2000-2002. A factor analysis cross-validation design guided the development of rCBS's seven scales: memorization, conceptualization, reflection, independent learning, critical thinking, meaningful learning experience, and attitude toward educational experience. Interscale correlations and MANOVA provided preliminary evidence of scale construct validity. RESULTS: A total of 424 residents (44%) responded. With several minor exceptions, items for each scale loaded .40 or higher. Memorization did not correlate with any other scale, except correlating negatively with critical thinking. Higher-order thinking scales (conceptualization, reflection, independent learning, critical thinking) correlated with one another and with meaningful learning experience and attitude toward educational experience. The one exception: conceptualization did not correlate with critical thinking. MANOVA results reveal that residents who scored in the top 20% on the reflection scale conceptualized, learned independently, and thought critically more than did the bottom 20%. CONCLUSIONS: Results provide preliminary support for scale reliability and construct validity. As residencies seek to meet expectations of the Accreditation Council for Graduate Medical Education's Outcome Project, rCBS could prove useful in program evaluation, residents' self-assessment, and assessment by serving as a means to explore how residents learn, how residency programs affect learning behavior, and how clinically strong and weak residents differ in learning behaviors.