If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The COVID-19 pandemic prompted the virtualization of historically in-person radiology rotations for medical students. As students return to in-person clinical education, there is an opportunity to reevaluate teaching strategies and incorporate best practices from the pandemic. We describe our experience with the conversion of a four-week radiology clerkship from an in-person (IP) to remote learning (RL) to hybrid model (HM) and its impact on student performance and satisfaction.
Materials and Methods
Stratified by curriculum (Group 1 IP, Group 2 RL, Group 3 HM), student standardized final examination scores, final grades, lecture evaluation scores, and satisfaction scores were compared. Additional analysis was performed for Group 3 clinical divisions in which IP or RL models predominated.
Results
A significant decrease in mean final exam score was noted in Group 2 (p < 0.0001). Average lecture rating decreased in Group 3 compared to Group 1 (p < 0.001). Group 3 students reported improved faculty (Group 1: 59, Group 2: 61, Group 3: 82; p < 0.001) and resident (Group 1: 76.5, Group 2: 68, Group 3: 90; p < 0.001) teaching effectiveness. Student-reported quantity and quality of formative feedback were also highest for Group 3 (Quantity; Group 1: 60.6, Group 2: 74, Group 3: 93; p < 0.001) (Quality; Group 1: 59.1, Group 2: 77, Group 3: 97; p < 0.001). Group 3 subanalysis demonstrated increased student-perceived usefulness of activities within IP divisions (p < 0.01) and a decrease for RL divisions (p < 0.05).
Conclusion
A hybrid curriculum resulted in improved student satisfaction and preserved student performance after an emergent conversion to remote learning.
Clinical rotations are a cornerstone of undergraduate medical education, constituting a large portion of medical school curricula. While only 16% of medical schools in the United States require a dedicated radiology clerkship, radiology is widely perceived as an important curricular component with 96% of medical schools incorporating radiology content in coursework (
). Most undergraduate clinical radiology education occurs by way of traditional block rotations, which rely heavily on in-person, apprenticeship-style interactions between medical students and radiologists, trainees, and technologists, including participation in imaging interpretations and procedures (
The abrupt onset of the COVID-19 pandemic has had a profound and ongoing effect on clinical undergraduate medical education, given the historic emphasis on face-to-face learning. On March 17, 2020, the Association of American Medical Colleges issued a statement recommending a temporary suspension of medical student participation on clinical rotations (
Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
The design and preliminary experience with a virtual diversity visiting student acting internship in radiology for underrepresented minority medical students.
), had to emergently restructure, with medical educators forced to balance the constraints of COVID-19 mitigation policies with the requirements of undergraduate medical education (
). Our institution suspended second-year (i.e., the core clinical year) medical student participation in direct patient care activities on March 17, 2020. Our Department of Radiology rapidly transitioned from an in-person to a remote learning clerkship model, which was later converted to a hybrid model incorporating both in-person and remote learning components (Table 1).
Table 1Timeline of the COVID-19-Mediated Evolution of the Radiology Clerkship
March 17, 2020
Association of American Medical College press release addressing medical student participation on clinical rotations
March 17, 2020
Suspension of in-person clinical involvement for second-year medical students
March 17, 2020
First day of radiology clerkship under a remote learning model
June 15, 2020
Re-immersion of second-year medical students into in-person clinical environments
August 21, 2020
Final day of radiology clerkship under a remote learning model
August 24, 2020
First day of radiology clerkship under a hybrid model (ongoing)
As the thralls of the COVID-19 pandemic wane and medical students have returned to in-person clinical education, there is an opportunity to reevaluate existing teaching strategies to incorporate best practices learned from the pandemic. While numerous studies report positive learning outcomes within COVID-19-driven virtual radiology rotations, several emphasize the student- and faculty-perceived value of live instruction and the importance of in-person education in the reading room (
Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
). Few, however, provide concrete guidance on the next best steps for “post”-COVID radiology curricula. Blended learning models, or the balanced integration of in-person and technology-assisted (i.e., e-learning) environments, offer a promising approach (
In this article, we describe the COVID-19-mediated conversion of a radiology clerkship from an in-person (IP) to remote learning (RL) to a hybrid model (HM) and investigate the impact on student performance and satisfaction.
MATERIALS AND METHODS
This retrospective study was deemed exempt by the Institutional Review Board.
Structure of the Traditional IP Clerkship
Duke University, School of Medicine is an academic institution that has approximately 123 medical students per class who undergo core clinical rotations during their second year. The Radiology clerkship is housed within a large tertiary and quaternary care teaching hospital and is historically conducted as a required four-week in-person rotation. The student population included almost all second-year medical students and a select number of fourth-year medical students on a self-selected primary care track who took a near-identical clerkship in their fourth year rather than their second year.
Students participated in approximately 40 live in-person lecture hours and 140 in-person clinical hours across a student-dependent combination of nine subspecialty subrotations (e.g., breast imaging, nuclear medicine). Additional required didactic activities included resident teaching conferences, departmental conferences (e.g., Grand Rounds), multispecialty conferences (e.g., tumors boards), hands-on workshops (e.g., US-guided biopsy), and subrotation-dependent ancillary materials (e.g., guided review of archived cases). Evaluation of student performance relied on subrotation-specific assignments (quizzes and case presentations), subjective evaluations of clinical work, two presentations (one clinical case and one imaging utilization), and two multiple-choice exams (midterm and final), which culminated in an Honors/High Pass/Pass/Fail grade. The course director met with students at the midpoint of the clerkship to provide and receive feedback.
Structure of the RL Clerkship
Conversion of the traditional IP clerkship to a RL format necessitated an iterative restructuring of course content and delivery. While the overarching requirements were retained, the course was initially split into two phases: (1) a didactic phase, where all lectures and exams occurred remotely; and a (2) clinical phase, where a subrotation-dependent mixture of remote and/or in-person participation within the department occurred. Students participated in approximately 40 hours of live videoconference lecture hours and 140 remote clinical hours (videoconference readouts and conferences) with the opportunity to meet remotely with subspecialty radiologists during “office hours” as needed. The clinical component of videoconference readouts focused on the examinations reviewed by trainees (e.g., residents); however, the educational component was targeted to all learners in virtual attendance, including medical students. Videoconference readouts were scheduled throughout the workday at the discretion of the subspecialty attending radiologists. In subsequent rotations, a predominantly remote structure was maintained, however with reintegration of lecture hours and clinical hours. While the student population and methods of evaluation were maintained, due to the stressors of the COVID-19 pandemic, final student grades were converted to a Satisfactory/Unsatisfactory scale.
Structure of the HM Clerkship
In academic year (AY) 2020-2021, major revisions were implemented to the content and structure of the clerkship, including the addition of several interactive lectures. As restrictions on in-person medical student engagement were lifted, the clerkship was restructured to emphasize blended learning, combining the ease of virtual didactics with the return to completely in-person instruction in the reading room. Second- and fourth-year medical students participated in approximately 14 live videoconference lecture hours, 25 prerecorded lecture hours, 70 in-person clinical hours, and 70 remote clinical hours (videoconference readouts and conferences) across eight subspecialty subrotations. A sample student clerkship schedule is provided in Figure 1.
Figure 1Sample medical student schedule on HM radiology clerkship. PACS, picture archiving and communication system; MSK, musculoskeletal; VIR, vascular and interventional radiology.
Evaluation of student performance under the HM clerkship was modified to include subjective evaluations of clinical work, one case presentation, one multiple-choice final exam, and one oral final exam, culminating in a Satisfactory/Unsatisfactory grade. A team of four third- and fourth-year medical student teaching assistants was also added as a separate innovation, who helped co-facilitate teaching sessions and provide logistical and educational support for clerkship students. A summary of changes between the IP, RL, and HM clerkships is provided in Figure 2.
Figure 2Summary of changes made between IP, RL, and HM radiology clerkship models. CT, computed tomography; MRI, magnetic resonance imaging; GI, gastrointestinal; US, ultrasound; MSK, musculoskeletal.
Return to the Reading Room: IP Elements of the HM Clerkship
In-Person Subrotations
Students spent two one-week blocks on two student-prioritized subrotations; the remaining two weeks were divided between the other six subrotations. Of the 140 total clinical hours, approximately 70 were in-person and 70 were remote (videoconference readouts and conferences), with the precise division between IP and RL dependent on subrotation preference. Clinical responsibilities similarly varied, but included observation and participation in patient rounds, procedures, imaging studies, and case readouts.
Virtualizing Radiology Education: RL Elements of the HM Clerkship
Prerecorded Lectures and Virtual Didactics
Our institution provides students and staff with licensed accounts on the Zoom videoconference platform and free accounts on the WebEx videoconference platform. Medical students are also provided with instructions to download the institution's picture archiving and communication system (PACS) client to their personal laptops, which can be accessed at any time via an institutional virtual private network.
Orientation, a departmental tour, and PACS training were presented live over Zoom videoconference. PACS training included guidance on how to access patient information and academic folders and use key features (e.g., scroll, zoom, window/level adjustment). Of the required lectures, some lecturers presented their talks live over Zoom (14 hours); others pre-recorded content (25 hours) where recordings and other materials were made available to students via an institutional learning management system for self-directed study (Table A1). Where possible, resident teaching conferences, departmental conferences, pertinent multispecialty conferences, and other subrotation-dependent didactics occurred in-person; virtual alternatives, largely via Zoom, occurred at the discretion of the subrotation or specific conference.
Interactive Chest Workshops
Two interactive, half-day-long chest radiograph workshops were administered via Zoom by a senior faculty cardiothoracic radiologist. Prior to the first workshop, students accessed introductory educational material for independent review and completed a single mock dictation of a chest radiograph on PACS. The first workshop was spent establishing search patterns, reviewing anatomic structures, and discussing what vocabulary to use in chest radiograph interpretations. Prior to the second workshop, students completed several more mock dictations for commonly encountered pathologies, which were reviewed during the second workshop.
Interactive Mock Dictation Session
On the first day of the clerkship, students were assigned a group of ten unknown cases accessible via PACS and provided with blank dictation templates for each case (Fig 3). The selected cases included a variety of normal and pathologic studies across imaging modalities (XR, CT, and US) and patient ages. Mock dictations were to be completed prior to an end-of-week group meeting, during which the cases were discussed using Zoom's chat, screen sharing, and annotation features. This session was co-facilitated by teaching assistants and offered a dedicated opportunity for clerkship students to ask questions about the clerkship experience and share feedback.
Figure 3Sample dictation template and representative ultrasound image. MRN, medical record number; ACC, accession; US, ultrasound; RUQ, right upper quadrant.
As a course assignment, students were required to create and give oral case presentations via Zoom on a patient and topic of their choice. Representative topics included imaging findings of particular pathologies (e.g., moyamoya), the role of imaging in patient treatment (e.g., tumor embolization), and appropriate imaging workups (e.g., for a breast mass). The week prior to the case presentation date, a clerkship co-director or teaching assistant gave an interactive case presentation preparation session live via Zoom that reviewed the assignment rubric and tips for creating effective presentations. Students were also paired via Zoom's breakout rooms feature to trade presentation drafts and provide live feedback on one another's slides. After all case presentations were complete, a co-clerkship director met individually with students over Zoom to provide and receive feedback.
Methods of Clerkship Analysis
During the last week of each clerkship block (AYs 2016-2021), a standardized multiple-choice written exam was administered. At the end of each clerkship, all students were required to complete an anonymous online course evaluation on the Qualtrics platform. Stratified by curriculum (Group 1 IP, Group 2 RL, and Group 3 HM), student standardized final written examination scores, final grades, lecture evaluation scores, and student satisfaction scores were compared. Additional analysis was performed for Group 3 clinical divisions in which IP or RL predominated. Two-sample t-tests and three-way analyses of variance were conducted for comparisons between different means. Tukey's test was performed for post hoc analysis. A p value of <0.05 was considered statistically significant. Student comments were also compiled.
RESULTS
Student Enrollment
Between AYs 2016-2021, 549 total medical students participated in the core radiology clerkship (Table 2). 100% of students completed the course evaluation.
Table 2Characteristics of Group 1 (IP), Group 2 (RL), and Group 3 (HM) Students
Group 1
Group 2
Group 3
Date Range
August 22, 2016 to March 16, 2020
March 17, 2020 to August 21, 2020
August 24, 2020 to August 20, 2021
Total Student Enrollment
371
71
107
Number of Block Rotations
42
6
12
IP, in-person; RL, remote learning; HM, hybrid model
No reports of student mistreatment (abuse or neglect) occurred in Groups 2 or 3, compared to two per year in Group 1. No students in any group had a Fail/Unsatisfactory grade or received an exam score that prevented them from passing the course.
No students in any group had a failing final exam score or received an Unsatisfactory grade.
Student Satisfaction
Average lecture rating decreased from 4.5 in Group 1 to 4.2 in Group 3 (p < 0.001). The lecture rating in Group 2 was 4.0; however, due to the small number of students in Group 2, differences between Group 2 and other groups did not reach significance.
Group 3 students reported significantly improved faculty and resident teaching effectiveness (p < 0.01) (Fig 5).
Figure 5Percentage of medical students who rated faculty and resident teaching effectiveness "favorable” or “very favorable.” IP, in-person; RL, remote learning; HM, hybrid model.
Student-reported quantity and quality of formative feedback was also significantly highest for Group 3 (p < 0.01) (Fig 6).
Figure 6Percentage of medical students who rated quantity and quality of formative feedback "favorable” or “very favorable.” IP, in-person; RL, remote learning; HM, hybrid model.
Students were asked to rate the usefulness of clerkship activities in helping them meet clerkship goals (e.g., understanding imaging techniques) on a five-point Likert scale. Group 3 subanalysis demonstrated significantly increased student-perceived usefulness for IP-predominant subrotations (p < 0.01) and a significantly decreased score for RL-predominant subrotations (p < 0.05) (Fig 7).
Figure 7Percentage of medical students who reported “favorable” or “very favorable” usefulness scores. IP, in-person; RL, remote learning; HM, hybrid model.
Within the comments from Group 3 students, 73% of student suggestions for improvement were centered around remote readouts. Representative comments are shown in Table 3.
Table 3Representative Anonymized Student Comments from Group 3 (HM) Students
I recognize that readout schedules vary in frequency and duration, but it felt disappointing when my only exposure to certain services was a brief virtual readout in which teaching/asking questions was not incorporated, and I primarily watched passively
It was difficult to stay engaged and learn over zoom. Sometimes attendings wouldn't remember to call the med students, or the readouts happened so quickly there weren't great learning opportunities.
the Webex readouts were often not an effective teaching tool because they are meant for the residents and there doesn't seem to be adequate time/space to teach medical students in that space
Virtual readouts are difficult. The one opportunity I was given to readout with the attending in person was so helpful to have someone point things out on the screen and directly engage with me. I felt safe doing this, as it was just two of us at a computer with masks on.
Our results demonstrate that a hybrid radiology clerkship curriculum improved student satisfaction and preserved student performance compared to purely in-person or remote-learning curricula. Group 3 (HM) students reported the highest ratings for teaching effectiveness and feedback quantity and quality, which may reflect the increased opportunities for meaningful student-instructor exchanges in in-person clinical settings and in several new interactive teaching sessions (e.g., mock dictations). Importantly, Group 3 (HM) students also demonstrated a significantly more favorable appraisal of subrotations that were predominantly in-person rather than remote. Several student comments cited the “helpful” nature of in-person clinical education, as opposed to “difficult” and less effective virtual readouts. Although average lecture ratings decreased significantly between Group 1 (IP) and Group 3 (HM) students, we believe this may represent the cumulative mental fatigue that accompanies extended periods of virtual learning (
). This concept was particularly applicable to our Group 3 (HM) students, as they began clinical rotations after over three months of virtual preclinical education. An additional contributor to lecture satisfaction may have been differences in lecture format: Group 1 (IP) students had exclusively live lectures whereas Group 3 (HM) students had a combination of live and prerecorded lectures, the latter of which has a reduced capacity for meaningful interaction with educators (Table A1).
Table A1Lecture Concepts by Organ System Under the HM Model
Regarding course outcomes, the mean final exam score was restored in Group 3 (HM) after declining in Group 2 (RL). This pattern likely reflects the acute and unfamiliar stress that the COVID-19 pandemic placed on Group 2 (RL) students at its inception, compared to the adjustment period that Group 3 (HM) students were afforded. In addition, Group 2 (RL) students were required to learn four weeks of curricular material in a two-week period and had to take their final exams before any clinical experience was offered; clinical application of studied material would likely have helped solidify the content. Group 3 (HM) students additionally had the novel opportunity to take advantage of medical student teaching assistants, who provided peer support and an interactive examination review session.
Reports of student mistreatment (abuse or neglect) also declined from two per year in Group 1 (IP) to none in Groups 2 (RL) and 3 (HM). Due to the short time frame and smaller population, this decrease may not be significant, but the trend may suggest that students attributed possible negative interactions with faculty/residents to the stressful learning adaptations (i.e., learning environment differences due to COVID-19 policies) rather than mistreatment.
This study has several limitations. Although our overall sample size is adequate, the sizes of Groups 2 (RL) and 3 (HM) are small relative to Group 1 (IP), limiting the ability for us to reach statistical significance for some differences such as lecture ratings. Pre-/post-rotation assessments were also not performed; instead, our conclusions rely exclusively on end-of-rotation examinations and post-rotation surveys that may be influenced by the time of year or recall bias. Additionally, several unique variables were introduced in Group 3 (HM) that were not present in Groups 1 (IP) or 2 (RL), including new leadership and a new team of medical student teaching assistants.
Altogether, these findings suggest that a hybrid radiology educational model offers a unique opportunity to thoughtfully integrate the most successful aspects of traditional in-person and remote “COVID-era” methods. Future hybrid models should emphasize the maintenance of in-person immersive experiences (e.g., procedures, reading room participation) over videoconference readouts, which were important both for our students and those in existing literature (
Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
). In-person clinical education offers significant merits, including the opportunity to foster personal relationships with faculty and peers, directly observe the activities of radiologists, participate in interventions, and witness “live” cases unfold in the reading room (
). As such, our current clerkship model exclusively features live participation in the reading room. Similarly, the flexibility and independence afforded by virtual didactics cannot be overlooked; the use of interactive teaching sessions (e.g., mock dictations, participatory or flipped lectures) is convenient while maintaining high-quality learning outcomes (
Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
). However, the balanced integration of the two formats is critical to prevent strained clinical environments or “Zoom fatigue.”
CONCLUSION
Our findings suggest that a hybrid radiology clerkship model can preserve medical student performance while improving student satisfaction relative to purely in-person or remote learning models. Hybrid educational models offer a unique opportunity to leverage the best practices of both in-person and remote learning methods.
Acknowledgments
None.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
The design and preliminary experience with a virtual diversity visiting student acting internship in radiology for underrepresented minority medical students.
Exclusively presented in recorded lecture content.COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.
00481-0&id=gr1.jpg){kind=link}
00481-0&id=gr2.jpg){kind=link}
00481-0&id=gr3.jpg){kind=link}
00481-0&id=gr4.jpg){kind=link}
00481-0&id=gr5.jpg){kind=link}
00481-0&id=gr6.jpg){kind=link}
00481-0&id=gr7.jpg){kind=link}