Advertisement

Return to the Reading Room: Implementation of a Hybrid Radiology Clerkship Model after Emergent Conversion to Remote Learning in the COVID-19 Pandemic

Published:August 28, 2022DOI:https://doi.org/10.1016/j.acra.2022.08.027

      Rationale and Objectives

      The COVID-19 pandemic prompted the virtualization of historically in-person radiology rotations for medical students. As students return to in-person clinical education, there is an opportunity to reevaluate teaching strategies and incorporate best practices from the pandemic. We describe our experience with the conversion of a four-week radiology clerkship from an in-person (IP) to remote learning (RL) to hybrid model (HM) and its impact on student performance and satisfaction.

      Materials and Methods

      Stratified by curriculum (Group 1 IP, Group 2 RL, Group 3 HM), student standardized final examination scores, final grades, lecture evaluation scores, and satisfaction scores were compared. Additional analysis was performed for Group 3 clinical divisions in which IP or RL models predominated.

      Results

      A significant decrease in mean final exam score was noted in Group 2 (p < 0.0001). Average lecture rating decreased in Group 3 compared to Group 1 (p < 0.001). Group 3 students reported improved faculty (Group 1: 59, Group 2: 61, Group 3: 82; p < 0.001) and resident (Group 1: 76.5, Group 2: 68, Group 3: 90; p < 0.001) teaching effectiveness. Student-reported quantity and quality of formative feedback were also highest for Group 3 (Quantity; Group 1: 60.6, Group 2: 74, Group 3: 93; p < 0.001) (Quality; Group 1: 59.1, Group 2: 77, Group 3: 97; p < 0.001). Group 3 subanalysis demonstrated increased student-perceived usefulness of activities within IP divisions (p < 0.01) and a decrease for RL divisions (p < 0.05).

      Conclusion

      A hybrid curriculum resulted in improved student satisfaction and preserved student performance after an emergent conversion to remote learning.

      Key Words

      Abbreviations:

      AY (academic year), HM (hybrid model), IP (in-person), PACS (picture archiving and communication system), RL (remote learning)

      INTRODUCTION

      Clinical rotations are a cornerstone of undergraduate medical education, constituting a large portion of medical school curricula. While only 16% of medical schools in the United States require a dedicated radiology clerkship, radiology is widely perceived as an important curricular component with 96% of medical schools incorporating radiology content in coursework (

      Association of American Medical Colleges. Clerkship requirements by discipline. LCME Annual Medical School Questionnaire Part II, 2011-2012 through 2019-2020. Accessed January 15, 2022. Available at:https://www.aamc.org/data-reports/curriculum-reports/interactive-data/clerkship-requirements-discipline.

      ,
      • Kondo K.L.
      • Swerdlow M.
      Medical student radiology curriculum: what skills do residency program directors believe are essential for medical students to attain?.
      ,
      • Lee H.
      • Kim D.H.
      • Hong P.P.
      Radiology clerkship requirements in Canada and the United States: current state and impact on residency application.
      ,
      • Poot J.D.
      • Hartman M.S.
      • Daffner R.H.
      Understanding the US medical school requirements and medical students’ attitudes about radiology rotations.
      ,
      • du Cret R.P.
      • Weinberg E.
      • Sellers T.
      • et al.
      Role of radiology in medical education: perspective of nonradiologists.
      ,
      • Straus C.M.
      • Webb E.M.
      • Kondo K.L.
      • et al.
      Medical student radiology education: summary and recommendations from a national survey of medical school and radiology department leadership.
      ). Most undergraduate clinical radiology education occurs by way of traditional block rotations, which rely heavily on in-person, apprenticeship-style interactions between medical students and radiologists, trainees, and technologists, including participation in imaging interpretations and procedures (
      • Poot J.D.
      • Hartman M.S.
      • Daffner R.H.
      Understanding the US medical school requirements and medical students’ attitudes about radiology rotations.
      ,
      • Straus C.M.
      • Webb E.M.
      • Kondo K.L.
      • et al.
      Medical student radiology education: summary and recommendations from a national survey of medical school and radiology department leadership.
      ,
      • Barlev D.M.
      • Lautin E.M.
      • Amis E.S.
      • et al.
      A survey of radiology clerkships at teaching hospitals in the United States.
      ,
      • Slanetz P.J.
      • Naeger D.M.
      • Avery L.L.
      • et al.
      Mixed Practice in Radiology Education—Has the Time Come?.
      ).
      The abrupt onset of the COVID-19 pandemic has had a profound and ongoing effect on clinical undergraduate medical education, given the historic emphasis on face-to-face learning. On March 17, 2020, the Association of American Medical Colleges issued a statement recommending a temporary suspension of medical student participation on clinical rotations (
      • Prescott J.E.
      Important guidance for medical students on clinical rotations during the coronavirus (COVID-19) outbreak.
      ). Existing clinical experiences, including radiology electives (
      • Smith E.
      • Boscak A.
      A virtual emergency: learning lessons from remote medical student education during the COVID-19 pandemic.
      ,
      • Gomez E.
      • Azadi J.
      • Magid D.
      Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
      ,
      • DePietro D.M.
      • Santucci S.E.
      • Harrison N.E.
      • et al.
      Medical student education during the COVID-19 pandemic: initial experiences implementing a virtual interventional radiology elective course.
      ,
      • Belfi L.M.
      • Dean K.E.
      • Bartolotta R.J.
      • et al.
      Medical student education in the time of COVID-19: a virtual solution to the introductory radiology elective.
      ,
      • Dodelzon K.
      • Belfi L.
      • Schweitzer A.D.
      • Lamparello N.
      • Legasto A.C.
      The design and preliminary experience with a virtual diversity visiting student acting internship in radiology for underrepresented minority medical students.
      ,
      • Adams C.C.
      • Shih R.
      • Peterson P.G.
      • Lee M.H.
      • Heltzel D.A.
      • Lattin Jr., G.E.
      The impact of a virtual radiology medical student rotation: maintaining engagement during COVID-19 mitigation.
      ) and clerkships (
      • Durfee S.M.
      • Goldenson R.P.
      • Gill R.R.
      • Rincon S.P.
      • Flower E.
      • Aver L.L.
      Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue.
      ,
      • Alamer A.
      • Alharbi F.
      Synchronous distance teaching of radiology clerkship promotes medical students’ learning and engagement.
      ,
      • Goldenson R.P.
      • Avery L.L.
      • Gill R.R.
      • et al.
      The virtual homeroom: utility and benefits of small group online learning in the COVID-19 era.
      ,
      • Creagh S.
      • Pigg N.
      • Gordillo C.
      • et al.
      Virtual medical student radiology clerkships during the COVID-19 pandemic: distancing is not a barrier.
      ), had to emergently restructure, with medical educators forced to balance the constraints of COVID-19 mitigation policies with the requirements of undergraduate medical education (
      • Smith E.B.
      • Boscak A.
      • Friedman E.M.
      • et al.
      Radiology medical student education 2020: surveys of the alliance of medical student educators in radiology and medical students.
      ). Our institution suspended second-year (i.e., the core clinical year) medical student participation in direct patient care activities on March 17, 2020. Our Department of Radiology rapidly transitioned from an in-person to a remote learning clerkship model, which was later converted to a hybrid model incorporating both in-person and remote learning components (Table 1).
      Table 1Timeline of the COVID-19-Mediated Evolution of the Radiology Clerkship
      March 17, 2020Association of American Medical College press release addressing medical student participation on clinical rotations
      March 17, 2020Suspension of in-person clinical involvement for second-year medical students
      March 17, 2020First day of radiology clerkship under a remote learning model
      June 15, 2020Re-immersion of second-year medical students into in-person clinical environments
      August 21, 2020Final day of radiology clerkship under a remote learning model
      August 24, 2020First day of radiology clerkship under a hybrid model (ongoing)
      As the thralls of the COVID-19 pandemic wane and medical students have returned to in-person clinical education, there is an opportunity to reevaluate existing teaching strategies to incorporate best practices learned from the pandemic. While numerous studies report positive learning outcomes within COVID-19-driven virtual radiology rotations, several emphasize the student- and faculty-perceived value of live instruction and the importance of in-person education in the reading room (
      • Smith E.
      • Boscak A.
      A virtual emergency: learning lessons from remote medical student education during the COVID-19 pandemic.
      ,
      • Gomez E.
      • Azadi J.
      • Magid D.
      Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
      ,
      • Adams C.C.
      • Shih R.
      • Peterson P.G.
      • Lee M.H.
      • Heltzel D.A.
      • Lattin Jr., G.E.
      The impact of a virtual radiology medical student rotation: maintaining engagement during COVID-19 mitigation.
      ,
      • Durfee S.M.
      • Goldenson R.P.
      • Gill R.R.
      • Rincon S.P.
      • Flower E.
      • Aver L.L.
      Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue.
      ,
      • Smith E.B.
      • Boscak A.
      • Friedman E.M.
      • et al.
      Radiology medical student education 2020: surveys of the alliance of medical student educators in radiology and medical students.
      ). Few, however, provide concrete guidance on the next best steps for “post”-COVID radiology curricula. Blended learning models, or the balanced integration of in-person and technology-assisted (i.e., e-learning) environments, offer a promising approach (
      • Smith E.B.
      • Boscak A.
      • Friedman E.M.
      • et al.
      Radiology medical student education 2020: surveys of the alliance of medical student educators in radiology and medical students.
      ,
      • Osguthorpe R.T.
      • Graham C.R.
      Blended learning environments: definitions and directions.
      ,
      • Ellaway R.
      • Masters K.
      AMEE Guide 32: e-Learning in medical education Part 1: learning, teaching and assessment.
      ).
      In this article, we describe the COVID-19-mediated conversion of a radiology clerkship from an in-person (IP) to remote learning (RL) to a hybrid model (HM) and investigate the impact on student performance and satisfaction.

      MATERIALS AND METHODS

      This retrospective study was deemed exempt by the Institutional Review Board.

      Structure of the Traditional IP Clerkship

      Duke University, School of Medicine is an academic institution that has approximately 123 medical students per class who undergo core clinical rotations during their second year. The Radiology clerkship is housed within a large tertiary and quaternary care teaching hospital and is historically conducted as a required four-week in-person rotation. The student population included almost all second-year medical students and a select number of fourth-year medical students on a self-selected primary care track who took a near-identical clerkship in their fourth year rather than their second year.
      Students participated in approximately 40 live in-person lecture hours and 140 in-person clinical hours across a student-dependent combination of nine subspecialty subrotations (e.g., breast imaging, nuclear medicine). Additional required didactic activities included resident teaching conferences, departmental conferences (e.g., Grand Rounds), multispecialty conferences (e.g., tumors boards), hands-on workshops (e.g., US-guided biopsy), and subrotation-dependent ancillary materials (e.g., guided review of archived cases). Evaluation of student performance relied on subrotation-specific assignments (quizzes and case presentations), subjective evaluations of clinical work, two presentations (one clinical case and one imaging utilization), and two multiple-choice exams (midterm and final), which culminated in an Honors/High Pass/Pass/Fail grade. The course director met with students at the midpoint of the clerkship to provide and receive feedback.

      Structure of the RL Clerkship

      Conversion of the traditional IP clerkship to a RL format necessitated an iterative restructuring of course content and delivery. While the overarching requirements were retained, the course was initially split into two phases: (1) a didactic phase, where all lectures and exams occurred remotely; and a (2) clinical phase, where a subrotation-dependent mixture of remote and/or in-person participation within the department occurred. Students participated in approximately 40 hours of live videoconference lecture hours and 140 remote clinical hours (videoconference readouts and conferences) with the opportunity to meet remotely with subspecialty radiologists during “office hours” as needed. The clinical component of videoconference readouts focused on the examinations reviewed by trainees (e.g., residents); however, the educational component was targeted to all learners in virtual attendance, including medical students. Videoconference readouts were scheduled throughout the workday at the discretion of the subspecialty attending radiologists. In subsequent rotations, a predominantly remote structure was maintained, however with reintegration of lecture hours and clinical hours. While the student population and methods of evaluation were maintained, due to the stressors of the COVID-19 pandemic, final student grades were converted to a Satisfactory/Unsatisfactory scale.

      Structure of the HM Clerkship

      In academic year (AY) 2020-2021, major revisions were implemented to the content and structure of the clerkship, including the addition of several interactive lectures. As restrictions on in-person medical student engagement were lifted, the clerkship was restructured to emphasize blended learning, combining the ease of virtual didactics with the return to completely in-person instruction in the reading room. Second- and fourth-year medical students participated in approximately 14 live videoconference lecture hours, 25 prerecorded lecture hours, 70 in-person clinical hours, and 70 remote clinical hours (videoconference readouts and conferences) across eight subspecialty subrotations. A sample student clerkship schedule is provided in Figure 1.
      Figure 1
      Figure 1Sample medical student schedule on HM radiology clerkship. PACS, picture archiving and communication system; MSK, musculoskeletal; VIR, vascular and interventional radiology.
      Evaluation of student performance under the HM clerkship was modified to include subjective evaluations of clinical work, one case presentation, one multiple-choice final exam, and one oral final exam, culminating in a Satisfactory/Unsatisfactory grade. A team of four third- and fourth-year medical student teaching assistants was also added as a separate innovation, who helped co-facilitate teaching sessions and provide logistical and educational support for clerkship students. A summary of changes between the IP, RL, and HM clerkships is provided in Figure 2.
      Figure 2
      Figure 2Summary of changes made between IP, RL, and HM radiology clerkship models. CT, computed tomography; MRI, magnetic resonance imaging; GI, gastrointestinal; US, ultrasound; MSK, musculoskeletal.

      Return to the Reading Room: IP Elements of the HM Clerkship

      In-Person Subrotations

      Students spent two one-week blocks on two student-prioritized subrotations; the remaining two weeks were divided between the other six subrotations. Of the 140 total clinical hours, approximately 70 were in-person and 70 were remote (videoconference readouts and conferences), with the precise division between IP and RL dependent on subrotation preference. Clinical responsibilities similarly varied, but included observation and participation in patient rounds, procedures, imaging studies, and case readouts.

      Virtualizing Radiology Education: RL Elements of the HM Clerkship

      Prerecorded Lectures and Virtual Didactics

      Our institution provides students and staff with licensed accounts on the Zoom videoconference platform and free accounts on the WebEx videoconference platform. Medical students are also provided with instructions to download the institution's picture archiving and communication system (PACS) client to their personal laptops, which can be accessed at any time via an institutional virtual private network.
      Orientation, a departmental tour, and PACS training were presented live over Zoom videoconference. PACS training included guidance on how to access patient information and academic folders and use key features (e.g., scroll, zoom, window/level adjustment). Of the required lectures, some lecturers presented their talks live over Zoom (14 hours); others pre-recorded content (25 hours) where recordings and other materials were made available to students via an institutional learning management system for self-directed study (Table A1). Where possible, resident teaching conferences, departmental conferences, pertinent multispecialty conferences, and other subrotation-dependent didactics occurred in-person; virtual alternatives, largely via Zoom, occurred at the discretion of the subrotation or specific conference.

      Interactive Chest Workshops

      Two interactive, half-day-long chest radiograph workshops were administered via Zoom by a senior faculty cardiothoracic radiologist. Prior to the first workshop, students accessed introductory educational material for independent review and completed a single mock dictation of a chest radiograph on PACS. The first workshop was spent establishing search patterns, reviewing anatomic structures, and discussing what vocabulary to use in chest radiograph interpretations. Prior to the second workshop, students completed several more mock dictations for commonly encountered pathologies, which were reviewed during the second workshop.

      Interactive Mock Dictation Session

      On the first day of the clerkship, students were assigned a group of ten unknown cases accessible via PACS and provided with blank dictation templates for each case (Fig 3). The selected cases included a variety of normal and pathologic studies across imaging modalities (XR, CT, and US) and patient ages. Mock dictations were to be completed prior to an end-of-week group meeting, during which the cases were discussed using Zoom's chat, screen sharing, and annotation features. This session was co-facilitated by teaching assistants and offered a dedicated opportunity for clerkship students to ask questions about the clerkship experience and share feedback.
      Figure 3
      Figure 3Sample dictation template and representative ultrasound image. MRN, medical record number; ACC, accession; US, ultrasound; RUQ, right upper quadrant.

      Case Presentations and Mid-Clerkship Review

      As a course assignment, students were required to create and give oral case presentations via Zoom on a patient and topic of their choice. Representative topics included imaging findings of particular pathologies (e.g., moyamoya), the role of imaging in patient treatment (e.g., tumor embolization), and appropriate imaging workups (e.g., for a breast mass). The week prior to the case presentation date, a clerkship co-director or teaching assistant gave an interactive case presentation preparation session live via Zoom that reviewed the assignment rubric and tips for creating effective presentations. Students were also paired via Zoom's breakout rooms feature to trade presentation drafts and provide live feedback on one another's slides. After all case presentations were complete, a co-clerkship director met individually with students over Zoom to provide and receive feedback.

      Methods of Clerkship Analysis

      During the last week of each clerkship block (AYs 2016-2021), a standardized multiple-choice written exam was administered. At the end of each clerkship, all students were required to complete an anonymous online course evaluation on the Qualtrics platform. Stratified by curriculum (Group 1 IP, Group 2 RL, and Group 3 HM), student standardized final written examination scores, final grades, lecture evaluation scores, and student satisfaction scores were compared. Additional analysis was performed for Group 3 clinical divisions in which IP or RL predominated. Two-sample t-tests and three-way analyses of variance were conducted for comparisons between different means. Tukey's test was performed for post hoc analysis. A p value of <0.05 was considered statistically significant. Student comments were also compiled.

      RESULTS

      Student Enrollment

      Between AYs 2016-2021, 549 total medical students participated in the core radiology clerkship (Table 2). 100% of students completed the course evaluation.
      Table 2Characteristics of Group 1 (IP), Group 2 (RL), and Group 3 (HM) Students
      Group 1Group 2Group 3
      Date RangeAugust 22, 2016 to March 16, 2020March 17, 2020 to August 21, 2020August 24, 2020 to August 20, 2021
      Total Student Enrollment37171107
      Number of Block Rotations42612
      IP, in-person; RL, remote learning; HM, hybrid model

      Course Outcomes

      Hundred percent of students completed the final written exam. A significant decrease in mean final exam score was noted in Group 2 (Fig 4).
      Figure 4
      Figure 4Average medical student final written exam score. IP, in-person; RL, remote learning; HM, hybrid model.
      No reports of student mistreatment (abuse or neglect) occurred in Groups 2 or 3, compared to two per year in Group 1. No students in any group had a Fail/Unsatisfactory grade or received an exam score that prevented them from passing the course.
      No students in any group had a failing final exam score or received an Unsatisfactory grade.

      Student Satisfaction

      Average lecture rating decreased from 4.5 in Group 1 to 4.2 in Group 3 (p < 0.001). The lecture rating in Group 2 was 4.0; however, due to the small number of students in Group 2, differences between Group 2 and other groups did not reach significance.
      Group 3 students reported significantly improved faculty and resident teaching effectiveness (p < 0.01) (Fig 5).
      Figure 5
      Figure 5Percentage of medical students who rated faculty and resident teaching effectiveness "favorable” or “very favorable.” IP, in-person; RL, remote learning; HM, hybrid model.
      Student-reported quantity and quality of formative feedback was also significantly highest for Group 3 (p < 0.01) (Fig 6).
      Figure 6
      Figure 6Percentage of medical students who rated quantity and quality of formative feedback "favorable” or “very favorable.” IP, in-person; RL, remote learning; HM, hybrid model.
      Students were asked to rate the usefulness of clerkship activities in helping them meet clerkship goals (e.g., understanding imaging techniques) on a five-point Likert scale. Group 3 subanalysis demonstrated significantly increased student-perceived usefulness for IP-predominant subrotations (p < 0.01) and a significantly decreased score for RL-predominant subrotations (p < 0.05) (Fig 7).
      Figure 7
      Figure 7Percentage of medical students who reported “favorable” or “very favorable” usefulness scores. IP, in-person; RL, remote learning; HM, hybrid model.

      Student Recommendations

      Within the comments from Group 3 students, 73% of student suggestions for improvement were centered around remote readouts. Representative comments are shown in Table 3.
      Table 3Representative Anonymized Student Comments from Group 3 (HM) Students
      I recognize that readout schedules vary in frequency and duration, but it felt disappointing when my only exposure to certain services was a brief virtual readout in which teaching/asking questions was not incorporated, and I primarily watched passively
      It was difficult to stay engaged and learn over zoom. Sometimes attendings wouldn't remember to call the med students, or the readouts happened so quickly there weren't great learning opportunities.
      the Webex readouts were often not an effective teaching tool because they are meant for the residents and there doesn't seem to be adequate time/space to teach medical students in that space
      Virtual readouts are difficult. The one opportunity I was given to readout with the attending in person was so helpful to have someone point things out on the screen and directly engage with me. I felt safe doing this, as it was just two of us at a computer with masks on.
      HM, hybrid model.

      DISCUSSION

      Our results demonstrate that a hybrid radiology clerkship curriculum improved student satisfaction and preserved student performance compared to purely in-person or remote-learning curricula. Group 3 (HM) students reported the highest ratings for teaching effectiveness and feedback quantity and quality, which may reflect the increased opportunities for meaningful student-instructor exchanges in in-person clinical settings and in several new interactive teaching sessions (e.g., mock dictations). Importantly, Group 3 (HM) students also demonstrated a significantly more favorable appraisal of subrotations that were predominantly in-person rather than remote. Several student comments cited the “helpful” nature of in-person clinical education, as opposed to “difficult” and less effective virtual readouts. Although average lecture ratings decreased significantly between Group 1 (IP) and Group 3 (HM) students, we believe this may represent the cumulative mental fatigue that accompanies extended periods of virtual learning (
      • Bennett A.A.
      Videoconference fatigue? Exploring changes in fatigue after videoconference meetings during COVID-19.
      ). This concept was particularly applicable to our Group 3 (HM) students, as they began clinical rotations after over three months of virtual preclinical education. An additional contributor to lecture satisfaction may have been differences in lecture format: Group 1 (IP) students had exclusively live lectures whereas Group 3 (HM) students had a combination of live and prerecorded lectures, the latter of which has a reduced capacity for meaningful interaction with educators (Table A1).
      Table A1Lecture Concepts by Organ System Under the HM Model
      AbdominalBreastCardiothoracicGeneral Concepts
      Abdominal lymphoma
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Acute and inflammatory colitis

      Acute appendicitis

      Acute cholecystitis

      Acute diverticulitis

      Acute pancreatitis

      Biliary diseases and workup
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Bowel obstruction vs. ileus

      Deep venous thrombosis

      Ectopic pregnancy

      Gallstones and renal stones

      Hepatic metastases
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Pancreatic adenocarcinoma
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Pneumatosis intestinalis

      Pneumoperitoneum

      Sigmoid volvulus



      General overview of breast

        imaging modalities

      Screening, diagnostic, and

        interventional principles
      Acute alveolar processes
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Aortic aneurysm

      Aortic dissection 

      Atelectasis

      Congestive heart failure

      COPD

      Lines and tubes on CXR

      Pleural effusion

      Pneumomediastinum
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Pneumoperitoneum

      Pneumothorax

      Pulmonary hypertension

      Pulmonary foreign bodies

      Pneumonia vs. contusion

      Rib fracture

      Sarcoidosis

      Signs on thoracic imaging
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.
      Basic principles of CT,

        fluoroscopy, MRI, US, and

        XR

      Basic principles of pediatric

        imaging

      Cross-sectional CT anatomy
      InterventionalMusculoskeletalNuclearPediatric
      BRTO
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Gastrointestinal bleeding

      General IR terminology and

        patient workup
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Pulmonary emboli

      TIPS
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.


      Fracture basics

      Spine anatomy on radiographs
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.
      General principles in nuclear

        medicine and PET

      Medical radiation physics

      Overview of radiation oncology
      Exclusively presented in recorded lecture content. COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.
      Annular pancreas

      Bowel atresia and stenosis

      Bowel obstruction workup

      Hirschsprung disease

      Hypertrophic pyloric stenosis

      Intussusception

      Meconium aspiration

      Meconium ileus

      Midgut volvulus

      Necrotizing enterocolitis

      NRDS

      Pediatric lines and tubes

      Pulmonary interstitial edema

      Small left colon syndrome
      low asterisk Exclusively presented in recorded lecture content.COPD, chronic obstructive pulmonary disease; CXR, chest x-ray/radiograph; CT, computed tomography; MRI, magnetic resonance imaging; US, ultrasound; XR, x-ray/radiography; BRTO, balloon-occluded retrograde transvenous obliteration; IR, interventional radiology; TIPS, transjugular intrahepatic portosystemic shunt; PET, positron emission tomography; NRDS, neonatal respiratory distress syndrome.
      Regarding course outcomes, the mean final exam score was restored in Group 3 (HM) after declining in Group 2 (RL). This pattern likely reflects the acute and unfamiliar stress that the COVID-19 pandemic placed on Group 2 (RL) students at its inception, compared to the adjustment period that Group 3 (HM) students were afforded. In addition, Group 2 (RL) students were required to learn four weeks of curricular material in a two-week period and had to take their final exams before any clinical experience was offered; clinical application of studied material would likely have helped solidify the content. Group 3 (HM) students additionally had the novel opportunity to take advantage of medical student teaching assistants, who provided peer support and an interactive examination review session.
      Reports of student mistreatment (abuse or neglect) also declined from two per year in Group 1 (IP) to none in Groups 2 (RL) and 3 (HM). Due to the short time frame and smaller population, this decrease may not be significant, but the trend may suggest that students attributed possible negative interactions with faculty/residents to the stressful learning adaptations (i.e., learning environment differences due to COVID-19 policies) rather than mistreatment.
      This study has several limitations. Although our overall sample size is adequate, the sizes of Groups 2 (RL) and 3 (HM) are small relative to Group 1 (IP), limiting the ability for us to reach statistical significance for some differences such as lecture ratings. Pre-/post-rotation assessments were also not performed; instead, our conclusions rely exclusively on end-of-rotation examinations and post-rotation surveys that may be influenced by the time of year or recall bias. Additionally, several unique variables were introduced in Group 3 (HM) that were not present in Groups 1 (IP) or 2 (RL), including new leadership and a new team of medical student teaching assistants.
      Altogether, these findings suggest that a hybrid radiology educational model offers a unique opportunity to thoughtfully integrate the most successful aspects of traditional in-person and remote “COVID-era” methods. Future hybrid models should emphasize the maintenance of in-person immersive experiences (e.g., procedures, reading room participation) over videoconference readouts, which were important both for our students and those in existing literature (
      • Smith E.
      • Boscak A.
      A virtual emergency: learning lessons from remote medical student education during the COVID-19 pandemic.
      ,
      • Gomez E.
      • Azadi J.
      • Magid D.
      Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
      ,
      • Adams C.C.
      • Shih R.
      • Peterson P.G.
      • Lee M.H.
      • Heltzel D.A.
      • Lattin Jr., G.E.
      The impact of a virtual radiology medical student rotation: maintaining engagement during COVID-19 mitigation.
      ,
      • Durfee S.M.
      • Goldenson R.P.
      • Gill R.R.
      • Rincon S.P.
      • Flower E.
      • Aver L.L.
      Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue.
      ). In-person clinical education offers significant merits, including the opportunity to foster personal relationships with faculty and peers, directly observe the activities of radiologists, participate in interventions, and witness “live” cases unfold in the reading room (
      • Durfee S.M.
      • Goldenson R.P.
      • Gill R.R.
      • Rincon S.P.
      • Flower E.
      • Aver L.L.
      Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue.
      ,
      • Matalon S.A.
      • Guenette J.P.
      • Smith S.E.
      • et al.
      Factors influencing choice of radiology and relationship to resident job satisfaction.
      ). Student-faculty mentorship can additionally influence the consideration of radiology as a career choice (
      • Matalon S.A.
      • Guenette J.P.
      • Smith S.E.
      • et al.
      Factors influencing choice of radiology and relationship to resident job satisfaction.
      ). As such, our current clerkship model exclusively features live participation in the reading room. Similarly, the flexibility and independence afforded by virtual didactics cannot be overlooked; the use of interactive teaching sessions (e.g., mock dictations, participatory or flipped lectures) is convenient while maintaining high-quality learning outcomes (
      • Smith E.
      • Boscak A.
      A virtual emergency: learning lessons from remote medical student education during the COVID-19 pandemic.
      ,
      • Gomez E.
      • Azadi J.
      • Magid D.
      Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
      ). However, the balanced integration of the two formats is critical to prevent strained clinical environments or “Zoom fatigue.”

      CONCLUSION

      Our findings suggest that a hybrid radiology clerkship model can preserve medical student performance while improving student satisfaction relative to purely in-person or remote learning models. Hybrid educational models offer a unique opportunity to leverage the best practices of both in-person and remote learning methods.

      Acknowledgments

      None.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Authors’ contributions

      Alexis Musick: Conceptualization, Writing – Original Draft, Visualization. Deeksha Malhotra: Writing – Original Draft, Visualization. Robert French: Conceptualization, Methodology, Formal analysis, Investigation, Writing – Review & Editing, Supervision. Caroline Carrico: Investigation, Writing – Review & Editing. Jonathan Martin: Conceptualization, Methodology, Formal analysis, Investigation, Writing – Review & Editing, Supervision.

      Citations

      1. Association of American Medical Colleges. Clerkship requirements by discipline. LCME Annual Medical School Questionnaire Part II, 2011-2012 through 2019-2020. Accessed January 15, 2022. Available at:https://www.aamc.org/data-reports/curriculum-reports/interactive-data/clerkship-requirements-discipline.

        • Kondo K.L.
        • Swerdlow M.
        Medical student radiology curriculum: what skills do residency program directors believe are essential for medical students to attain?.
        Acad. Radiol. 2013; 20: 263-271
        • Lee H.
        • Kim D.H.
        • Hong P.P.
        Radiology clerkship requirements in Canada and the United States: current state and impact on residency application.
        J. Am. Coll. Radiol. 2020; 17: 515-522
        • Poot J.D.
        • Hartman M.S.
        • Daffner R.H.
        Understanding the US medical school requirements and medical students’ attitudes about radiology rotations.
        Acad. Radiol. 2012; 19 (Epub 2011 Dec 15. PMID: 22177282): 369-373https://doi.org/10.1016/j.acra.2011.11.005
        • du Cret R.P.
        • Weinberg E.
        • Sellers T.
        • et al.
        Role of radiology in medical education: perspective of nonradiologists.
        Acad. Radiol. 1994; 1: 70-74
        • Straus C.M.
        • Webb E.M.
        • Kondo K.L.
        • et al.
        Medical student radiology education: summary and recommendations from a national survey of medical school and radiology department leadership.
        J Am Coll Radiol. 2014; 11: 606-610
        • Barlev D.M.
        • Lautin E.M.
        • Amis E.S.
        • et al.
        A survey of radiology clerkships at teaching hospitals in the United States.
        Invest Radiol. 1994; 29: 105-108
        • Slanetz P.J.
        • Naeger D.M.
        • Avery L.L.
        • et al.
        Mixed Practice in Radiology Education—Has the Time Come?.
        J Am Coll Radiol. 2020; 17: 976-978
        • Prescott J.E.
        Important guidance for medical students on clinical rotations during the coronavirus (COVID-19) outbreak.
        AAMC. 2020; (Accessed January 15, 2022. Available at:)
        • Smith E.
        • Boscak A.
        A virtual emergency: learning lessons from remote medical student education during the COVID-19 pandemic.
        Emerg Radiol. 2021; 28: 445-452
        • Gomez E.
        • Azadi J.
        • Magid D.
        Innovation born in isolation: rapid transformation of an in-person medical student radiology elective to a remote learning experience during the COVID-19 pandemic.
        Acad Radiol. 2020; 27: 1285-1290
        • DePietro D.M.
        • Santucci S.E.
        • Harrison N.E.
        • et al.
        Medical student education during the COVID-19 pandemic: initial experiences implementing a virtual interventional radiology elective course.
        Acad Radiol. 2021; 28: 128-135
        • Belfi L.M.
        • Dean K.E.
        • Bartolotta R.J.
        • et al.
        Medical student education in the time of COVID-19: a virtual solution to the introductory radiology elective.
        Clin Imaging. 2021; 75: 67-74
        • Dodelzon K.
        • Belfi L.
        • Schweitzer A.D.
        • Lamparello N.
        • Legasto A.C.
        The design and preliminary experience with a virtual diversity visiting student acting internship in radiology for underrepresented minority medical students.
        Acad Radiol. 2021; S1076-6332 (00092–1)https://doi.org/10.1016/j.acra.2021.02.021
        • Adams C.C.
        • Shih R.
        • Peterson P.G.
        • Lee M.H.
        • Heltzel D.A.
        • Lattin Jr., G.E.
        The impact of a virtual radiology medical student rotation: maintaining engagement during COVID-19 mitigation.
        Mil Med. 2020; usaa293https://doi.org/10.1093/milmed/usaa293
        • Durfee S.M.
        • Goldenson R.P.
        • Gill R.R.
        • Rincon S.P.
        • Flower E.
        • Aver L.L.
        Medical student education roadblock due to COVID-19: virtual radiology core clerkship to the rescue.
        Acad Radiol. 2020; 27: 1461-1466
        • Alamer A.
        • Alharbi F.
        Synchronous distance teaching of radiology clerkship promotes medical students’ learning and engagement.
        Insights Imaging. 2021; 12: 41
        • Goldenson R.P.
        • Avery L.L.
        • Gill R.R.
        • et al.
        The virtual homeroom: utility and benefits of small group online learning in the COVID-19 era.
        Curr Probl Diagn Radiol. 2021; S0363-0188 (00110–9)https://doi.org/10.1067/j.cpradiol.2021.06.012
        • Creagh S.
        • Pigg N.
        • Gordillo C.
        • et al.
        Virtual medical student radiology clerkships during the COVID-19 pandemic: distancing is not a barrier.
        Clin Imaging. 2021; 80: 420-423
        • Smith E.B.
        • Boscak A.
        • Friedman E.M.
        • et al.
        Radiology medical student education 2020: surveys of the alliance of medical student educators in radiology and medical students.
        Acad Radiol. 2022; 29: 298-311
        • Osguthorpe R.T.
        • Graham C.R.
        Blended learning environments: definitions and directions.
        Q Rev Distance Educ. 2003; 4: 227-233
        • Ellaway R.
        • Masters K.
        AMEE Guide 32: e-Learning in medical education Part 1: learning, teaching and assessment.
        Med Teach. 2008; 30: 455-473
        • Bennett A.A.
        Videoconference fatigue? Exploring changes in fatigue after videoconference meetings during COVID-19.
        J Appl Psychol. 2021; 106: 330
        • Matalon S.A.
        • Guenette J.P.
        • Smith S.E.
        • et al.
        Factors influencing choice of radiology and relationship to resident job satisfaction.
        Curr Probl Diagn Radiol. 2019; 48: 333-341