Artificial Intelligence Literacy: Developing a Multi-institutional Infrastructure for AI Education

Published:October 30, 2022DOI:

      Rationale and Objectives

      To evaluate the effectiveness of an artificial intelligence (AI) in radiology literacy course on participants from nine radiology residency programs in the Southeast and Mid-Atlantic United States.

      Materials and Methods

      A week-long AI in radiology course was developed and included participants from nine radiology residency programs in the Southeast and Mid-Atlantic United States. Ten 30 minutes lectures utilizing a remote learning format covered basic AI terms and methods, clinical applications of AI in radiology by four different subspecialties, and special topics lectures on the economics of AI, ethics of AI, algorithm bias, and medicolegal implications of AI in medicine. A proctored hands-on clinical AI session allowed participants to directly use an FDA cleared AI-assisted viewer and reporting system for advanced cancer. Pre- and post-course electronic surveys were distributed to assess participants’ knowledge of AI terminology and applications and interest in AI education.


      There were an average of 75 participants each day of the course (range: 50–120). Nearly all participants reported a lack of sufficient exposure to AI in their radiology training (96.7%, 90/93). Mean participant score on the pre-course AI knowledge evaluation was 8.3/15, with a statistically significant increase to 10.1/15 on the post-course evaluation (p= 0.04). A majority of participants reported an interest in continued AI in radiology education in the future (78.6%, 22/28).


      A multi-institutional AI in radiology literacy course successfully improved AI education of participants, with the majority of participants reporting a continued interest in AI in radiology education in the future.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • West E
        • Mutasa S
        • Zhu Z
        • et al.
        Global trend in artificial intelligence-based publications in radiology from 2000 to 2018.
        AJR Am J Roentgenol. 2019; 213: 1204-1206
        • Chartrand G
        • Cheng PM
        • Vorontsov E
        • et al.
        Deep learning: a primer for radiologists.
        Radiographics. 2017; 37: 2113-2131
        • Choy G
        • Khalilzadeh O
        • Michalski M
        • et al.
        Current applications and future impact of machine learning in radiology.
        Radiology. 2018; 288: 318-328
      1. Dreyer, KJ, Wald C, Allen Jr. B, et al. AI Central. American College of Radiology Data Science Institute. Available at: Accessed December 1, 2021.

        • Recht MP
        • Dewey M
        • Dreyer K
        • et al.
        Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations.
        Eur Radiol. 2020; 30: 3576-3584
        • Palmer K.
        Artificial intelligence is making a pitch to transform radiology. will it pay off for hospitals?.
        Stat Health Tech. 2021; (Published online: 2 Dec)
        • Nguyen GK
        • Shetty AS.
        Artificial intelligence and machine learning: opportunities for radiologists in training.
        J Am Coll Radiol. 2018; 15: 1320-1321
        • Langlotz CP
        • Allen B
        • Erickson BJ
        • et al.
        A roadmap for foundational research on artificial intelligence in medical imaging: from the 2018 NIH/RSNA/ACR/The Academy workshop.
        Radiology. 2019; 291: 781-791
        • Wood MJ
        • Teneholtz NA
        • Geis JR
        • et al.
        The need for a machine learning curriculum for radiologists.
        J Am Coll Radiol. 2019; 16: 740-742
        • Soffer S
        • Ben-Cohen A
        • Shimon O
        • et al.
        Convolutional neural networks for radiologic images: a radiologist's guide.
        Radiology. 2019; 290: 590-606
        • Slanetz PJ
        • Daye D
        • Chen P
        • et al.
        Artificial intelligence and machine learning in radiology education is ready for prime time.
        J Am Coll Radiol. 2020; 17: 1705-1707
        • Tejani AS
        • Fielding JR
        • Peshock RM.
        What should radiology residency and fellowship training in artificial intelligence include? a trainee's perspective – radiology in training.
        Radiology. 2021; (Published online: 9 Mar)
        • Gowda J
        • Jordan SG
        • Awan OA.
        Artificial intelligence in radiology education: a longitudinal approach.
        Acad Radiol. 2022; 29: 788-790
        • Hedderich DM
        • Keicher M
        • Wiestler
        • et al.
        AI for doctors – a course to educate medical professionals in artificial intelligence for medical imaging.
        Healthcare. 2021; 9: 1278
        • Huisman M
        • Ranschaert E
        • Parker W
        • et al.
        An international survey on AI in radiology in 1041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude.
        Eur Radiol. 2021; 31: 7058-7066
        • Jungmann F
        • Jorg T
        • Hahn F
        • et al.
        Attitudes toward artificial intelligence among radiologists, IT specialists, and industry.
        Acad Radiol. 2021; 28: 834-840
        • Stempniak M.
        Less than half of radiology departments surveyed offer AI-related education initiatives.
        Radiol Business. 2022; (Published online: January 12)
        • Wiggins WF
        • Caton MT
        • Magudia K
        • et al.
        Preparing radiologists to lead in the era of artificial intelligence: designing and implementing a focused data science pathway for senior radiology residents.
        Radiol: Artificial Intelligence. 2020; (Published online: 4 Nov)
        • Lindqwister AL
        • Hassanpour S
        • Lewis PJ
        • et al.
        AI-RADS: an artificial intelligence curriculum for residents.
        Acad Radiol. 2021; 28: 1810-1816
        • Wiggins WF
        • Caton Jr., MT.
        • Magudia K
        • Rosenthal MH
        • Andriole KP
        A Conference-Friendly, Hands-on Introduction to Deep Learning for Radiology Trainees.
        Journal of Digital Imaging. 2021; 34: 1026-1033
        • Lanier MH
        • Wheeler CA
        • Ballard DH.
        A new normal in radiology resident education: lessons learned from the COVID -19 pandemic.
        RadioGraphics. 2021; 41 (Published online: 3 May 2021)
        • Lo L
        • Awan OA.
        To engage or not to engage: a new era for medical student education in radiology.
        RadioGraphics. 2020; 40 (Published online: 2 Nov 2020)
      2. RSNA AI Certificate Program. Radiological Society of North America. Available at: Accessed 11 April 2022.

        • Jarmko JL
        • Azar M
        • Bromwich R
        • et al.
        Canadian association of radiologists white paper on ethical and legal issues related to artificial intelligence in radiology.
        Can Assoc Radiol J. 2019; 70: 107-118
        • Safdar NM
        • Banja JD
        • Meltzer CC.
        Ethical considerations in artificial intelligence.
        Eur J Radiol. 2020; 122108768
        • Gupta A
        • Heath V.
        AI ethics groups are repeating one of society's classic mistakes.
        MIT Technology Rev. 2020; (Published September 14. Available at)