Does Objective Scoring of Applications for Radiology Residency Affect Diversity?

Published:December 02, 2021DOI:

      Rationale and Objectives

      It is important to recognize if reliance on certain factors in applications affects selection of trainees from under-represented groups. Our purpose is to determine if objective scoring of radiology residency applications based on quantifiable data regarding academic performance, United States Medical Licensing Examination (USMLE) scores and research productivity affects selection of female and under-represented minority (URM) candidates.

      Materials and Methods

      We reviewed 502 applications from three successive match cycles from United States allopathic medical students. Each application was scored for academic performance, USMLE results and research productivity determining an overall score. The scores of males were compared to females and URM were compared to non-URM candidates. USMLE cutoff scores were evaluated for disparate effects.


      There were 348 male, 154 female, 73 URM and 429 non-URM candidates. For male versus female applicants, there was no significant difference in mean academic performance, USMLE Step 2 Clinical Knowledge and research productivity scores. Males had higher mean USMLE Step 1 (p = 0.005) and overall candidate scores (p = 0.02). Between URM and non-URM candidates there was no significant difference in academic performance. Non-URM applicants had higher mean USMLE Step 1 (p = 0.008), USMLE Step 2 (p = 0.002), research productivity (p = 0.001) and overall scores (p = 0.02). Use of USMLE cutoff scores demonstrated disparate effects on female and URM candidates.


      Objective scoring of applications and use of USMLE cutoff scores can disadvantage candidates from underrepresented groups. Screening filters can affect the diversity of candidate pools for radiology residency.

      Key Words


      U.S (United States), URM (Under-represented minority), AOA (Alpha Omega Alpha Honor Medical Society), USMLE (United States Medical Licensing Examination), ERAS (Electronic Residency Application Service), NRMP (National Residency Matching Program), CK (Clinical Knowledge), MSPE (Medical student performance evaluation), AAMC (American Association of Medical Colleges)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Association of American Medical Colleges. 2020 Fall Applicant, Matriculant, and Enrollment Data Tables. Available at: Accessed September 27, 2021.

        • Brotherton SE
        • Etzel SI.
        Graduate medical education, 2019-2020.
        JAMA. 2020; 324: 1230-1250
        • Francone NO
        • Simon MA
        • Ortega P.
        Exploring race and ethnicity representational inequities in Illinois medical schools.
        Health Equity. 2021; 5: 526-533
        • Lightfoote JB
        • Deville C
        • Ma LD
        • et al.
        Diversity, inclusion and representation: it is time to act.
        J Am Coll Radiol. 2016; 13: 1421-1425
        • Spector AR
        • Railey KM.
        Reducing reliance on test scores reduces racial bias in neurology residency recruitment.
        J Natl Med Assoc. 2019; 111: 471-474
        • Williams M
        • Kim EJ
        • Pappas K
        • et al.
        The impact of United States Licensing Exam (USMLE) step 1 cutoff scores on recruitment of underrepresented minorities in medicine: a retrospective cross-sectional study.
        Health Sci Rep. 2020; 3: e2161
        • Wijesekera TP
        • Kim M
        • Moore EZ
        • et al.
        All other things being equal: exploring racial and gender disparities in medical school honor society induction.
        Acad Med. 2019; 94: 562-569
        • Boatright D
        • Ross D
        • O'Connor P
        • et al.
        Racial disparities in medical student membership in the Alpha Omega Alpha Honor Society.
        JAMA Intern Med. 2017; 177: 659-665
        • Zhang N
        • Blissett S
        • Anderson D
        • et al.
        Race and gender bias in internal medicine program director letters of recommendation.
        J Grad Med Educ. 2021; 13: 335-344
        • Khan S
        • Kirubarajan A
        • Shamsheri T
        • et al.
        Gender bias in reference letters for residency and academic medicine: a systematic review.
        Postgrad Med J. 2021; (Jun 2:postgradmedj-2021-140045Online ahead of print)
        • Grimm LJ
        • Redmond RA
        • Campbell JC
        • et al.
        Gender and racial bias in radiology residency letters of recommendation.
        J Am Coll Radiol. 2020; 17: 64-71
      2. USMLE Score Interpretation Guidelines. Available at: Accessed September 27, 2021.

        • Rubright JD
        • Jodoin M
        • Barone MA.
        Examining demographics, prior academic performance, and United States Medical Licensing Examination scores.
        Acad Med. 2019; 94: 364-370
        • Gauer JL
        • Jackson JB.
        Relationships of demographic variables to USMLE physician licensing exam scores: a statistical analysis on five years of medical student data.
        Adv Med Educ Pract. 2018; 9: 39-44
        • National Resident Matching Program
        Results and Data: 2018 Main Residency Match.
        National Resident Matching Program, Washington, DC2018 (Available at) (Accessed September 27, 2021)
        • National Resident Matching Program
        Results and Data: 2020 Main Residency Match.
        National Resident Matching Program, Washington, DC2020 (Available at) (Accessed September 27, 2021)
        • Zhang RV
        • Awan OA
        • Resnik CS
        • et al.
        Potential impact of a pass or fail United States Medical Licensing Exam Step 1 scoring system on radiology residency applications.
        Acad Radiol. 2020; (Nov 5;S1076-6332(20)30603-6Online ahead of print)
      3. AAMC Table C-2: Residency Applicants from U.S. MD-Granting Medical Schools to ACGME-Accredited Programs by Specialty and Sex, 2020-2021. Available at: Accessed September 27, 2021.

        • Hewett LH
        • Lesis M
        • Collins H
        • et al.
        Gender bias in diagnostic radiology resident selection, does it exist?.
        Acad Radiol. 2016; 23: 101-107
        • Zener R
        • Lee SY
        • Visscher KL
        • et al.
        Women in radiology: exploring the gender disparity.
        J Am Coll Radiol. 2016; 13: 344-350
      4. Association of American Medical Colleges. Table B-5.1: Total Enrollment by U.S. Medical School and Race/Ethnicity (Alone), 2020-2021. Available at: Accessed September 27, 2021.

      5. Association of American Medical Colleges. Aggregated specialty specific data on the number of ERAS applicants and the average number of applications by applicant and by program. Available at: Accessed September 27, 2021.

        • Spottswood SE
        • Spalluto LB
        • Washington ER
        • et al.
        Design, implementation, and evaluation of a diversity program for radiology.
        J Am Coll Radiol. 2019; 16: 983-991
        • DeBenedectis CM
        • Heitkamp DE
        • England E
        • et al.
        A program director's guide to cultivating diversity and inclusion in radiology residency recruitment.
        Acad Radiol. 2020; 27: 864-867
        • Gaetke-Udager K
        • Magid D
        • Smith M
        • et al.
        Recruiting future radiologists: How can we do better?.
        Acad Radiol. 2020; 27: 1311-1315
        • National Resident Matching Program
        Data Release and Research Committee: Results of the 2021 NRMP Program Director Survey.
        National Resident Matching Program, Washington, DC2021 (Available at) (Accessed September 27, 2021)
        • Association of American Medical Colleges
        Recommendations for revising the medical student performance evaluation (MSPE).
        Association of American Medical Colleges, Washington, DC2017 (Available at) (Accessed September 27, 2021)
        • Boysen-Osborn M
        • Yanuck J
        • Mattson J
        • et al.
        Who to interview? Low adherence by U.S. Medical schools to medical student performance evaluation format makes resident selection difficult.
        West J Emerg Med. 2017; 18: 50-55
        • Brenner JM
        • Bird JB
        • Brenner J
        • et al.
        Current state of the medical student performance evaluation: a tool for reflection for residency programs.
        J Grad Med Educ. 2021; 13: 576-580
        • Ramakrishnan D
        • Van Le-Bucklin K
        • Saba T
        • et al.
        What does honors mean? National analysis of medical school clinical clerkship grading.
        J Surg Educ. 2021; (Sep 12:S1931-7204(21)00244-0Online ahead of print)
        • Vokes J
        • Greenstein A
        • Carmody E
        • et al.
        The current status of medical school clerkship grades in residency applicants.
        J Grad Med Educ. 2020; 12: 145-149
        • Westerman ME
        • Boe C
        • Bole R
        • et al.
        Evaluation of Medical school grading variability in the United States: Are all honors the same?.
        Acad Med. 2019; 94: 1939-1945