Potential Impact of a Pass or Fail United States Medical Licensing Exam Step 1 Scoring System on Radiology Residency Applications

Published:November 07, 2020DOI:https://doi.org/10.1016/j.acra.2020.10.016

      Rationale and Objectives

      To assess the anticipated impact of the change in United States Medical Licensing Examination Step 1 scoring from numerical to pass or fail on the future selection of radiology residents.

      Materials and Methods

      An anonymous electronic 14-item survey was distributed to 308 members of the Association of Program Directors in Radiology and included questions regarding the anticipated importance of various application metrics when Step 1 becomes pass or fail. Secondary analyses compared responses based on the current use of a Step 1 scoring screen.

      Results

      Eighty eight respondents (28.6% [88/308]) completed the survey. Most (64% [56/88]) noted that the United States Medical Licensing Examination Step 2 Clinical Knowledge (CK) score will likely be one of the top three most important factors in assessing applicants, followed by class ranking or quartile (51% [45/88]) and the Medical Student Performance Evaluation/Dean's Letter (42% [37/88]). Over 90% (82/88) of respondents anticipate potentially or definitively requiring Step 2 CK scores before application review, and 50% (44/88) of respondents anticipate extending interview invites at a later date to receive Step 2 CK scores. These results did not significantly differ between programs who currently use a Step 1 scoring screen and those who do not.

      Conclusion

      As Step 1 transitions from a numerical score to pass or fail, radiology residency program directors will likely rely on Step 2 CK scores as an objective and standardized metric to screen applicants. Further investigation is needed to identify other objective metrics to evaluate applicants before Step 1 becomes pass or fail.

      Key Words

      To read this article in full you will need to make a payment

      Subscribe:

      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Huebner C
        • Adnan M
        • Kraeutler MJ
        • et al.
        Use of the United States Medical Licensing Examination Step-1 Score as a Screening Tool for Orthopaedic Surgery Away Rotations.
        J Bone Jt Surg. 2019; 101: e106https://doi.org/10.2106/JBJS.19.00265
        • Release National Resident Matching Program Data
        • Committee Research
        Results of the 2018 NRMP Program Director Survey.
        Published online. June 2018;
        • Chen DR
        • Priest KC
        • Batten JN
        • et al.
        Student Perspectives on the “Step 1 Climate” in Preclinical Medical Education.
        Acad Med. 2019; 94: 302-304https://doi.org/10.1097/ACM.0000000000002565
        • Green M
        • Angoff N
        • Encandela J
        Test anxiety and United States Medical Licensing Examination scores.
        Clin Teach. 2016; 13: 142-146https://doi.org/10.1111/tct.12386
        • Rubright JD
        • Jodoin M
        • Barone MA
        Examining demographics, prior academic performance, and United States Medical Licensing Examination Scores.
        Acad Med. 2019; 94: 364-370https://doi.org/10.1097/ACM.0000000000002366
        • Teherani A
        • Hauer KE
        • Fernandez A
        • et al.
        How small differences in assessed clinical performance amplify to large differences in grades and awards: a cascade with serious consequences for students underrepresented in medicine.
        Acad Med. 2018; 93: 1286-1292https://doi.org/10.1097/ACM.0000000000002323
      1. United States Medical Licensing Examination. InCUS - Invitational Conference on USMLE Scoring: Change to pass/fail score reporting for Step 1. https://www.usmle.org/incus/

      2. Maxfield CM, Grimm LJ. The Value of Numerical USMLE Step 1 Scores in Radiology Resident Selection. Acad Radiol. Published online August 2019:S1076633219303861. doi:10.1016/j.acra.2019.08.007.

        • Otero HJ
        • Erturk SM
        • Ondategui-Parra S
        • et al.
        Key criteria for selection of radiology residents.
        Acad Radiol. 2006; 13: 1155-1164https://doi.org/10.1016/j.acra.2006.06.012
        • Calisi N
        • Gondi KT
        • Asmar J
        • et al.
        Predictors of Success on the ABR core examination.
        J Am Coll Radiol. 2019; 16: 1193-1200https://doi.org/10.1016/j.jacr.2019.03.007
        • Agarwal V
        • Bump GM
        • Heller MT
        • et al.
        Do residency selection factors predict radiology resident performance.
        Acad Radiol. 2018; 25: 397-402https://doi.org/10.1016/j.acra.2017.09.020
        • United States Medical Licensing Examination
        USMLE score interpretation guidelines.
        Published online. January 2020;
        • Case S
        • Swanson D
        • Ripkey D
        • et al.
        Performance of the Class of 1994 in the New Era of USMLE.
        Acad Med. 1996; 71: S91-S93
        • Gruppuso PA
        • Adashi EY
        Residency placement fever: is it time for a reevaluation?.
        Acad Med. 2017; 92: 923-926https://doi.org/10.1097/ACM.0000000000001468
      3. Angus SV, Williams CM, Stewart EA, et al. Internal medicine residency program directors’ screening practices and perceptions about recruitment challenges: Acad Med. Published online November 2019:1. doi:10.1097/ACM.0000000000003086.

        • Schrock JB
        • Kraeutler MJ
        • Dayton MR
        • et al.
        A cross-sectional Analysis of Minimum USMLE Step 1 and 2 Criteria Used by Orthopaedic Surgery Residency Programs in Screening Residency Applications.
        J Am Acad Orthop Surg. 2017; 25: 464-468https://doi.org/10.5435/JAAOS-D-16-00725
        • Rozenshtein A
        • Mullins ME
        • Marx MV
        The USMLE Step 1 Pass/Fail Reporting Proposal: The APDR Position.
        Acad Radiol. 2019; 26: 1400-1402https://doi.org/10.1016/j.acra.2019.06.004
        • Salari S
        • Deng F
        A Stepping Stone Toward Necessary Change: How the New USMLE Step 1 Scoring System Could Affect the Residency Application Process.
        Acad Med. 2020; (Publish Ahead of Print)https://doi.org/10.1097/ACM.0000000000003501
        • Sharma A
        • Schauer DP
        • Kelleher M
        • et al.
        USMLE Step 2 CK: Best Predictor of Multimodal Performance in an Internal Medicine Residency.
        J Grad Med Educ. 2019; 11: 412-419https://doi.org/10.4300/JGME-D-19-00099.1
      4. Association of American Medical Colleges. Time in the curriculum in which medical schools require students to take United States Medical Licensing Examinations (USMLE). Published 2019. https://www.aamc.org/initiatives/cir/406430/10c.html

        • Keim SM
        • Rein JA
        • Chisholm C
        • et al.
        A standardized letter of recommendation for residency application.
        Acad Emerg Med. 1999; 6: 1141-1146https://doi.org/10.1111/j.1553-2712.1999.tb00117.x
      5. Deitte LA, Catanzano TM, Ho CP, et al. The pass/fail United States Medical Licensing Examination Step 1: impact on radiology residency selection. J Am Coll Radiol. Published online July 2020:S1546144020306633. doi:10.1016/j.jacr.2020.06.014.

        • Green M
        • Jones P
        • Thomas JX
        Selection criteria for residency: results of a National Program Directors Survey.
        Acad Med. 2009; 84: 362-367https://doi.org/10.1097/ACM.0b013e3181970c6b
        • Makhoul AT
        • Pontell ME
        • Ganesh Kumar N
        • et BC
        • et al.
        Objective Measures Needed — Program Directors’ Perspectives on a Pass/Fail USMLE Step 1.
        N Engl J Med. 2020; 382: 2389-2392https://doi.org/10.1056/NEJMp2006148