Advertisement

The USMLE Step 1 Pass/Fail Reporting Proposal: Another View

      The Association of Program Directors in Radiology recently issued a statement endorsing continued reporting of results of the United States Medical Licensing Examination (USMLE) as a three-digit score. While this position was approved by the Association of Program Directors in Radiology Board of Directors, it does not reflect the opinions of all radiology program directors. Here, we present an argument in support of reporting USMLE results as pass/fail.
      As a psychometric instrument, the USMLE Step 1 is designed to assess basic science knowledge and intended to inform a binary decision on licensure. Due to a steadily-increasing burden of applications to review, program directors have increasingly relied upon scores for candidate screening. Such use has multiple adverse consequences. Student focus on Step 1 systematically devalues educational content not evaluated on the exam, and the reliance on Step 1 scores almost certainly works against efforts to increase workforce diversity. Moreover, the increasing pressure of “Step 1 Mania” has negative consequences for trainee mental health and wellness.
      Despite the widespread use of Step 1 scores to select applicants, there are little data to correlate scores to meaningful outcomes related to patient care or clinical practice. We find the current situation untenable, and believe a necessary first step toward reform is making Step 1 a pass/fail only examination.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      REFERENCES

        • Rozenshtein A.
        • Mullins M.E.
        • Marx M.V.
        The USMLE step 1 pass/fail reporting proposal: the APDR position.
        Acad Radiol. 2019; 26: 1400-1402
        • National Board of Medical Examiners
        “National Board Examinations, use of scores in residency selection, and staff changes”.
        Natl Board Examiner. 1988; 35: 3
        • National Resident Matching Program
        Results and Data: 2019 Main Residency Match.
        National Resident Matching Program, Washington, DC2019
        • Weissbart S.J.
        • Kim S.J.
        • Feinn R.S.
        • et al.
        Relationship between the number of residency applications and the yearly match rate: time to start thinking about an application limit.
        J Grad Med Educ. 2015; 7: 81-85
      1. ERAS Statistics - Preliminary Data (ERAS 2019). Online at: https://www.aamc.org/services/eras/stats/359278/stats.html. Accessed 22 May 2019.

        • National Resident Matching Program
        Data Release and Research Committee: Results of the 2010 NRMP Program Director Survey.
        National Resident Matching Program, Washington, DC2010 (Online at:)
        • National Resident Matching Program
        Data Release and Research Committee: Results of the 2018 NRMP Program Director Survey.
        National Resident Matching Program, Washington, DC2018 (Online at:)
        • Swanson D.B.
        • Case S.M.
        • Luecht R.M.
        • et al.
        Retention of basic science information by fourth-year medical students.
        Acad Med. 1996; 71: S80-S82
        • Ling Y.
        • Swanson D.B.
        • Holtzman K.
        • et al.
        Retention of basic science information by senior medical students.
        Acad Med. 2008; 83: S82-S85
        • McGaghie W.C.
        • Cohen E.R.
        • Wayne D.B.
        Are united states medical licensing exam step 1 and 2 scores valid measures for postgraduate medical residency selection decisions?.
        Acad Med. 2011; 86: 48-52
        • Boyse T.D.
        • Petterson S.K.
        • Cohan R.H.
        • et al.
        Does medical school performance predict radiology resident performance?.
        Acad Radiol. 2002; 9: 437-445
        • Calisi N.
        • Gondi K.T.
        • Asmar J.
        • et al.
        Predictors of success on the ABR core examination.
        J Am Coll Radiol. 2019; (in press)
        • McCaskill Q.E.
        • Kirk J.J.
        • Barata D.M.
        • et al.
        USMLE step 1 scores as a significant predictor of future board passage in pediatrics.
        Ambul Pediatr. 2007; 7: 192-195
        • Kay C.
        • Jackson J.L.
        • Frank M.
        The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE step 1 examination.
        Acad Med. 2015; 90: 100-104
        • Shellito J.L.
        • Osland J.S.
        • Helmer S.D.
        • et al.
        American board of surgery examinations: can we identify surgery residency applicants and residents who will pass the examinations on the first attempt?.
        Am J Surg. 2010; 199: 216-222
        • Armstrong A.
        • Alvero R.
        • Nielsen P.
        • et al.
        Do U.S. medical licensure examination step 1 scores correlate with council on resident education in obstetrics and gynecology in-training examination scores and american board of obstetrics and gynecology written examination performance?.
        Mil Med. 2007; 172: 640-643
        • Swanson D.B.
        • Sawhill A.
        • Holtzman K.Z.
        • et al.
        Relationship between performance on part i of the american board of orthopaedic surgery certifying examination and scores on USMLE steps 1 and 2.
        Acad Med. 2009; 84: S21-S24
        • Dillon G.F.
        • Swanson D.B.
        • McClintock J.C.
        • et al.
        The relationship between the american board of anesthesiology part 1 certifying examination and the united states medical licensing examination.
        J Grad Med Educ. 2013; 5: 276-283
        • Prober C.G.
        • Kolars J.C.
        • First L.R.
        • et al.
        A plea to reassess the role of united states medical licensing examination step 1 scores in residency selection.
        Acad Med. 2016; 91: 12-15
        • Chen D.R.
        • Priest K.C.
        • Batten J.N.
        • et al.
        Student perspectives on the “Step 1 Climate” in preclinical medical education.
        Acad Med. 2019; 94: 303-304
        • Burk-Rafel J.
        • Santen S.A.
        • Purkiss J.
        Study behaviors and USMLE step 1 performance: implications of a student self-directed parallel curriculum.
        Acad Med. 2017; 92: S67-S74
        • Chapman C.H.
        • Hwang W.T.
        • Both S.
        • et al.
        Current status of diversity by race, hispanic ethnicity, and sex in diagnostic radiology.
        Radiology. 2014; 270: 232-240
        • Rubright J.D.
        • Jodoin M.
        • Barone M.A.
        Examining demographics, prior academic performance, and united states medical licensing examination scores.
        Acad Med. 2019; 94: 364-370
        • Dyrbye L.N.
        • West C.P.
        • Satele D.
        • et al.
        Burnout among US medical students, residents, and early career physicians relative to the general US population.
        Acad Med. 2014; 89: 445-451
        • Slavin S.
        Reflections on a decade leading a medical student well-being initiative.
        Acad Med. 2019; 94: 771-774
      2. USMLE Score Interpretation Guidelines. Online at: https://www.usmle.org/pdfs/transcripts/USMLE_Step_Examination_Score_Interpretation_Guidelines.pdf. Accessed 23 May 2019.

        • National Resident Matching Program
        Charting Outcomes in the Match: U.S. Allopathic Seniors, 2018.
        National Resident Matching Program, Washington, DC2018 (Online at:)
        • Dyrbye L.N.
        • Massie F.S.
        • Eaker A.
        • et al.
        Relationship between burnout and professional conduct and attitudes among US medical students.
        JAMA. 2010; 304: 1173-1180
        • Schwenk T.L.
        • Davis L.
        • Wimsatt L.A.
        Depression, stigma, and suicidal ideation in medical students.
        JAMA. 2010; 301: 1181-1190
        • Vilwock J.A.
        • Hamill C.S.
        • Sale K.A.
        • et al.
        Beyond the USMLE: the STAR algorithm for initial residency applicant screening and interview selection.
        J Surg Res. 2019; 235: 447-452
        • Gardner A.K.
        • Dunkin B.J.
        Pursuing excellence: the power of selection science to provide meaningful data and enhance efficiency in selecting surgical trainees.
        Ann Surg. 2018; (Epub ahead of print)https://doi.org/10.1097/SLA.0000000000002806
        • Love J.N.
        • Smith J.
        • Weizberg M.
        • et al.
        Council on emergency medicine residency Directors’ standardized letter of recommendation: the program director's perspective.
        Acad Emerg Med. 2014; 21: 680-687
        • Conrad S.C.
        • Addams A.N.
        • Young G.H.
        Holistic review in medical school admissions and selection: a strategic, mission-driven response to shifting societal needs.
        Acad Med. 2016; 91: 1472-1474
        • Kirch D.G.
        Transforming admissions: the gateway to medicine.
        JAMA. 2012; 308: 2250-2251