Original Investigation| Volume 30, ISSUE 5, P798-806, May 2023

Download started.


Patterns of Screening Recall Behavior Among Subspecialty Breast Radiologists

      Rationale and Objectives

      Determine whether there are patterns of lesion recall among breast imaging subspecialists interpreting screening mammography, and if so, whether recall patterns correlate to morphologies of screen-detected cancers.

      Materials and Methods

      This Institutional Review Board-approved, retrospective review included all screening examinations January 3, 2012–October 1, 2018 interpreted by fifteen breast imaging subspecialists at a large academic medical center and two outpatient imaging centers. Natural language processing identified radiologist recalls by lesion type (mass, calcifications, asymmetry, architectural distortion); proportions of callbacks by lesion types were calculated per radiologist. Hierarchical cluster analysis grouped radiologists based on recall patterns. Groups were compared to overall practice and each other by proportions of lesion types recalled, and overall and lesion-specific positive predictive value-1 (PPV1).


      Among 161,859 screening mammograms with 13,086 (8.1%) recalls, Hierarchical cluster analysis grouped 15 radiologists into five groups. There was substantial variation in proportions of lesions recalled: calcifications 13%–18% (Chi-square 45.69, p < 0.00001); mass 16%–44% (Chi-square 498.42, p < 0.00001); asymmetry 13%–47% (Chi-square 660.93, p < 0.00001) architectural distortion 6%–20% (Chi-square 283.81, p < 0.00001). Radiologist groups differed significantly in overall PPV1 (range 5.6%–8.8%; Chi-square 17.065, p = 0.0019). PPV1 by lesion type varied among groups: calcifications 9.2%–15.4% (Chi-square 2.56, p = 0.6339); mass 5.6%–8.5% (Chi-square 1.31, p = 0.8597); asymmetry 3.4%–5.9% (Chi-square 2.225, p = 0.6945); architectural distortion 5.6%–10.8% (Chi-square 5.810, p = 0.2138). Proportions of recalled lesions did not consistently correlate to proportions of screen-detected cancer.


      Breast imaging subspecialists have patterns for screening mammography recalls, suggesting differential weighting of imaging findings for perceived malignant potential. Radiologist recall patterns are not always predictive of screen-detected cancers nor lesion-specific PPV1s.

      Key Words


      CDR (cancer detection rate), DBT (digital breast tomosynthesis), FFDM (full field digital mammogram), HCA (hierarchical cluster analysis), NLP (natural language processing), PPV1 (positive predictive value of recalled screening examinations]), US (United States)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Sickles EA
        • D'Orsi CJ
        • et al.
        ACR BI-RADS follow-up and outcome monitoring 2013.
        in: D'Orsi CJ Sickles EA Mendelson EB ACR BI-RADS Atlas Breast Imaging Reporting and Data System. 5th ed. American College of Radiology, Reston, VA2013
        • Lehman CD
        • Arao RF
        • Sprague BL
        • et al.
        National performance benchmarks for modern screening digital mammography: update from the breast cancer surveillance consortium.
        Radiology. 2017; 283: 49-58
        • D'Orsi CJ
        • Bassett LW
        • Berg WA
        • et al.
        BI-RADS mammography.
        in: D'Orsi CJ Mendelson EB Ikeda DM Breast imaging and reporting data system: ACR BI-RADS-breast imaging atlas. 4th ed. American College of Radiology, Reston, VA2003
        • Elmore JG
        • Miglioretti DL
        • Reisch LM
        • et al.
        Screening mammograms by community radiologists: variability in false-positive rates.
        J Natl Cancer Inst. 2002; 94: 1373-1380
        • Barlow WE
        • Chi C
        • Carney PA
        • et al.
        Accuracy of screening mammography interpretation by characteristics of radiologists.
        J Natl Cancer Inst. 2004; 96: 1840-1850
        • Smith-Bindman R
        • Chu P
        • Miglioretti DL
        • et al.
        Physician predictors of mammographic accuracy.
        J Natl Cancer Inst. 2005; 97: 358-367
        • Tan A
        • Freeman DH
        • Goodwin JS
        • et al.
        Variation in false-positive rates of mammography reading among 1067 radiologists: a population-based assessment.
        Breast Cancer Res Treat. 2006; 100: 309-318
        • Buist DSM
        • Anderson ML
        • Haneuse SJPA
        • et al.
        Influence of annual interpretive volume on screening mammography performance in the United States.
        Radiology. 2011; 259: 72-83
        • Nelson HD
        • O'Meara ES
        • Kerlikowske K
        • et al.
        Factors associated with rates of false-positive and false-negative results from digital mammography screening: an analysis of registry data.
        Ann Intern Med. 2016; 164: 226-235
        • Conant EF
        • Zuckerman SP
        • McDonald ES
        • et al.
        Five consecutive years of screening with digital breast tomosynthesis: outcomes by screening year and round.
        Radiology. 2020; 295: 285-293
        • Lourenco AP
        • Barry-Brooks M
        • Baird G
        • et al.
        Changes in recall type and patient treatment following implementation of screening digital breast tomosynthesis.
        Radiology. 2015; 274: 337-342
        • Durand MA
        • Haas BM
        • Yao X
        • et al.
        Early clinical experience with digital breast tomosynthesis for screening mammography.
        Radiology. 2015; 274: 85-92
        • Giess CS
        • Pourjabbar S
        • Ip IK
        • et al.
        Comparing diagnostic performance of digital breast tomosynthesis and full-field digital mammography in a hybrid imaging environment.
        AJR Am J Roentgenol. 2017; 209: 929-934
        • Giess CS
        • Wang A
        • Ip IK
        • et al.
        Patient, radiologist, and examination characteristics affecting screening mammography recall rates in a large academic practice.
        J Am Coll Radiol. 2019; 16: 411-418
        • Mullen LA
        • Panigrahi B
        • Hollada J
        • et al.
        Strategies for decreasing screening mammography recall rates while maintaining performance metrics.
        Acad Radiol. 2017; 24: 1556-1560
        • Sickles EA
        • D'Orsi CJ
        • Bassett LW
        • et al.
        ACR BI-RADS Mammography.
        ACR BI-RADS atlas, breast imaging reporting and data system. American College of Radiology, Reston, VA2013
        • Lacson R
        • Harris K
        • Brawarsky P
        • et al.
        Evaluation of an automated information extraction tool for imaging data elements to populate a breast cancer screening registry.
        J Digit Imaging. 2015; 28: 567-575
        • Zhang Z
        • Murtagh F
        • Van Poucke S
        • et al.
        Hierarchical cluster analysis in clinical research with heterogenous study population: highlighting its visualization with R.
        Ann Transl Med. 2017; 5: 75
        • Venkatesan A
        • Chu P
        • Kerlikowske K
        • et al.
        Positive predictive value of specific mammographic findings according to reader and patient variables.
        Radiology. 2009; 250: 648-657
        • Welch HG
        • Passow HJ.
        Quantifying the benefits and harms of screening mammography.
        JAMA Intern Med. 2014; 174: 448-454
        • Siu AL.
        Screening for breast cancer: U.S. Preventive Services Task Force recommendations statement.
        Ann Intern Med. 2016; 164: 279-296
        • Hardesty LA
        • Klym AH
        • Shindel BE
        • et al.
        Is maximum positive predictive value a good indicator of an optimal screening mammography practice?.
        AJR Am J Roentgenol. 2005; 184: 1505-1507
        • Miglioretti DL
        • Ichikawa L
        • Smith RA
        • et al.
        Criteria for identifying radiologists with acceptable screening mammography interpretive performance based on multiple performance measures.
        AJR Am J Roentgenol. 2015; 204: W486-W491
        • Busby JP
        • Courtier JL
        • Glastonbury CM.
        Bias in radiology: the how and why of misses and misinterpretations.
        RadioGraphics. 2018; 38: 236-247
        • Alpert HR
        • Hillman BJ.
        Quality and variability in diagnostic radiology.
        J Am Coll Radiol. 2004; 1: 127-132
        • McKinney SM
        • Sieniek M
        • Godbole V
        • et al.
        International evaluation of an AI system for breast cancer screening.
        Nature. 2020; 577: 89-94
        • Cochon LR
        • Kapoor N
        • Carrodeguas E
        • et al.
        Variation in follow-up imaging recommendations in radiology reports: patient, modality, and radiologist predictors.
        Radiology. 2019; 291: 700-707