Deep Learning-based Automatic Diagnosis of Breast Cancer on MRI Using Mask R-CNN for Detection Followed by ResNet50 for Classification

Published:January 10, 2023DOI:

      Rationale and Objectives

      Diagnosis of breast cancer on MRI requires, first, the identification of suspicious lesions; second, the characterization to give a diagnostic impression. We implemented Mask Reginal-Convolutional Neural Network (R-CNN) to detect abnormal lesions, followed by ResNet50 to estimate the malignancy probability.

      Materials and Methods

      Two datasets were used. The first set had 176 cases, 103 cancer, and 73 benign. The second set had 84 cases, 53 cancer, and 31 benign. For detection, the pre-contrast image and the subtraction images of left and right breasts were used as inputs, so the symmetry could be considered. The detected suspicious area was characterized by ResNet50, using three DCE parametric maps as inputs. The results obtained using slice-based analyses were combined to give a lesion-based diagnosis.


      In the first dataset, 101 of 103 cancers were detected by Mask R-CNN as suspicious, and 99 of 101 were correctly classified by ResNet50 as cancer, with a sensitivity of 99/103 = 96%. 48 of 73 benign lesions and 131 normal areas were identified as suspicious. Following classification by ResNet50, only 16 benign and 16 normal areas remained as malignant. The second dataset was used for independent testing. The sensitivity was 43/53 = 81%. Of the total of 121 identified non-cancerous lesions, only 6 of 31 benign lesions and 22 normal tissues were classified as malignant.


      ResNet50 could eliminate approximately 80% of false positives detected by Mask R-CNN. Combining Mask R-CNN and ResNet50 has the potential to develop a fully-automatic computer-aided diagnostic system for breast cancer on MRI.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Siegel RL
        • Miller KD
        • Fuchs HE
        • Jemal A.
        Cancer statistics, 2021.
        CA Cancer J Clin. 2021; 71: 7-33
        • Oeffinger KC
        • Fontham ET
        • Etzioni R
        • et al.
        Breast cancer screening for women at average risk: 2015 Guideline Update From the American Cancer Society.
        JAMA. 2015; 314: 1599-1614
        • Saslow D
        • Boetes C
        • Burke W
        • et al.
        American Cancer Society guidelines for breast screening with MRI as an adjunct to mammography.
        CA Cancer J Clin. 2007; 57: 75-89
        • Raikhlin A
        • Curpen B
        • Warner E
        • et al.
        Breast MRI as an adjunct to mammography for breast cancer screening in high-risk patients: retrospective review.
        AJR Am J Roentgenol. 2015; 204: 889-897
        • Marino MA
        • Helbich T
        • Baltzer P
        • Pinker-Domenig K.
        Multiparametric MRI of the breast: A review.
        J Magn Reson Imaging. 2018; 47: 301-315
        • Mann RM
        • Kuhl CK
        • Moy L.
        Contrast-enhanced MRI for breast cancer screening.
        J Magn Reson Imaging. 2019; 50: 377-390
        • Gilhuijs KG
        • Giger ML
        • Bick U.
        Computerized analysis of breast lesions in three dimensions using dynamic magnetic-resonance imaging.
        Med Phys. 1998; 25: 1647-1654
        • Nie K
        • Chen JH
        • Yu HJ
        • et al.
        Quantitative analysis of lesion morphology and texture features for diagnostic prediction in breast MRI.
        Acad Radiol. 2008; 15: 1513-1525
        • Gweon HM
        • Cho N
        • Seo M
        • Chu AJ
        • Moon WK.
        Computer-aided evaluation as an adjunct to revised BI-RADS Atlas: improvement in positive predictive value at screening breast MRI.
        Eur Radiol. 2014; 24: 1800-1807
        • Newell D
        • Nie K
        • Chen JH
        • et al.
        Selection of diagnostic features on breast MRI to differentiate between malignant and benign lesions using computer-aided diagnosis: differences in lesions presenting as mass and non-mass-like enhancement.
        Eur Radiol. 2010; 20: 771-781
        • Cho N
        • Kim SM
        • Park JS
        • et al.
        Contralateral lesions detected by preoperative MRI in patients with recently diagnosed breast cancer: application of MR CAD in differentiation of benign and malignant lesions.
        Eur J Radiol. 2012; 81: 1520-1526
        • Eun NL
        • Son EJ
        • Gweon HM
        • Youk JH
        • Kim JA.
        The value of breast MRI for BI-RADS category 4B mammographic microcalcification: based on the 5th edition of BI-RADS.
        Clin Radiol. 2018; 73: 750-755
        • Gallego-Ortiz C
        • Martel AL.
        Improving the accuracy of computer-aided diagnosis for breast mr imaging by differentiating between mass and nonmass lesions.
        Radiology. 2016; 278: 679-688
        • Lambin P
        • Rios-Velazquez E
        • Leijenaar R
        • et al.
        Radiomics: extracting more information from medical images using advanced feature analysis.
        Eur J Cancer. 2012; 48: 441-446
        • Gillies RJ
        • Kinahan PE
        • Hricak H.
        Radiomics: images are more than pictures, they are data.
        Radiology. 2016; 278: 563-577
        • Truhn D
        • Schrading S
        • Haarburger C
        • et al.
        Radiomic versus convolutional neural networks analysis for classification of contrast-enhancing lesions at multiparametric breast MRI.
        Radiology. 2019; 290: 290-297
        • Zhou J
        • Zhang Y
        • Chang KT
        • et al.
        Diagnosis of benign and malignant breast lesions on DCE-MRI by using radiomics and deep learning with consideration of peritumor tissue.
        J Magn Reson Imaging. 2020; 51: 798-809
        • Zhang Y
        • Chen JH
        • Lin Y
        • et al.
        Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centers.
        Eur Radiol. 2021; 31: 2559-2567
        • Lee JG
        • Jun S
        • Cho YW
        • et al.
        Deep learning in medical imaging: general overview.
        Korean J Radiol. 2017; 18: 570-584
        • Al-Masni MA
        • Al-Antari MA
        • Park JM
        • et al.
        Detection and classification of the breast abnormalities in digital mammograms via regional Convolutional Neural Network.
        Annu Int Conf IEEE Eng Med Biol Soc. 2017; 2017: 1230-1233
        • Codari M
        • Schiaffino S
        • Sardanelli F
        • Trimboli RM.
        Artificial intelligence for breast MRI in 2008-2018: a systematic mapping review.
        AJR Am J Roentgenol. 2019; 212: 280-292
        • Sheth D
        • Giger ML.
        Artificial intelligence in the interpretation of breast cancer on MRI.
        J Magn Reson Imaging. 2020; 51: 1310-1324
        • Dalmış MU
        • Vreemann S
        • Kooi T
        • et al.
        Fully automated detection of breast cancer in screening MRI using convolutional neural networks.
        J Med Imaging (Bellingham). 2018; 5014502
        • Yap MH
        • Pons G
        • Marti J
        • et al.
        Automated breast ultrasound lesions detection using convolutional neural networks.
        IEEE J Biomed Health Inform. 2018; 22: 1218-1226
        • Kooi T
        • Litjens G
        • van Ginneken B
        • et al.
        Large scale deep learning for computer aided detection of mammographic lesions.
        Med Image Anal. 2017; 35: 303-312
        • Samala RK
        • Chan HP
        • Hadjiiski L
        • et al.
        Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography.
        Med Phys. 2016; 43: 6654
        • Ribli D
        • Horváth A
        • Unger Z
        • Pollner P
        • Csabai I.
        Detecting and classifying lesions in mammograms with Deep Learning.
        Sci Rep. 2018; 8: 4165
        • Zhang Y
        • Chan S
        • Park VY
        • et al.
        Automatic detection and segmentation of breast cancer on MRI Using Mask R-CNN trained on non-fat-sat images and tested on fat-sat images.
        Acad Radiol. 2022; 29: S135-S144
        • Ehteshami Bejnordi B
        • Veta M
        • Johannes van Diest P
        • et al.
        Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.
        JAMA. 2017; 318: 2199-2210
        • Kim EK
        • Kim HE
        • Han K
        • et al.
        Applying data-driven imaging biomarker in mammography for breast cancer screening: preliminary study.
        Sci Rep. 2018; 8: 2762
        • Lu SY
        • Wang SH
        • Zhang YD.
        SAFNet: A deep spatial attention network with classifier fusion for breast cancer detection.
        Comput Biol Med. 2022; 148105812
        • Gao F
        • Wu T
        • Li J
        • et al.
        SD-CNN: A shallow-deep CNN for improved breast cancer diagnosis.
        Comput Med Imaging Graph. 2018; 70: 53-62
        • Herent P
        • Schmauch B
        • Jehanno P
        • et al.
        Detection and characterization of MRI breast lesions using deep learning.
        Diagn Interv Imaging. 2019; 100: 219-225
        • Zhou J
        • Luo LY
        • Dou Q
        • et al.
        Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images.
        J Magn Reson Imaging. 2019; 50: 1144-1151
        • Jing X
        • Wielema M
        • Cornelissen LJ
        • et al.
        Using deep learning to safely exclude lesions with only ultrafast breast MRI to shorten acquisition and reading time.
        Eur Radiol. 2022;
        • Ayatollahi F
        • Shokouhi SB
        • Mann RM
        • Teuwen J.
        Automatic breast lesion detection in ultrafast DCE-MRI using deep learning.
        Med Phys. 2021; 48: 5897-5907
        • Zhou J
        • Liu YL
        • Zhang Y
        • et al.
        BI-RADS reading of non-mass lesions on dce-mri and differential diagnosis performed by radiomics and deep learning.
        Front Oncol. 2021; 11728224
        • Lin M
        • Chen JH
        • Nie K
        • et al.
        Algorithm-based method for detection of blood vessels in breast MRI for development of computer-aided diagnosis.
        J Magn Reson Imaging. 2009; 30: 817-824