Original Investigation| Volume 29, SUPPLEMENT 1, S135-S144, January 2022

Download started.


Automatic Detection and Segmentation of Breast Cancer on MRI Using Mask R-CNN Trained on Non–Fat-Sat Images and Tested on Fat-Sat Images

Published:December 11, 2020DOI:

      Rationale and Objectives

      Computer-aided methods have been widely applied to diagnose lesions on breast magnetic resonance imaging (MRI). The first step was to identify abnormal areas. A deep learning Mask Regional Convolutional Neural Network (R-CNN) was implemented to search the entire set of images and detect suspicious lesions.

      Materials and Methods

      Two DCE-MRI datasets were used, 241 patients acquired using non–fat-sat sequence for training, and 98 patients acquired using fat-sat sequence for testing. All patients have confirmed unilateral mass cancers. The tumor was segmented using fuzzy c-means clustering algorithm to serve as the ground truth. Mask R-CNN was implemented with ResNet-101 as the backbone. The neural network output the bounding boxes and the segmented tumor for evaluation using the Dice Similarity Coefficient (DSC). The detection performance, and the trade-off between sensitivity and specificity, was analyzed using free response receiver operating characteristic.


      When the precontrast and subtraction image of both breasts were used as input, the false positive from the heart and normal parenchymal enhancements could be minimized. The training set had 1469 positive slices (containing lesion) and 9135 negative slices. In 10-fold cross-validation, the mean accuracy = 0.86 and DSC = 0.82. The testing dataset had 1568 positive and 7264 negative slices, with accuracy = 0.75 and DSC = 0.79. When the obtained per-slice results were combined, 240 of 241 (99.5%) lesions in the training and 98 of 98 (100%) lesions in the testing datasets were identified.


      Deep learning using Mask R-CNN provided a feasible method to search breast MRI, localize, and segment lesions. This may be integrated with other artificial intelligence algorithms to develop a fully automatic breast MRI diagnostic system.

      Key Words


      3D-FLASH (three-dimensional fast low angle shot), AFROC (Alternative Free-response ROC), AI (Artificial Intelligence), BI-RADS (Breast Imaging Reporting and Data System), CAD (Computer-Aided Diagnosis), CNN (Convolutional Neural Network), DCE (Dynamic Contrast-Enhanced), DSC (Dice Similarity Coefficient), FCM (Fuzzy C-Means), FPN (Feature Pyramid Network), FROC (Free-response ROC), IoU (Intersection over Union), MIP (Maximum Intensity Projection), TE (Echo Time), TR (Repetition Time), R-CNN (Regional Convolutional Neural Network), ROC (Receiver Operating Characteristic), ROI (Region Of Interest)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Sites A
        SEER cancer statistics review 1975-2011.
        National Cancer Institute, Bethesda, MD2014
        • Kuhl C
        The current status of breast MR imaging part I. Choice of technique, image interpretation, diagnostic accuracy, and transfer to clinical practice.
        Radiology. 2007; 244: 356-378
        • Kuhl CK
        Current status of breast MR imaging part 2. Clinical applications.
        Radiology. 2007; 244: 672-691
        • Montemurro F
        • Martincich L
        • Sarotto I
        • et al.
        Relationship between DCE-MRI morphological and functional features and histopathological characteristics of breast cancer.
        Eur Radiol. 2007; 17: 1490-1497
        • Raikhlin A
        • Curpen B
        • Warner E
        • et al.
        Breast MRI as an adjunct to mammography for breast cancer screening in high-risk patients: retrospective review.
        AJR Am J Roentgenol. 2015; 204: 889-897
        • Ikeda DM
        • Hylton NM
        • Kinkel K
        • et al.
        Development, standardization, and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies.
        J Magn Reson Imaging. 2001; 13: 889-895
        • Renz D
        • Baltzer P
        • Kullnig P
        • et al.
        Clinical value of computer-assisted analysis in MR mammography. A comparison between two systems and three observers with different levels of experience.
        RoFo: Fortschritte Gebiete Rontgenstrahl Nuklearmed. 2008; 180: 968-976
        • Lehman CD
        • Blume JD
        • DeMartini WB
        • et al.
        Accuracy and interpretation time of computer-aided detection among novice and experienced breast MRI readers.
        Am J Roentgenol. 2013; 200: W683-W689
        • Djilas-Ivanovic D
        • Prvulovic N
        • Bogdanovic-Stojanovic D
        • et al.
        Breast MRI: intraindividual comparative study at 1.5 and 3.0 T; initial experience.
        J BU ON. 2012; 17: 65-72
        • Pediconi F
        • Catalano C
        • Occhiato R
        • et al.
        Breast lesion detection and characterization at contrast-enhanced MR mammography: gadobenate dimeglumine versus gadopentetate dimeglumine.
        Radiology. 2005; 237: 45-56
        • Pediconi F
        • Catalano C
        • Padula S
        • et al.
        Contrast-enhanced MR mammography: improved lesion detection and differentiation with gadobenate dimeglumine.
        Am J Roentgenol. 2008; 191: 1339-1346
        • Martincich L
        • Faivre-Pierret M
        • Zechmann CM
        • et al.
        Multicenter, double-blind, randomized, intraindividual crossover comparison of gadobenate dimeglumine and gadopentetate dimeglumine for breast MR imaging (DETECT Trial).
        Radiology. 2011; 258: 396-408
        • Gubern-Mérida A
        • Vreemann S
        • Martí R
        • et al.
        Automated detection of breast cancer in false-negative screening MRI studies from women at increased risk.
        Eur J Radiol. 2016; 85: 472-479
        • Chang Y-C
        • Huang Y-H
        • Huang C-S
        • et al.
        Computerized breast lesions detection using kinetic and morphologic analysis for dynamic contrast-enhanced MRI.
        Magn Resona Imaging. 2014; 32: 514-522
        • Dorrius MD
        • Jansen-van der Weide MC
        • van Ooijen PM
        • et al.
        Computer-aided detection in breast MRI: a systematic review and meta-analysis.
        Eur Radiol. 2011; 21: 1600-1608
        • Renz DM
        • Böttcher J
        • Diekmann F
        • et al.
        Detection and classification of contrast‐enhancing masses by a fully automatic computer‐assisted diagnosis system for breast MRI.
        J Magn Reson Imaging. 2012; 35: 1077-1088
        • Vignati A
        • Giannini V
        • De Luca M
        • et al.
        Performance of a fully automatic lesion detection system for breast DCE‐MRI.
        J Mag Reson Imaging. 2011; 34: 1341-1351
        • Codari M
        • Schiaffino S
        • Sardanelli F
        • et al.
        Artificial intelligence for breast MRI in 2008–2018: a systematic mapping review.
        Am J Roentgenol. 2019; 212: 280-292
        • Lee J-G
        • Jun S
        • Cho Y-W
        • et al.
        Deep learning in medical imaging: general overview.
        Korean J Radiol. 2017; 18: 570-584
        • Al-masni MA
        • Al-antari MA
        • Park J-M
        • et al.
        Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system.
        Comput Methods Prog Biomed. 2018; 157: 85-94
        • Dalmış MU
        • Vreemann S
        • Kooi T
        • et al.
        Fully automated detection of breast cancer in screening MRI using convolutional neural networks.
        J Med Imaging. 2018; 5014502
        • Sheth D
        • Giger ML
        Artificial intelligence in the interpretation of breast cancer on MRI.
        J Magn Reson Imaging. 2020; 51: 1310-1324
        • Yap MH
        • Pons G
        • Martí J
        • et al.
        Automated breast ultrasound lesions detection using convolutional neural networks.
        IEEE J Biomed Health Informatics. 2017; 22: 1218-1226
        • Kooi T
        • Litjens G
        • Van Ginneken B
        • et al.
        Large scale deep learning for computer aided detection of mammographic lesions.
        Med Image Analysis. 2017; 35: 303-312
        • Samala RK
        • Chan HP
        • Hadjiiski L
        • et al.
        Mass detection in digital breast tomosynthesis: deep convolutional neural network with transfer learning from mammography.
        Med Phys. 2016; 43: 6654-6666
        • Kim E-K
        • Kim H-E
        • Han K
        • et al.
        Applying data-driven imaging biomarker in mammography for breast cancer screening: preliminary study.
        Sci Rep. 2018; 8: 2762
        • Zhou J
        • Luo LY
        • Dou Q
        • et al.
        Weakly supervised 3D deep learning for breast cancer classification and localization of the lesions in MR images.
        J Magn Reson Imaging. 2019; 50: 1144-1151
        • Ribli D
        • Horváth A
        • Unger Z
        • et al.
        Detecting and classifying lesions in mammograms with deep learning.
        Sci Rep. 2018; 8: 4165
        • He K
        • Gkioxari G
        • Dollár P
        • et al.
        Mask r-cnn.
        in: Computer Vision (ICCV), 2017 IEEE International Conference on. IEEE, 2017: 2980-2988
        • Chang P
        • Kuoy E
        • Grinband J
        • et al.
        Hybrid 3D/2D convolutional neural network for hemorrhage evaluation on head CT.
        Am J Neuroradiol. 2018; 39: 1609-1616
        • Rohit Malhotra K
        • Davoudi A
        • Siegel S
        • et al.
        Autonomous detection of disruptions in the intensive care unit using deep mask R-CNN.
        in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018: 1863-1865
        • Couteaux V
        • Si-Mohamed S
        • Nempont O
        • et al.
        Automatic knee meniscus tear detection and orientation classification with Mask-RCNN.
        Diagn Interv Imaging. 2019; 100: 235-242
        • Nie K
        • Chen J-H
        • Hon JY
        • et al.
        Quantitative analysis of lesion morphology and texture features for diagnostic prediction in breast MRI.
        Acad Radiol. 2008; 15: 1513-1525
        • He K
        • Zhang X
        • Ren S
        • Sun J
        Deep residual learning for image recognition.
        in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778
        • Lin T-Y
        • Goyal P
        • Girshick R
        • He K
        • Dollár P
        Focal loss for dense object detection.
        IEEE Trans Pattern Analysis Machine Intelligence. 2020; 42: 318-327
        • Kingma D
        • Ba J
        Adam: a method for stochastic optimization.
        arXiv Preprint arXiv. 2014; 1412.6980
        • Abadi M
        • Barham P
        • Chen J
        • et al.
        TensorFlow: a system for large-scale machine learning.
        in: 2016: 265-283
        • Chakraborty DP
        • Winter LHL
        Free-response methodology: alternate analysis and a new observer performance experiment.
        Radiology. 1990; 174: 873-881
        • Bejnordi BE
        • Veta M
        • van Diest PJ
        • et al.
        Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.
        JAMA. 2017; 318: 2199-2210
        • Rajpurkar P
        • Irvin J
        • Zhu K
        • et al.
        CheXNet: radiologist-level pneumonia detection on chest X-Rays with Deep Learning.
        arXiv Preprint arXiv. 2017; 1711.05225
        • Wang J
        • Fang Z
        • Lang N
        • Yuan H
        • et al.
        A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks.
        Comput Biol Med. 2017; 84: 137-146
        • Zhang Y
        • Chen JH
        • Lin Y
        • et al.
        Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centers.
        Eur Radiol. 2020; (accepted manuscript ID#: EURA-D-20-01789R2)
        • Zhou J
        • Zhang Y
        • Chang KT
        • et al.
        Diagnosis of benign and malignant breast lesions on DCE-MRI by using radiomics and deep learning with consideration of peritumor tissue.
        J Magn Reson Imaging. 2020; 51: 798-809
        • Yurttakal AH
        • Erbay H
        • İkizceli T
        • et al.
        Detection of breast cancer via deep convolution neural networks using MRI images.
        Multimed Tools Appl. 2020; 79: 15555-15573