Original Investigation|Articles in Press

Image Augmentation based on Variational Autoencoder for Breast Tumor Segmentation

Published:February 16, 2023DOI:

      Rationale and Objectives

      Breast tumor segmentation based on Dynamic Contrast-Enhanced Magnetic Resonance Imaging is significant step for computable radiomics analysis of breast cancer. Manual tumor annotation is time-consuming process and involves medical acquaintance, biased, inclined to error, and inter-user discrepancy. A number of modern trainings have revealed the capability of deep learning representations in image segmentation.

      Materials and Methods

      Here, we describe a 3D Connected-UNets for tumor segmentation from 3D Magnetic Resonance Imagings based on encoder-decoder architecture. Due to a restricted training dataset size, a variational auto-encoder outlet is supplementary to renovate the input image itself in order to identify the shared decoder and execute additional controls on its layers. Based on initial segmentation of Connected-UNets, fully connected 3D provisional unsystematic domain is used to enhance segmentation outcomes by discovering 2D neighbor areas and 3D volume statistics. Moreover, 3D connected modules evaluation is used to endure around large modules and decrease segmentation noise.


      The proposed method has been assessed on two widely offered datasets, explicitly INbreast and the curated breast imaging subset of digital database for screening mammography The proposed model has also been estimated using a private dataset.


      The experimental results show that the proposed model outperforms the state-of-the-art methods for breast tumor segmentation.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Siegel R.L.
        • Miller K.D.
        • Fuchs H.E.
        • et al.
        Cancer statistics, 2022.
        CA Cancer J Clin. 2022; 72: 7-33
        • Lauby-Secretan B.
        • Chiara S.
        • Dana L.
        • et al.
        Breast-cancer screening-viewpoint of the IARC working group.
        New Engl J Med. 2015; 372: 2353-2358
        • Celik Y
        • Talo M
        • Yildirim O
        • et al.
        Automated invasive ductal carcinoma detection based using deep transfer learning with whole-slide images.
        Pattern Recognit Lett. 2020; 133: 232-239
        • Taghanaki SA
        • Abhishek K
        • Cohen JP
        • et al.
        Deep semantic segmentation of natural and medical images: a review.
        Artif Intell Rev. 2021; 54: 137-178
        • Wang G.
        • Li W.
        • Ourselin S.
        • et al.
        Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks.
        International MICCAI Brainlesion Workship. Springer, Cham2017: 178-190
        • Giacomello E.
        • Loiacono D.,
        • Mainradi L.
        Brain MRI tumor segmentation with adversarial networks.
        in: International Joint Conference on Neural Networks (IJCNN). IEEE, 2020: 1-8
        • Dhungel N.
        • Carneiro G.
        • Bradley A.P.
        A deep learning approach for the analysis of masses in mammograms with minimal user intervention.
        Med Image Anal. 2017; 37: 114-128
        • Shi P.
        • Zhong J.
        • Rampun A.
        A hierarchical pipeline for breast boundary segmentation and calcification detection in mammograms.
        Comput Biol Med. 2018; 96: 178-188
        • Gao Y.
        • Geras K.J.
        • Lewin A.A.
        New frontiers: an update on computer aided diagnosis for breast imaging in the age of artificial intelligence.
        Am J Roentgenol. 2019; 212: 300-307
        • Henriksen E.L.
        • Carlsen J.F.
        • Vejborg I.M.
        The efficacy of using computer-aided detection (CAD) for detection of breast cancer in mammography screening: a systematic review.
        Acta Raiol. 2019; 60: 13-18
        • Mullooly M
        • Bejnordi BE
        • Pfeiffer RM
        • et al.
        Application of convolutional neural networks to breast biopsies to delineate tissue correlates of mammographic breast density.
        npj Breast Cancer. 2019; 5: 1-11
        • Dhungel N.
        • Carneiro G.,
        • Bradley A.P.
        Deep learning and structured prediction for the segmentation of mass in mammograms.
        in: International Conference on Medical Image Computing and Computer- Assisted Intervention, Cham Springer, 2015: 605-612
        • Hesamian M.H.
        • Jia W.
        • He X..
        Deep learning techniques for medical image segmentation: achievements and challenges.
        J. Digit. Imaging. 2019; 32: 582-596
        • Murtaza G
        • Shuib L
        • Wahab AWA
        • et al.
        Deep learning-based breast cancer classification through medical imaging modalities: state of the art and research challenges.
        Artif Intell Rev. 2020; 53: 1655-1720
        • Tajbakhsh N
        • Jeyaseelan L
        • Li Q
        • et al.
        Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation.
        Med Image Anal. 2020; 63101693
        • Xu X
        • Fu L
        • Chen Y
        • et al.
        Breast region segmentation being convolutional neural network in dynamic contrast enhanced MRI.
        in: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2018: 750-753
        • Huang G.
        • Liu Z.
        • Van Der Maaten L.
        Densely connected convolutional networks.
        in: Proc. IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 4700-4708
        • Zhou S
        • Nie D
        • Adeli E
        • et al.
        High-resolution encoder-decoder networks for low-contrast medical image segmentation.
        IEEE Trans Image Process. 2019; 29: 461-475
        • Tang P
        • Liang Q
        • Yan X
        • et al.
        Efficient skin lesion segmentation using separable-UNet with stochastic weight averaging.
        Comput Methods Programs Biomed. 2019; 178: 289-301
        • Li S.
        • Chen Y.
        • Yang S.
        Cascade dense-UNet for prostate segmentation in MR images.
        in: International Conference on Intelligent Computing, Cham Springer, 2019: 481-490
        • Jalalian A
        • Mashohor S
        • Mahmud R
        • et al.
        Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection.
        EXCLI J. 2017; 16: 113
        • Zhu W.
        • Xiang X.
        • Tran T.D.
        Adversarial deep structured nets for mass segmentation from mammograms.
        in: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE, 2018: 847-850
        • Sing VK
        • Rashwan H
        • Romani S
        • et al.
        Breast tumor segmentation and shape classification in mammograms using generative adversarial and convolutional neural network.
        Expert Syst Appl. 2020; 139112855
        • Ronneberger O.
        • Fischer P.
        • Brox T.
        U-net: convolutional networks for bio-medical image segmentation.
        in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham Springer, 2015: 234-241
        • Soulami K.B.
        • Kaabouch N.
        • Saidi M.N.
        Breast cancer: one-stage automated detection, segmentation, and classification of digital mammograms using UNet model based-semantic segmentation.
        Biomed. Signal Process Control. 2021; 66102481
        • Abdelhafiz D.
        • Bi J.
        • Ammar R.
        Convolutional neural network for automated mass segmentation in mammography.
        BMC Bioinformatics. 2020; 21: 1-9
        • Baccouche A.
        • Garcia-Zapirain B.
        • Castillo Olea
        • et al.
        Connected-UNets: a deep learning architecture for breast mass segmentation.
        NPJ Breast Cancer. 2021; 7: 1-12
        • Moreira I.C
        • Amaral I.
        • Domingues I.
        • et al.
        Inbreast: toward a full-field digital mammographic database.
        Acad Radiol. 2012; 19: 236-248
        • Huang M.L.
        • Lin T.Y.
        Dataset of breast mammography images with masses.
        Data Brief. 2020; 31105928
        • Heath M.
        • Bowyer K.
        • Kopans D.
        • et al.
        Current status of the digital database for screening mammography.
        Digital Mammography. Springer, Cham, Switzerland1998: 457-460
        • Lee R.S.
        • Gimenez F.
        • Hoogi A.
        • et al.
        A curated mammography dataset for use in computer-aided detection and diagnosis research.
        Sci. Data. 2017; 4: 1-9
        • AI-Masni M.A.
        • AI-Antari M.A.
        • Park J.M.
        • et al.
        Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system.
        Compu. Methods Programs Biomed. 2018; 157: 85-94
        • Hai J.
        • Qiao K.
        • Chen J.
        • et al.
        Fully convolutional densenet with multiscale context for automated breast tumor segmentation.
        J. Healthc. Eng. 2019; 8415485
        • Dhal K.G.
        • Das A.
        • Ray S.
        • et al.
        Histogram equalization variants as optimization problems: a review.
        Arch. Comput. Methods Eng. 2021; 28: 1471-1496
        • Huang Z.
        • Zhang Y.
        • Li Q.
        • et al.
        Spatially adaptive denoising for X-ray cardiovascular angiogram images.
        Biomed. Signal Process. Control. 2018; 40: 131-139
        • Huang Z.
        • Li X.
        • Wang N.
        • et al.
        Simultaneous denoising and enhancement for X-ray angiograms by employing spatial-frequency filter.
        Optik. 2020; 208164287
        • Huang Z.
        • Zhang Y.
        • Li Q.
        • et al.
        Joint anlaysis and weighted synthesis sparsity priors for simultaneous denoising and destriping optical remote sensing images.
        IEEE Trans Geosci Remote Sens. 2020; 58: 6958-6982
        • Rao B.S.
        Dynamic histogram equalization for contrast enhancement for digital images.
        Appl Soft Comput. 2020; 89106114
        • Kingma Diederik P
        • Welling Max
        An introduction to variational autoencoders.
        Found Trends Mach Learn. 2019; 12: 307-392
      1. Kingma Diederik P, Welling Max, Auto-encoding variational bayes, foundations and [email protected] in machine learning: 2022.

      2. Wu Yuxin, He Kaiming, Group normalization, computer vision and pattern recognition, 2018, arXiv.1803.08494.

      3. He Kaiming, Zhang Xiangyu, Reri Shaoqing, et al. Delving deep into rectifiers: surpassing human-level performance on imageNet classification, computer vision and pattern recognition, 2015, arXiv:1502.01852.

      4. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980.