A Cascaded Deep Learning–Based Artificial Intelligence Algorithm for Automated Lesion Detection and Classification on Biparametric Prostate Magnetic Resonance Imaging

Published:September 28, 2021DOI:

      Rationale and objectives

      Prostate MRI improves detection of clinically significant prostate cancer; however, its diagnostic performance has wide variation. Artificial intelligence (AI) has the potential to assist radiologists in the detection and classification of prostatic lesions. Herein, we aimed to develop and test a cascaded deep learning detection and classification system trained on biparametric prostate MRI using PI-RADS for assisting radiologists during prostate MRI read out.

      Materials and Methods

      T2-weighted, diffusion-weighted (ADC maps, high b value DWI) MRI scans obtained at 3 Tesla from two institutions (n = 1043 in-house and n = 347 Prostate-X, respectively) acquired between 2015 to 2019 were used for model training, validation, testing. All scans were retrospectively reevaluated by one radiologist. Suspicious lesions were contoured and assigned a PI-RADS category. A 3D U-Net-based deep neural network was used to train an algorithm for automated detection and segmentation of prostate MRI lesions. Two 3D residual neural network were used for a 4-class classification task to predict PI-RADS categories 2 to 5 and BPH. Training and validation used 89% (n = 1290 scans) of the data using 5 fold cross-validation, the remaining 11% (n = 150 scans) were used for independent testing. Algorithm performance at lesion level was assessed using sensitivities, positive predictive values (PPV), false discovery rates (FDR), classification accuracy, Dice similarity coefficient (DSC). Additional analysis was conducted to compare AI algorithm's lesion detection performance with targeted biopsy results.


      Median age was 66 years (IQR = 60-71), PSA 6.7 ng/ml (IQR = 4.7-9.9) from in-house cohort. In the independent test set, algorithm correctly detected 111 of 198 lesions leading to 56.1% (49.3%-62.6%) sensitivity. PPV was 62.7% (95% CI 54.7%-70.7%) with FDR of 37.3% (95% CI 29.3%-45.3%). Of 79 true positive lesions, 82.3% were tumor positive at targeted biopsy, whereas of 57 false negative lesions, 50.9% were benign at targeted biopsy. Median DSC for lesion segmentation was 0.359. Overall PI-RADS classification accuracy was 30.8% (95% CI 24.6%-37.8%).


      Our cascaded U-Net, residual network architecture can detect, classify cancer suspicious lesions at prostate MRI with good detection, reasonable classification performance metrics.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Siegel RL
        • Miller KD
        • Jemal A.
        Cancer statistics, 2020.
        CA Cancer J Clin. 2020; 70: 7-30
        • Ahmed HU
        • El-Shater Bosaily A
        • Brown LC
        • Gabe R
        • Kaplan R
        • Parmar MK
        • et al.
        Diagnostic accuracy of multi-parametric MRI and TRUS biopsy in prostate cancer (PROMIS): a paired validating confirmatory study.
        Lancet (London, England). 2017; 389: 815-822
        • Kasivisvanathan V
        • Rannikko AS
        • Borghi M
        • et al.
        MRI-targeted or standard biopsy for prostate-cancer diagnosis.
        The New England journal of medicine. 2018; 378: 1767-1777
        • Ahdoot M
        • Wilbur AR
        • Reese SE
        • et al.
        MRI-Targeted, systematic, and combined biopsy for prostate cancer diagnosis.
        The New England journal of medicine. 2020; 382: 917-928
        • Siddiqui MM
        • Rais-Bahrami S
        • Turkbey B
        • et al.
        Comparison of MR/ultrasound fusion-guided biopsy with ultrasound-guided biopsy for the diagnosis of prostate cancer.
        JAMA. 2015; 313: 390-397
        • Weinreb JC
        • Barentsz JO
        • Choyke PL
        • et al.
        PI-RADS Prostate Imaging - Reporting and Data System: 2015, Version 2.
        Eur Urol. 2016; 69: 16-40
        • Turkbey B
        • Rosenkrantz AB
        • Haider MA
        • et al.
        Prostate imaging reporting and data system version 2.1: 2019 update of prostate imaging reporting and data system version 2.
        European Urology. 2019; 0232: 1-12
        • Park KJ
        • Choi SH
        • Lee JS
        • et al.
        Interreader Agreement with Prostate Imaging Reporting and Data System Version 2 for Prostate Cancer Detection: A Systematic Review and Meta-Analysis.
        The Journal of urology. 2020; 204: 661-670
        • McKinney SM
        • Sieniek M
        • Godbole V
        • et al.
        International evaluation of an AI system for breast cancer screening.
        Nature. 2020; 577: 89-94
        • Gulshan V
        • Peng L
        • Coram M
        • et al.
        Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
        JAMA. 2016; 316: 2402-2410
        • Schelb P
        • Kohl S
        • Radtke JP
        • et al.
        Classification of Cancer at Prostate MRI: Deep Learning versus Clinical PI-RADS Assessment.
        Radiology. 2019; 293: 607-617
        • Litjens G
        • Debats O
        • Barentsz J
        • et al.
        Computer-Aided Detection of Prostate Cancer in MRI.
        IEEE Transactions on Medical Imaging. 2014; 33: 1083-1092
        • Clark K
        • Vendt B
        • Smith K
        • et al.
        The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository.
        Journal of Digital Imaging. 2013; 26: 1045-1057
        • Sanford T
        • Harmon SA
        • Turkbey EB
        • et al.
        Deep-Learning-Based Artificial Intelligence for PI-RADS Classification to Assist Multiparametric Prostate MRI Interpretation: A Development Study.
        Journal of magnetic resonance imaging: JMRI. 2020; 52: 1499-1507
        • Turkbey B
        • Fotin SV
        • Huang RJ
        • et al.
        Fully automated prostate segmentation on MRI: comparison with manual segmentation methods and specimen volumes.
        AJR Am J Roentgenol. 2013; 201: W720-W729
      1. Çiçek Ö, Abdulkadir A, Lienkamp SS, et al. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation2016; Cham: Springer International Publishing.

        • Greer MD
        • Shih JH
        • Barrett T
        • et al.
        All over the map: An interobserver agreement study of tumor location based on the PI-RADSv2 sector map.
        Journal of Magnetic Resonance Imaging. 2018; 48: 482-490
        • Rosenkrantz AB
        • Ginocchio LA
        • Cornfeld D
        • et al.
        Interobserver Reproducibility of the PI-RADS Version 2 Lexicon: A Multicenter Study of Six Experienced Prostate Radiologists.
        Radiology. 2016; 280: 793-804
        • Arif M
        • Schoots IG
        • Castillo Tovar J
        • et al.
        Clinically significant prostate cancer detection and segmentation in low-risk patients using a convolutional neural network on multi-parametric MRI.
        European radiology. 2020; 30: 6582-6592