Advertisement

Utility of a Rule-Based Algorithm in the Assessment of Standardized Reporting in PI-RADS

      Rationale and Objectives

      Adoption of the Prostate Imaging Reporting & Data System (PI-RADS) has been shown to increase detection of clinically significant prostate cancer on prostate mpMRI. We propose that a rule-based algorithm based on Regular Expression (RegEx) matching can be used to automatically categorize prostate mpMRI reports into categories as a means by which to assess for opportunities for quality improvement.

      Materials and Methods

      All prostate mpMRIs performed in the Duke University Health System from January 2, 2015, to January 29, 2021, were analyzed. Exclusion criteria were applied, for a total of 5343 male patients and 6264 prostate mpMRI reports. These reports were then analyzed by our RegEx algorithm to be categorized as PI-RADS 1 through PI-RADS 5, Recurrent Disease, or “No Information Available.” A stratified, random sample of 502 mpMRI reports was reviewed by a blinded clinical team to assess performance of the RegEx algorithm.

      Results

      Compared to manual review, the RegEx algorithm achieved overall accuracy of 92.6%, average precision of 88.8%, average recall of 85.6%, and F1 score of 0.871. The clinical team also reviewed 344 cases that were classified as “No Information Available,” and found that in 150 instances, no numerical PI-RADS score for any lesion was included in the impression section of the mpMRI report.

      Conclusion

      Rule-based processing is an accurate method for the large-scale, automated extraction of PI-RADS scores from the text of radiology reports. These natural language processing approaches can be used for future initiatives in quality improvement in prostate mpMRI reporting with PI-RADS.

      Keywords

      Abbreviations:

      mpMRI (multiparametric magnetic resonance imaging), csPCa (clinically significant prostate cancer), PI-RADS (Prostate Imaging Reporting & Data System), NLP (natural language processing), AI (artificial intelligence), ML (machine learning), RegEx (regular expression), QI (quality improvement)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. Siegel RL, Miller KD, Fuchs HE, et al.Cancer statistics, 2021. CA Cancer J Clin2021;71(1):7-33. doi:10.3322/caac.21654

        • Oberlin DT
        • Casalino DD
        • Miller FH
        • et al.
        Dramatic increase in the utilization of multiparametric magnetic resonance imaging for detection and management of prostate cancer.
        Abdom Radiol (NY). 2017; 42: 1255-1258https://doi.org/10.1007/s00261-016-0975-5
        • Turkbey B
        • Rosenkrantz AB
        • Haider MA
        • et al.
        Prostate imaging reporting and data system version 2.1: 2019 update of prostate imaging reporting and data system version 2.
        Eur Urol. 2019; 76: 340-351https://doi.org/10.1016/j.eururo.2019.02.033
        • Woo S
        • Suh CH
        • Kim SY
        • et al.
        Diagnostic performance of prostate imaging reporting and data system version 2 for detection of prostate cancer: a systematic review and diagnostic meta-analysis.
        Eur Urol. 2017; 72: 177-188https://doi.org/10.1016/j.eururo.2017.01.042
        • Greer MD
        • Brown AM
        • Shih JH
        • et al.
        Accuracy and agreement of PIRADSv2 for prostate cancer mpMRI: a multireader study.
        J Magn Reson Imaging. 2017; 45: 579-585https://doi.org/10.1002/jmri.25372
        • Greer MD
        • Shih JH
        • Lay N
        • et al.
        Interreader variability of prostate imaging reporting and data system version 2 in detecting and assessing prostate cancer lesions at prostate MRI.
        AJR Am J Roentgenol. 2019; : 1-8https://doi.org/10.2214/ajr.18.20536
        • Smith CP
        • Harmon SA
        • Barrett T
        • et al.
        Intra- and interreader reproducibility of PI-RADSv2: a multireader study.
        J Magn Reson Imaging. 2019; 49: 1694-1703https://doi.org/10.1002/jmri.26555
        • Sonn GA
        • Fan RE
        • Ghanouni P
        • et al.
        Prostate magnetic resonance imaging interpretation varies substantially across radiologists.
        Eur Urol Focus. 2019; 5: 592-599https://doi.org/10.1016/j.euf.2017.11.010
        • Stabile A
        • Giganti F
        • Kasivisvanathan V
        • et al.
        Factors influencing variability in the performance of multiparametric magnetic resonance imaging in detecting clinically significant prostate cancer: a systematic literature review.
        Eur Urol Oncol. 2020; 3: 145-167https://doi.org/10.1016/j.euo.2020.02.005
        • Chen F
        • Cen S
        • Palmer S.
        Application of rostate maging eporting and ata ystem version 2 (PI-RADS v2): interobserver agreement and positive predictive value for localization of intermediate- and high-grade prostate cancers on multiparametric magnetic resonance imaging.
        Acad Radiol. 2017; 24: 1101-1106https://doi.org/10.1016/j.acra.2017.03.019
        • Mussi TC
        • Yamauchi FI
        • Tridente CF
        • et al.
        Interobserver agreement and positivity of PI-RADS version 2 among radiologists with different levels of experience.
        Acad Radiol. 2019; 26: 1017-1022https://doi.org/10.1016/j.acra.2018.08.013
        • Schwartz LH
        • Panicek DM
        • Berk AR
        • et al.
        Improving communication of diagnostic radiology findings through structured reporting.
        Radiology. 2011; 260: 174-181https://doi.org/10.1148/radiol.11101913
        • Kahn Jr., CE
        • Heilbrun ME
        • Applegate KE
        From guidelines to practice: how reporting templates promote the use of radiology practice guidelines.
        J Am Coll Radiol. 2013; 10: 268-273https://doi.org/10.1016/j.jacr.2012.09.025
        • Shaish H
        • Feltus W
        • Steinman J
        • et al.
        Impact of a structured reporting template on adherence to prostate imaging reporting and data system version 2 and on the diagnostic performance of prostate MRI for clinically significant prostate cancer.
        J Am Coll Radiol. 2018; 15: 749-754https://doi.org/10.1016/j.jacr.2018.01.034
        • Spilseth B
        • Ghai S
        • Patel NU
        • et al.
        A comparison of radiologists' and urologists' opinions regarding prostate MRI reporting: results from a survey of specialty societies.
        AJR Am J Roentgenol. 2018; 210: 101-107https://doi.org/10.2214/ajr.17.18241
        • Caputo JM
        • Pina LA
        • Sebesta EM
        • et al.
        Innovative standardized reporting template for prostate mpMRI improves clarity and confidence in the report.
        World J Urol. 2021; 39: 2447-2452https://doi.org/10.1007/s00345-020-03487-3
        • Barrett T
        • Ghafoor S
        • Gupta RT
        • et al.
        Prostate MRI qualification: AJR expert panel narrative review.
        AJR Am J Roentgenol. 11 2022; https://doi.org/10.2214/ajr.22.27615
        • Meystre SM
        • Savova GK
        • Kipper-Schuler KC
        Extracting information from textual documents in the electronic health record: a review of recent research.
        Yearb Med Inform. 2008; : 128-144
        • Mozayan A
        • Fabbri AR
        • Maneevese M
        • et al.
        Practical guide to natural language processing for radiology.
        Radiographics. 2021; 41: 1446-1453https://doi.org/10.1148/rg.2021200113
        • Sippo DA
        • Warden GI
        • Andriole KP
        • et al.
        Automated extraction of BI-RADS final assessment categories from radiology reports with natural language processing.
        J Digit Imaging. 2013; 26: 989-994https://doi.org/10.1007/s10278-013-9616-5
        • Zheng C
        • Huang BZ
        • Agazaryan AA
        • et al.
        Natural language processing to identify pulmonary nodules and extract nodule characteristics from radiology reports.
        Chest. 2021; 160: 1902-1914https://doi.org/10.1016/j.chest.2021.05.048
        • Steinkamp J
        • Cook TS.
        Basic artificial intelligence techniques: natural language processing of radiology reports.
        Radiol Clin North Am. 2021; 59: 919-931https://doi.org/10.1016/j.rcl.2021.06.003
        • Bui DD
        • Zeng-Treitler Q.
        Learning regular expressions for clinical text classification.
        J Am Med Inform Assoc. 2014; 21: 850-857https://doi.org/10.1136/amiajnl-2013-002411
        • Davenport MS
        • Downs E
        • George AK
        • et al.
        Prostate imaging and data reporting system version 2 as a radiology performance metric: an analysis of 18 abdominal radiologists.
        J Am Coll Radiol. 2021; 18: 1069-1076https://doi.org/10.1016/j.jacr.2021.02.032
        • Weinreb JC
        • Barentsz JO
        • Choyke PL
        • et al.
        PI-RADS prostate imaging - reporting and data system: 2015, version 2.
        Eur Urol. 2016; 69: 16-40https://doi.org/10.1016/j.eururo.2015.08.052
      2. R: A language and environment for statistical computing.R foundation for statistical computing; 2021. Available at: https://www.R-project.org/. Accessed 20 May 2022.

        • Esses SJ
        • Taneja SS
        • Rosenkrantz AB.
        Imaging facilities' adherence to PI-RADS v2 minimum technical standards for the performance of prostate MRI.
        Acad Radiol. 2018; 25: 188-195https://doi.org/10.1016/j.acra.2017.08.013
        • Sackett J
        • Shih JH
        • Reese SE
        • et al.
        Quality of prostate MRI: is the PI-RADS standard sufficient?.
        Acad Radiol. 2021; 28: 199-207https://doi.org/10.1016/j.acra.2020.01.031
        • Cullivan O
        • Roche E
        • Hegazy M
        • et al.
        A critical analysis of deficiencies in the quality of information contained in prostate multiparametric MRI requests and reports.
        Ir J Med Sci. 2022; https://doi.org/10.1007/s11845-021-02875-x
        • Rosenkrantz AB
        • Pujara AC
        • Taneja SS.
        Use of a quality improvement initiative to achieve consistent reporting of level of suspicion for tumor on multiparametric prostate MRI.
        AJR Am J Roentgenol. 2016; 206: 1040-1044https://doi.org/10.2214/ajr.15.15768
        • Magudia K
        • Bridge CP
        • Andriole KP
        • et al.
        The Trials and Tribulations of Assembling Large Medical Imaging Datasets for Machine Learning Applications.
        J Digit Imaging. 2021; 34: 1424-1429https://doi.org/10.1007/s10278-021-00505-7
        • Tushar FI
        • D'Anniballe VM
        • Hou R
        • et al.
        Classification of multiple diseases on body CT scans using weakly supervised deep learning.
        Radiol Artif Intell. 2022; 4e210026https://doi.org/10.1148/ryai.210026
        • D'Anniballe VM
        • Tushar FI
        • Faryna K
        • et al.
        Multi-label annotation of text reports from computed tomography of the chest, abdomen, and pelvis using deep learning.
        BMC Med Inform Decis Mak. 2022; 22: 102https://doi.org/10.1186/s12911-022-01843-4
        • Cook TS
        • Paulus R
        • Gillis LB
        • et al.
        Development and implementation of a multisite registry using structured templates for actionable findings in the kidney.
        J Am Coll Radiol. 2022; https://doi.org/10.1016/j.jacr.2022.02.030