Original Investigation|Articles in Press

Reduction in Radiologist Interpretation Time of Serial CT and MR Imaging Findings with Deep Learning Identification of Relevant Priors, Series and Finding Locations

      Rationale and Objectives

      Finding comparison to relevant prior studies is a requisite component of the radiology workflow. The purpose of this study was to evaluate the impact of a deep learning tool simplifying this time-consuming task by automatically identifying and displaying the finding in relevant prior studies.

      Materials and Methods

      The algorithm pipeline used in this retrospective study, TimeLens (TL), is based on natural language processing and descriptor-based image-matching algorithms. The dataset used for testing comprised 3872 series of 246 radiology examinations from 75 patients (189 CTs, 95 MRIs). To ensure a comprehensive testing, five finding types frequently encountered in radiology practice were included: aortic aneurysm, intracranial aneurysm, kidney lesion, meningioma, and pulmonary nodule. After a standardized training session, nine radiologists from three university hospitals performed two reading sessions on a cloud-based evaluation platform resembling a standard RIS/PACS. The task was to measure the diameter of the finding-of-interest on two or more exams (a most recent and at least one prior exam): first without use of TL, and a second session at an interval of at least 21 days with the use of TL. All user actions were logged for each round, including time needed to measure the finding at all timepoints, number of mouse clicks, and mouse distance traveled. The effect of TL was evaluated in total, per finding type, per reader, per experience (resident vs. board-certified radiologist), and per modality. Mouse movement patterns were analyzed with heatmaps. To assess the effect of habituation to the cases, a third round of readings was performed without TL.


      Across scenarios, TL reduced the average time needed to assess a finding at all timepoints by 40.1% (107 vs. 65 seconds; p < 0.001). Largest accelerations were demonstrated for assessment of pulmonary nodules (−47.0%; p < 0.001). Less mouse clicks (−17.2%) were needed for finding evaluation with TL, and mouse distance traveled was reduced by 38.0%. Time needed to assess the findings increased from round 2 to round 3 (+27.6%; p < 0.001). Readers were able to measure a given finding in 94.4% of cases on the series initially proposed by TL as most relevant series for comparison. The heatmaps showed consistently simplified mouse movement patterns with TL.


      A deep learning tool significantly reduced both the amount of user interactions with the radiology image viewer and the time needed to assess findings of interest on cross-sectional imaging with relevant prior exams.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Acr. PRACTICE PARAMETER 1 Communication Diagnostic Imaging.
        • Haygood T.M.
        • Mullins B.
        • Sun J.
        • et al.
        Consultation and citation rates for prior imaging studies and documents in radiology.
        J Med Imaging. 2018; 5 (Available at: /pmc/articles/PMC5938465/. Accessed July 22, 2022): 1
        • Anon
        Radiology review: a national review of radiology reporting within the NHS in England.
        • Bruno M.A.
        • Walker E.A.
        • Abujudeh H.H.
        Understanding and confronting our mistakes: the epidemiology of error in radiology and strategies for error reduction.
        Radiographics. 2015; 35: 1668-1676
        • Infante M.
        • Berghmans T.
        • Heuvelmans M.A.
        • et al.
        Slow-growing lung cancer as an emerging entity: from screening to clinical management.
        Eur Respir J. 2013; 42 (Available at: Accessed March 9, 2023): 1706-1722
      2. Anon. Radiology: artificial intelligence. Available at: Accessed February 2, 2022.

        • Lakhani P.
        • Prater A.B.
        • Hutson R.K.
        • et al.
        Machine learning in radiology: applications beyond image interpretation.
        J Am Coll Radiol. 2018; 15: 350-359
        • Jungblut L.
        • Blüthgen C.
        • Polacin M.
        • et al.
        First performance evaluation of an artificial intelligence-based computer-aided detection system for pulmonary nodule evaluation in dual-source photon-counting detector CT at different low-dose levels.
        Invest Radiol. 2022; 57 (Available at: Accessed October 29, 2022): 108-114
        • Komolafe T.E.
        • Cao Y.
        • Nguchu B.A.
        • et al.
        Diagnostic test accuracy of deep learning detection of COVID-19: a systematic review and meta-analysis.
        Acad Radiol. 2021; 28 (Available at: Accessed February 5, 2023): 1507-1523
        • Mehralivand S.
        • Yang D.
        • Harmon S.A.
        • et al.
        A cascaded deep learning–based artificial intelligence algorithm for automated lesion detection and classification on biparametric prostate magnetic resonance imaging.
        Acad Radiol. 2022; 29 (Available at: Accessed February 5, 2023): 1159-1168
        • Wang J.
        • Sun J.
        • Xu J.
        • et al.
        Detection of intracranial aneurysms using multiphase CT angiography with a deep learning model.
        Acad Radiol. 2023; (Available at: Accessed February 5, 2023)
        • Krupinski E.A.
        • Radvany M.
        • Levy A.
        • et al.
        Enhanced visualization processing: effect on workflow.
        Acad Radiol. 2001; 8 (Available at: Accessed February 5, 2023): 1127-1133
        • Brown M.
        • Browning P.
        • Wahi-Anwar M.W.
        • et al.
        Integration of chest CT CAD into the clinical workflow and impact on radiologist efficiency.
        Acad Radiol. 2019; 26 (Available at: Accessed February 5, 2023): 626-631
        • Mikolov T.
        • Chen K.
        • Corrado G.
        • et al.
        Efficient estimation of word representations in vector space.
        in: 1st international conference on learning representations, ICLR 2013 – workshop track proceedings. 2013 (Available at: Accessed October 29, 2022.)
        • Cai J.
        • Tang Y.
        • Yan K.
        • et al.
        Deep lesion tracker: monitoring lesions in 4D longitudinal imaging, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
        (CVPR). 2021; : 15154-15164
        • Forsberg D.
        • Gupta A.
        • Mills C.
        • et al.
        Synchronized navigation of current and prior studies using image registration improves radiologist’s efficiency.
        Int J Comput Assist Radiol Surg. 2017; 12 (Available at: Accessed December 17, 2021): 431-438
        • Erickson B.J.
        • Mandrekar J.
        • Wang L.
        • et al.
        Effect of automated image registration on radiologist interpretation.
        J Digit Imaging. 2007; 20 (Available at: Accessed February 2, 2022): 105-113
        • Czaja S.J.
        • Charness N.
        • Fisk A.D.
        • et al.
        Factors predicting the use of technology: findings from the Center for Research and Education on Aging and Technology Enhancement (CREATE).
        Psychol Aging. 2006; 21 (Available at: /pmc/articles/PMC1524856/. Accessed February 2, 2022): 333
        • Dixon S.
        Diagnostic imaging dataset statistical release. 2021