Predicting High Imaging Utilization Based on Initial Radiology Reports:

A Feasibility Study of Machine Learning
Published:October 27, 2015DOI:

      Rationale and Objectives

      Imaging utilization has significantly increased over the last two decades, and is only recently showing signs of moderating. To help healthcare providers identify patients at risk for high imaging utilization, we developed a prediction model to recognize high imaging utilizers based on their initial imaging reports.

      Materials and Methods

      The prediction model uses a machine learning text classification framework. In this study, we used radiology reports from 18,384 patients with at least one abdomen computed tomography study in their imaging record at Stanford Health Care as the training set. We modeled the radiology reports in a vector space and trained a support vector machine classifier for this prediction task. We evaluated our model on a separate test set of 4791 patients. In addition to high prediction accuracy, in our method, we aimed at achieving high specificity to identify patients at high risk for high imaging utilization.


      Our results (accuracy: 94.0%, sensitivity: 74.4%, specificity: 97.9%, positive predictive value: 87.3%, negative predictive value: 95.1%) show that a prediction model can enable healthcare providers to identify in advance patients who are likely to be high utilizers of imaging services.


      Machine learning classifiers developed from narrative radiology reports are feasible methods to predict imaging utilization. Such systems can be used to identify high utilizers, inform future image ordering behavior, and encourage judicious use of imaging.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Medicare Payment Advisory Commission
        Options to improve Medicare's payments to physicians. Statement of Glenn M. Hackbarth, Chairman, Medicare Payment Advisory Commission.
        (Available at:) (Accessed August 25, 2015)
        • Levin D.C.
        • Rao V.M.
        • Parker L.
        Physician orders contribute to high-tech imaging slowdown.
        Health Aff (Millwood). 2010; 29: 189-195
        • Fazel R.
        • Krumholz H.M.
        • Wang Y.
        • et al.
        Exposure to low-dose ionizing radiation from medical imaging procedures.
        N Engl J Med. 2009; 361: 849-857
        • Bhargavan M.
        • Sunshine J.H.
        Utilization of radiology services in the United States: levels and trends in modalities, regions, and populations.
        Radiology. 2005; 234: 824-832
        • Winter A.
        • Stensland J.
        Introduction: expert panel on new research on use of imaging services [Presentation to Medicare Payment Advisory Commission, September 5, 2008].
        (Available at:) (Accessed August 25, 2015)
        • Iglehart J.K.
        Health insurers and medical-imaging policy—a work in progress.
        N Engl J Med. 2009; 360: 1030-1037
        • Lee D.W.
        • Duszak R.
        • Hughes D.R.
        Comparative analysis of Medicare spending for medical imaging: sustained dramatic slowdown compared with other services.
        AJR Am J Roentgenol. 2013; 201: 1277-1282
        • Lee D.W.
        • Levy F.
        The sharp slowdown in growth of medical imaging: an early analysis suggests combination of policies was the cause.
        Health Aff (Millwood). 2012; 31: 1876-1884
        • Dodoo M.S.
        • Duszak R.
        • Hughes D.R.
        Trends in the utilization of medical imaging from 2003 to 2011: clinical encounters offer a complementary patient-centered focus.
        J Am Coll Radiol. 2013; 10: 507-512
        • Levin D.C.
        • Rao V.M.
        • Parker L.
        • et al.
        Bending the curve: the recent marked slowdown in growth of noninvasive diagnostic imaging.
        AJR Am J Roentgenol. 2011; 196: W25-W29
        • America's Health Insurance Plans (AHIP)
        Ensuring quality through appropriate use of diagnostic imaging [white paper].
        (Accessed August 25, 2015)
        • Glabman M.
        Health plans strain to contain rapidly rising costs of imaging.
        (Managed Care magazine) (Accessed August 25, 2015)
        • Medicare Payment Advisory Commission (MEDPAC)
        June 2014: a data book: healthcare spending and the Medicare Program.
        (Available at:) (Accessed August 25, 2015)
        • United States Government Accountability Office
        Report to congressional requesters: Medicare Part B imaging services: rapid spending growth and shift to physician offices indicated need for CMS to consider additional management practices GAO-08-452.
        (Government Accountability Office; Available at:) (Accessed August 25, 2015)
        • Smith-bindman R.
        • Miglioretti D.L.
        • Johnson E.
        • et al.
        Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996–2010.
        JAMA. 2012; 307: 2400-2409
        • Bayati M.
        • Braverman M.
        • Gillam M.
        • et al.
        Data-driven decisions for reducing readmissions for heart failure: general methodology and case study.
        PLoS ONE. 2014; 9: e109264
        • Fitzgerald M.
        Introducing regular expressions.
        1st ed. O'Reilly Media, Sebastopol2012: 13-28
        • Cortes C.
        • Vapnik V.
        Support-vector networks.
        Mach Learn. 1995; 20: 273-297
        • Manning C.D.
        • Schütze H.
        Foundations of statistical natural language processing.
        MIT Press, Cambridge1999 (543 pp)
        • Chang C.-C.
        • Lin C.-J.
        ACM Trans Intell Syst Technol. 2011; 2: 1-27
        • Chen Y.-W.
        • Lin C.-J.
        Combining SVMs with various feature selection strategies.
        in: Guyon I. Nikravesh M. Gunn S. Feature extraction SE—13. Springer Berlin, Heidelberg2006: 315-324
        • Greene W.H.
        Econometric analysis.
        7th ed. Pearson Education, Boston2012: 803-806
        • Deng L.
        • Yu D.
        Deep learning: methods and applications.
        Found Trends Signal Processing. 2013; 7: 197-387