Academic Radiology
Volume 17, Issue 3 , Pages 323-332 , March 2010

Computer-Aided Diagnosis of Lung Nodules on CT Scans: ROC Study of Its Effect on Radiologists' Performance

  • Ted Way, PhD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Heang-Ping Chan, PhD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • Corresponding Author InformationAddress correspondence to: H.-P.C.
    • ,
  • Lubomir Hadjiiski, PhD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Berkman Sahiner, PhD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Aamer Chughtai, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Thomas K. Song, MD

      Affiliations

    • Department of Radiology, Henry Ford Hospital, Detroit, MI
    • ,
  • Chad Poopat, MD

      Affiliations

    • Department of Radiology, Henry Ford Hospital, Detroit, MI
    • ,
  • Jadranka Stojanovska, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Luba Frank, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Anil Attili, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Naama Bogot, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Philip N. Cascade, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842
    • ,
  • Ella A. Kazerooni, MD

      Affiliations

    • Department of Radiology, University of Michigan, Med-Inn Building C477, 1500 E Medical Center Drive, Ann Arbor, MI 48109-5842

References 

  1. Sone S, Takashima S, Li F, et al. Mass screening for lung cancer with mobile spiral computed tomography scanner. Lancet. 1998;351:1242–1245
  2. Henschke CI, McCauley DI, Yankelevitz DF, et al. Early lung cancer action project: overall design and findings from baseline screening. Lancet. 1999;354:99–105
  3. Nawa T, Nakagawa T, Kusano S, et al. Lung cancer screening using low-dose spiral CT: results of baseline and 1-year follow-up studies. Chest. 2002;122:15–20
  4. Diederich S, Wormanns D, Semik M, et al. Screening for early lung cancer with low-dose spiral CT: Prevalence in 817 asymptomatic smokers. Radiology. 2002;222:773–781
  5. Henschke CI, Yankelevitz DF, Libby DM, et al. Survival of patients with stage I lung cancer detected on CT screening. N Engl J Med. 2006;355:1763–1771
  6. Sobue T, Moriyama N, Kaneko M, et al. Screening for lung cancer with low-dose helical computed tomography: anti-lung cancer association project. J Clin Oncol. 2002;20:911–920
  7. Gierada DS, Pilgram TK, Ford M, et al. Lung cancer: interobserver agreement on interpretation of pulmonary findings at low-dose CT screening. Radiology. 2008;246:265–272
  8. Swensen SJ, Jett JR, Hartman TE, et al. Lung cancer screening with CT: Mayo Clinic experience. Radiology. 2003;226:756–761
  9. Das M, Muhlenbruch G, Mahnken AH, et al. Small pulmonary nodules: Effect of two computer-aided detection systems on radiologist performance. Radiology. 2006;241:564–571
  10. Brown MS, Goldin JG, Rogers S, et al. Computer-aided lung nodule detection in CT results of large-scale observer test. Acad Radiol. 2005;12:681–686
  11. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology. 2004;230:347–352
  12. Armato S, Li F, Giger M, et al. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology. 2002;225:685–692
  13. Li F, Arimura H, Suzuki K, et al. Computer-aided detection of peripheral lung cancers missed at CT: ROC analyses without and with localization. Radiology. 2005;237:684–690
  14. Sahiner B, Hadjiiski LM, Chan HP, et al. The effect of nodule segmentation on the accuracy of computerized lung nodule detection on CT scans: comparison on a data set annotated by multiple radiologists. Proc SPIE. 2007;6514:65140L–65141L65147
  15. Way TW, Hadjiiski LM, Sahiner B, et al. Computer-aided diagnosis of pulmonary nodules on CT scans: segmentation and classification using 3D active contours. Med Phys. 2006;33:2323–2337
  16. Shah SK, McNitt-Gray MF, Rogers SR, et al. Computer-aided diagnosis of the solitary pulmonary nodule. Acad Radiol. 2005;12:570–575
  17. McNitt-Gray MF, Hart EM, Wyckoff N, Sayre JW, Goldin JG, Aberle DR. A pattern classification approach to characterizing solitary pulmonary nodules imaged on high resolution CT: preliminary results. Med Phys. 1999;26:880–888
  18. Armato SG, Altman MB, Wilkie J. Automated lung nodule classification following automated nodule detection on CT: a serial approach. Med Phys. 2003;30:1188–1197
  19. Kawata Y, Niki N, Ohmatsu H, et al. Quantitative surface characterization of pulmonary nodules based on thin-section CT images. IEEE Trans Nucl Sci. 1998;45:2132–2138
  20. Lo SCB, Hsu LY, Freedman MT, et al. Classification of lung nodules in diagnostic CT: an approach based on 3-D vascular features, nodule density distributions, and shape features. Proc SPIE. 2003;5032:183–189
  21. Li F, Aoyama M, Shiraishi J, et al. Radiologists' performance for differentiating benign from malignant lung nodules on high-resolution CT using computer-estimated likelihood of malignancy. AJR Am J Roentgenol. 2004;183:1209–1215
  22. Aoyama M, Li Q, Katsuragawa S, et al. Computerized scheme for determination of the likelihood measure of malignancy for pulmonary nodules on low-dose CT images. Med Phys. 2003;30:387–394
  23. Suzuki K, Li F, Sone S, Doi K. Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network. IEEE Trans Med Imaging. 2005;24:1138–1150
  24. Matsuki Y, Nakamura K, Watanabe H, et al. Usefulness of an artificial neural network for differentiating benign from malignant pulmonary nodules on high-resolution CT: evaluation with receiver operating characteristic analysis. AJR Am J Roentgenol. 2002;178:657–663
  25. Shah SK, McNitt-Gray MF, De Zoysa KR, et al. Solitary pulmonary nodule diagnosis on CT results of an observer study. Acad Radiol. 2005;12:496–501
  26. Li F, Li Q, Engelmann R, et al. Improving radiologists' recommendations with computer-aided diagnosis for management of small nodules detected by CT. Acad Radiol. 2006;13:943–950
  27. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules: Estimation of malignancy at thin-section helical CT—effect of computer-aided diagnosis on performance of radiologists. Radiology. 2006;239:276–284
  28. Way TW, Sahiner B, Chan H-P, et al. Computer aided diagnosis of pulmonary nodules on CT scans: improvement of classification performance with nodule surface features. Med Phys. 2009;36:3086–3098
  29. Sahiner B, Chan HP, Petrick N, et al. Computerized characterization of masses on mammograms: the rubber band straightening transform and texture analysis. Med Phys. 1998;25:516–526
  30. Galloway MM. Texture classification using gray level run lengths. Comput Graph Image Process. 1975;4:172–179
  31. Dasarathy BR, Holder EB. Image characterizations based on joint gray-level run-length distributions. Patt Recog Lett. 1991;12:497–502
  32. Ge Z, Sahiner B, Chan HP, et al. Computer aided detection of lung nodules: false positive reduction using a 3D gradient field method and 3D ellipsoid fitting. Med Phys. 2005;32:2443–2454
  33. Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol. 1989;24:234–245
  34. Dorfman DD, Berbaum KS, Metz CE. ROC rating analysis: generalization to the population of readers and cases with the jackknife method. Invest Radiol. 1992;27:723–731
  35. Jiang Y, Metz CE, Nishikawa RM. A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology. 1996;201:745–750
  36. Sahiner B, Chan HP, Petrick N, et al. Design of a high-sensitivity classifier based on a genetic algorithm: application to computer-aided diagnosis. Phys Med Biol. 1998;43:2853–2871
  37. Obuchowski N. Nonparametric analysis of clustered ROC curve data. Biometrics. 1997;53:567–578
  38. Lee M-LT, Rosner BA. The average area under correlated receiver operating characteristic curves: a nonparametric approach based on generalized two-sample Wilcoxon statistics. J R Stat Soc Ser C Appl Stat. 2001;50:337–344
  39. Hadjiiski LM, Chan HP, Sahiner B, et al. Breast masses: computer-aided diagnosis with serial mammograms. Radiology. 2006;240:343–356
  40. Li Q, Li F, Shiraishi J, Katsuragawa S, et al. Investigation of new psychophysical measures for evaluation of similar images on thoracic computed tomography for distinction between benign and malignant nodules. Med Phys. 2003;30:2584–2593
  41. Gurney JW. Determining the likelihood of malignancy in solitary pulmonary nodules with Bayesian analysis—part I. Theory. Radiology. 1993;186:405–413
  42. Bogot NR, Kazerooni EA, Kelly AM, et al. Interobserver and intraobserver variability in the assessment of pulmonary nodule size on CT using film and computer display methods. Acad Radiol. 2005;12:948–956
  43. Meyer CR, Johnson TD, McLennan G, et al. Evaluation of lung MDCT nodule annotation across radiologists and methods. Acad Radiol. 2006;13:1254–1265
  44. Jiang Y, Nishikawa RM, Schmidt RA, et al. Potential of computer-aided diagnosis to reduce variability in radiologists' interpretations of mammograms depicting microcalcifications. Radiology. 2001;220:787–794
  45. Gur D, Bandos AI, Cohen CS, et al. The “laboratory” effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations. Radiology. 2008;249:47–53

 This work was supported by grant CA93517 from the US Public Health Service (Rockville, MD). Received July 2, 2009; accepted October 2, 2009.

PII: S1076-6332(09)00588-1

doi: 10.1016/j.acra.2009.10.016

Academic Radiology
Volume 17, Issue 3 , Pages 323-332 , March 2010

Article Tools

* Image Usage & Resolution