Effect of CAD on Radiologists' Detection of Lung Nodules on Thoracic CT Scans: Analysis of an Observer Performance Study by Nodule Size

Sahiner, Berkman; Chan, Heang-Ping; Hadjiiski, Lubomir M.; Cascade, Philip N.; Kazerooni, Ella A.; Chughtai, Aamer R.; Poopat, Chad; Song, Thomas; Frank, Luba; Stojanovska, Jadranka; Attili, Anil

doi:10.1016/j.acra.2009.08.006

« Previous Next »Academic Radiology
Volume 16, Issue 12 , Pages 1518-1530, December 2009

Effect of CAD on Radiologists' Detection of Lung Nodules on Thoracic CT Scans: Analysis of an Observer Performance Study by Nodule Size

Berkman Sahiner, PhD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
- Address correspondence to: B.S.
Heang-Ping Chan, PhD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Lubomir M. Hadjiiski, PhD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Philip N. Cascade, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Ella A. Kazerooni, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Aamer R. Chughtai, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Chad Poopat, MD
- Department of Radiology, Henry Ford Hospital, Detroit, MI
Thomas Song, MD
- Department of Radiology, Henry Ford Hospital, Detroit, MI
Luba Frank, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Jadranka Stojanovska, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109
Anil Attili, MD
- Department of Radiology, The University of Michigan, MIB C480A, 1500 East Medical Center Drive, Ann Arbor, MI 48109

Received 8 October 2008; accepted 10 August 2009.

Rationale and Objectives

To retrospectively investigate the effect of a computer-aided detection (CAD) system on radiologists' performance for detecting small pulmonary nodules in computed tomography (CT) examinations, with a panel of expert radiologists serving as the reference standard.

Materials and Methods

Institutional review board approval was obtained. Our dataset contained 52 CT examinations collected by the Lung Image Database Consortium, and 33 from our institution. All CTs were read by multiple expert thoracic radiologists to identify the reference standard for detection. Six other thoracic radiologists read the CT examinations first without and then with CAD. Performance was evaluated using free-response receiver operating characteristics (FROC) and the jackknife FROC analysis methods (JAFROC) for nodules above different diameter thresholds.

Results

A total of 241 nodules, ranging in size from 3.0 to 18.6 mm (mean, 5.3 mm) were identified as the reference standard. At diameter thresholds of 3, 4, 5, and 6 mm, the CAD system had a sensitivity of 54%, 64%, 68%, and 76%, respectively, with an average of 5.6 false positives (FPs) per scan. Without CAD, the average figures of merit (FOMs) for the six radiologists, obtained from JAFROC analysis, were 0.661, 0.729, 0.793, and 0.838 for the same nodule diameter thresholds, respectively. With CAD, the corresponding average FOMs improved to 0.705, 0.763, 0.810, and 0.862, respectively. The improvement achieved statistical significance for nodules at the 3 and 4 mm thresholds (P = .002 and .020, respectively), and did not achieve significance at 5 and 6 mm (P = .18 and .13, respectively). At a nodule diameter threshold of 3 mm, the radiologists' average sensitivity and FP rate were 0.56 and 0.67, respectively, without CAD, and 0.67 and 0.78 with CAD.

Conclusion

CAD improves thoracic radiologists' performance for detecting pulmonary nodules smaller than 5 mm on CT examinations, which are often overlooked by visual inspection alone.

Key Words: Lung Nodule, CT, computer-aided detection