Advertisement

Evaluation of Significant Coronary Artery Disease Based on CT Fractional Flow Reserve and Plaque Characteristics Using Random Forest Analysis in Machine Learning

Open AccessPublished:February 10, 2020DOI:https://doi.org/10.1016/j.acra.2019.12.013

      Rationale and Objectives

      Fractional flow reserve (FFR) is an established technique for detecting lesion-specific ischemia but is invasive. Our objective was to investigate the effects of combined assessment of coronary CT angiography (CCTA) imaging features and CT-FFR on detecting lesion-specific ischemia by comparing with invasive FFR.

      Materials and Methods

      Forty-seven patients who had 60 coronary vessels with 30%–90% stenosis were included. Six anatomic CCTA descriptors (Agatston score, stenosis severity, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, remodeling index) and a functional descriptor (CT-FFR) were measured. Random forest was used to identify which descriptors were useful to identify ischemia-related lesion. Receiver-operating characteristic (ROC) curves were calculated for 2 models: i.e. Model-1 for anatomical CT descriptors and Model-2 for anatomical CT descriptors plus CT-FFR.

      Results

      Stenosis severity (40.8 ± 15.7% vs 57.6 ± 14.1%), noncalcified plaque volume (190 ± 100 vs 254.8 ± 133.3), and remodeling index (1.04 ± 0.12 vs 1.11 ± 0.13) were significantly higher in ischemia-related lesions than nonischemia-related lesions. CT-FFR was 0.84 ± 0.14 and 0.71 ± 0.14, respectively, for ischemia-related and nonischemia-related lesions, and the difference was significant. The area under the ROC curve was 0.738 and 0.835 in Model-1 and Model-2, respectively. Reclassification of ischemic lesion risk was significantly improved after adding CT-FFR: net reclassification improvement was 0.297 and integrated discrimination improvement was 0.254.

      Conclusion

      Combined assessment of anatomical CCTA features and functional CT-FFR was helpful for detecting lesion-specific ischemia.

      Key Words

      Abbreviations:

      AUC (area under the receiver-operating characteristic curve), CAD (coronary artery disease), CI (confidence interval), CT (computed tomography), CCTA (coronary CT angiography), FFR (fractional flow reserve), ROC (receiver-operating characteristic)

      INTRODUCTION

      Coronary artery disease (CAD) was still the leading cause of death globally in 2015 (
      • Nowbar A.N.
      • Gitto M.
      • Howard J.P.
      • et al.
      Mortality from ischemic heart disease.
      ), and it is important to improve the prognosis in patients with CAD by appropriate diagnosis and treatment. Coronary computed tomography angiography (CCTA) is an established minimally invasive diagnostic technique and has high diagnostic accuracy for the evaluation of CAD (
      • Budoff M.J.
      • Dowe D.
      • Jollis J.G.
      • et al.
      Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial.
      ). In 2016, the National Institute for Health and Care Excellence in the UK endorsed CCTA as the first-line investigation for patients with angina and no history of CAD (
      National Institute for Health and Care Excellence
      Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin (update): clinical guideline 95.
      ). However, the functional assessment of lesion-specific ischemia is necessary to determine the appropriate treatment strategy in the patients with CAD, and anatomical stenosis on CCTA is not equal to the lesion-specific ischemia (
      • Meijboom W.B.
      • Van Mieghem C.A.
      • van Pelt N.
      • et al.
      Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina.
      ). Therefore, additional functional tests are recommended in patients with anatomically significant stenosis (≥50 %) on CCTA to identify the lesion-specific ischemia (
      • Cury R.C.
      • Abbara S.
      • Achenbach S.
      • et al.
      CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
      ).
      Fractional flow reserve (FFR), which is measured by a coronary pressure wire during catheterization, is considered the reference standard for assessment of lesion-specific ischemia (
      • Tonino P.A.
      • De Bruyne B.
      • Pijls N.H.
      • et al.
      Fractional flow reserve versus angiography for guiding percutaneous coronary intervention.
      ,
      • De Bruyne B.
      • Fearon W.F.
      • Pijls N.H.
      • et al.
      Fractional flow reserve-guided PCI for stable coronary artery disease.
      ,
      • Pijls N.H.
      • van Schaardenburgh P.
      • Manoharan G.
      • et al.
      Percutaneous coronary intervention of functionally nonsignificant stenosis: 5-year follow-up of the DEFER Study.
      ), while, it is an invasive time-consuming procedure that has a risk of complications. The recent advent of noninvasive FFR derived from CT (CT-FFR) has enabled both anatomical and functional evaluation of CAD and increased our ability to detect hemodynamically significant stenosis (
      • Norgaard B.L.
      • Leipsic J.
      • Gaur S.
      • et al.
      Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps).
      ,
      • Min J.K.
      • Leipsic J.
      • Pencina M.J.
      • et al.
      Diagnostic accuracy of fractional flow reserve from anatomic CT angiography.
      ,
      • Ko B.S.
      • Cameron J.D.
      • Munnur R.K.
      • et al.
      Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
      ,
      • Fujimoto S.
      • Kawasaki T.
      • Kumamaru K.K.
      • et al.
      Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio.
      ). However, a recent systematic review revealed wide limits of agreement between invasive FFR and CT-FFR, especially for CT-FFR values of 0.7–0.8, suggesting that the clinical ‘gray zone’ was larger for CT-FFR than for invasive FFR (
      • Cook C.M.
      • Petraco R.
      • Shun-Shin M.J.
      • et al.
      Diagnostic accuracy of computed tomography-derived fractional flow reserve: a systematic review.
      ,
      • Villines T.C.
      Can CT-derived FFR better inform clinical decision-making and improve outcomes in stable ischaemic heart disease?.
      ). Therefore, CT-FFR alone may not be enough for completely identifying lesion-specific ischemia, and additional information is needed to improve the diagnostic accuracy such as the characteristics and volume of coronary plaque on CCTA (
      • Gaur S.
      • Ovrehus K.A.
      • Dey D.
      • et al.
      Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions.
      ,
      • Ovrehus K.A.
      • Gaur S.
      • Leipsic J.
      • et al.
      CT-based total vessel plaque analyses improves prediction of hemodynamic significance lesions as assessed by fractional flow reserve in patients with stable angina pectoris.
      ). The purpose of this study was to investigate the diagnostic accuracy of comprehensive assessment of anatomical and functional indices derived from CCTA for detecting lesion-specific ischemia assessed by invasive FFR.

      MATERIALS AND METHODS

      The study protocol was approved by Ethics Committee of XXXX. The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data.

       Patient Population

      We retrospectively examined a total of 47 consecutive patients (36 men, 11 women; mean age ± standard deviation, 69.4 ± 11.2 years, range = 47–87 years) between May 2016 and March 2018. All patients firstly underwent CCTA/CT-FFR, followed by invasive FFR. Each patient had 30%–90% stenosis of ≥1 major epicardial vessel on CCTA and underwent invasive measurement of FFR in that vessel after CCTA between May 2016 and March 2018. The interval between CCTA and invasive FFR measurement was 23.7 ± 14.3 days. The invasive FFR was ≥0.8 (nonischemic) in 25 of the 60 vessels and <0.8 (ischemic) in 35.

       Acquisition of CCTA Data

      All patients underwent CCTA examination using a 320-row detector CT scanner (Aquilion One Vision or Aquilion One Genesis; Canon Medical Systems, Otawara, Japan). The CT protocol consisted of obtaining the calcium score followed by CCTA. The parameters used for CT scanning were as follows: prospective electrocardiogram-gatedaxial scans; 320 rows × 0.5-mm collimation; rotation time, 275 ms; tube voltage, 120 kV; and tube current, 350–650 mA (automatic exposure control). All patients underwent prospective ECG-gated CCTA with data acquisition over 70%–99% of the R-R interval. All patients received sublingual nitroglycerin. If the heart rate exceeded 65 bpm in the CT suite, 6–10 mg landiolol hydrochloride (Corebeta; Ono Pharmaceutical Co. Ltd., Osaka, Japan) was administered intravenously as an additional beta-blocker 5 minutes before CCTA scanning. For CCTA, contrast material with an iodine concentration of 370 mg/mL (Iopamiron 370; Bayer Health Care, Osaka, Japan) was administered via a 20-gauge catheter inserted into an antecubital vein using a double-head power injector. The start time of data acquisition was determined by a computer-assisted bolus-tracking program. The trigger threshold was set at 150 HU for the region of interest in the ascending aorta. Acquisition of the CT data was started 6 seconds after the trigger.

       CCTA Analysis

      The CCTA data were analyzed using a dedicated workstation (Vitrea, Canon Medical Systems) by consensus between two experienced cardiovascular radiologists in the core CCTA laboratory at Shin Koga Hospital. Both radiologists were blinded to the results of invasive coronary angiography and FFR. One target lesion per vessel was assessed when invasive FFR was performed. The following anatomical CTA parameters were measured: the Agatston score of the vessel containing the target lesion, severity of stenosis, mean plaque CT attenuation value (HU), noncalcified and calcified plaque volumes, and remodeling index. The diameters of the nonaffected vessel segments immediately proximal and distal to the lesion were measured as the reference for determination of the diameter of a stenosis; the degree of stenosis was calculated as the diameter of a lesion of interest divided by the reference diameter. Coronary plaque volume and morphology were assessed by the labeling method using dedicated software (CCTA Analysis, Canon Medical Systems) (
      • Fujimoto S.
      • Kondo T.
      • Kodama T.
      • et al.
      A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography.
      ).

       On-site CT-FFR Analysis

      The on-site CT-FFR analysis (CT-FFR Research Version, Canon Medical Systems) was performed by consensus between an experienced postprocessing technician and a cardiovascular radiologist using the dedicated Vitrea workstation, who were blinded to all information of the clinical data, CCTA, invasive coronary angiography, and FFR measurement. The CT-FFR analysis included four-dimensional CT image tracking and structural and fluid analysis by considering the deformation of vessels, with four-phase CT image series at 70%, 80%, 90%, and 99% of the R-R interval. At first, the vessel contour and the centerline of the coronary vessel were semi-automatically extracted on the workstation. Two operators consensually edit the vessel contour and centerline especially in the calcified segments. After the edit, the CT-FFR value was estimated using the following steps: (1) measurement of the shape and changes in the cross-sectional area of the coronary artery and aorta in four volumes reconstructed at different time-points during 70%–99% of the R-R interval extraction of luminal deformation data for the coronary arteries and aorta; (2) application of Bayesian hierarchical modeling and the Markov chain Monte Carlo method to determine the analysis conditions; (3) fluid simulation using a reduced order fluid model; and (4) determination of pressure and flow through the coronary artery tree and estimation of the CT-FFR value (
      • Ko B.S.
      • Cameron J.D.
      • Munnur R.K.
      • et al.
      Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
      ,
      • Fujimoto S.
      • Kawasaki T.
      • Kumamaru K.K.
      • et al.
      Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio.
      ). In the process of CT-FFR analysis, the machine-learning technique was not employed. The CT-FFR value was measured at the distal end of the coronary vessel where the diameter was 2 mm according to the previous study (
      • Ko B.S.
      • Cameron J.D.
      • Munnur R.K.
      • et al.
      Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
      ), in which the on-site CT-FFR (Canon Medical Systems) was used. The time for edit of coronary arterial contour and centerline was 30–60 minutes, and the calculation time for CT-FFR was 20–30 minutes.

       Machine Learning and Statistical Analysis

      In this investigation, we used the random forest, which is one of the most common machine learning algorithms because of its relatively good accuracy, robustness, and ease of use, and built two learning models: Model-1 (including anatomical CTA parameters, i.e., Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, and remodeling index) and Model-2 (the same anatomical CTA parameters plus the CT-FFR value). The anatomical CTA parameters were transferred to the random forest model as numbers. The forest consisted of many decision trees, where each tree was trained using randomly selected samples. Two thirds of the training samples were used to train the trees, and the remaining third was used for cross-validation to estimate how well the resulting random forest models performed. In the random forest algorithm, importance is calculated with the Gini importance index to calculate the value of each feature, and we ranked the importance of all descriptors in Model-2 to separate the patients into ischemic and nonischemic groups based on Gini importance (
      • Menze B.H.
      • Kelm B.M.
      • Masuch R.
      • et al.
      A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data.
      ). The machine learning analyses were performed using the Python-based software program (Orange version 3.23.1; https://orange.biolab.si). We adopted the random forest to evaluate the added value of CT-FFR compared to CCTA descriptors alone in this study. The comparison among the logistic regression, support vector machine, and random forest is shown in the Appendix Figure.
      The data are shown as the mean ± standard deviation or as the median and interquartile range depending on the data distribution. The Agatston score, severity of stenosis, mean plaque CT attenuation value, noncalcified and calcified plaque volumes, remodeling index, and CT-FFR values were compared between the nonischemic and ischemic groups using the Student's t-test. The area under the receiver-operating characteristic (ROC) curve (AUC) was calculated for Models-1 and -2 to differentiate between nonischemic and ischemic lesions. The AUCs for Models-1 and -2 were compared using Delong's test. The net reclassification improvement and integrated discrimination improvement values were calculated to assess the added value of CT-FFR for distinguishing between ischemic and nonischemic lesions. Two-tailed p-values < 0.05 were considered statistically significant. The statistical analyses were performed using open-source statistical software (Microsoft R Open, version 3.4.2; https://mran.microsoft.com/open).

      RESULTS

      The results of the univariate analysis are summarized in Table 1. The severity of stenosis, noncalcified plaque volume, remodeling index, and CT-FFR value were significantly different between the ischemic and nonischemic groups.
      Table 1Basic Characteristics of the Study Population According to Whether Ischemia was Present
      No Ischemia (n = 25)Ischemia (n = 35)p Value
      Agatston score37 (1.5–148.5)105 (9–258)0.59
      Stenosis severity40.8 ± 15.757.6 ± 14.1<0.01
      Plaque CT attenuation value70.1 ± 58.555.1 ± 27.90.19
      Plaque volume (noncalcified)167 (119–256)223 (177–295.7)<0.05
      Plaque volume (calcified)4.1 (0–25)23 (4.7–57.8)0.20
      Remodeling index1.04 ± 0.121.11 ± 0.13<0.05
      CT-FFR value0.84 ± 0.140.71 ± 0.14<0.01
      CT, computed tomography; FFR, fractional flow reserve.
      Model-1 (anatomical CTA parameters) and Model-2 (the same anatomical parameters plus CT-FFR) are compared in Figure 1. The AUC was 0.738 (95% confidence interval [CI] 0.609–0.843) for Model-1 and 0.835 (95% CI 0.717–0.919) for Model-2. The reclassification of the risk of the ischemia-related lesion was significantly improved after adding CT-FFR (Model-2) to the anatomical parameters (Model-1). The net reclassification improvement was 0.297 (95% CI 0.031–0.564; p = 0.03) and the integrated discrimination improvement was 0.254 (95% CI 0.127–0.382; p < 0.001).
      Figure 1
      Figure 1Receiver-operating characteristic curve analysis of machine learning (random forest) for Model 1 (anatomical CTA parameters) and Model 2 (anatomical CTA parameters plus CT-FFR). CT, computed tomography; CTA, CT angiography; CT-FFR, CT-derived fractional flow reserve.
      Figure 2 shows the Gini importance index of the descriptors in the random forest. The parameter best able to differentiate an ischemic lesion from a nonischemic lesion was the CT-FFR value (0.496) followed by the severity of stenosis (0.262), remodeling index (0.115), calcified plaque volume (0.048), mean plaque CT attenuation value (0.046), noncalcified plaque volume (0.033), and Agatston score (<0.001). Representative cases are shown in Figures 3,4 and Appendix Figure.
      Figure 2
      Figure 2Gini importance indices of seven descriptors for identification of ischemic lesions calculated using a random forest. CT-FFR was most important, followed by severity of stenosis and the remodeling index. CT-FFR, CT-derived fractional flow reserve.
      Figure 3
      Figure 3Coronary CT angiogram (A) of a 70s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). On the curved multiplanar reformation image in the CT-FFR display (B), the operators defined the measuring site (arrowheads [2 mm in vessel diameter]) distal to the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.72, 80%, and 1.11, respectively. Machine learning (random forest) classified the coronary vessel as “positive” (ischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.72. The labeling method identifies areas of low attenuation by comparing Hounsfield Unit measurements relative to surrounding areas, while differentiating low attenuation areas from image noise. Then, the algorithm excludes areas within the lesion with a low probability of clinically significant morphology. Residual low attenuation areas are considered as necrotic core, and remaining components not classified into either low-density areas or calcium are considered fibrous tissue. The labeling method can correctly identify the necrotic core area (red), calcified plaque (yellow), fibrous area (blue) and lumen area (green) (D). FFR, fractional flow reserve.
      Figure 4
      Figure 4Coronary CT angiogram (A) of a 60s woman shows partially calcified plaque in the proximal left anterior descending artery with intermediate stenosis (arrow). The CT-FFR display (B) shows the stenosis (arrow). Her CT-FFR, stenosis severity, and remodeling index values were 0.84, 60%, and 1.00. respectively. Machine learning (random forest) classified the coronary vessel as “negative” (nonischemic lesion). Coronary angiogram (C) shows moderate stenosis (arrow) with invasive FFR of 0.89. FFR, fractional flow reserve.

      DISCUSSION

      The two main findings of this study were that (1) comprehensive assessment of multiple CT-derived indices, including severity of stenosis, plaque density, plaque volume, coronary artery calcium score, remodeling index, and CT-FFR based on a machine learning approach showed high diagnostic accuracy for detecting hemodynamically significant CAD, and (2) the CT-FFR had a more important incremental diagnostic value than the conventional anatomical indices. The anatomical severity of coronary artery stenosis is not necessarily equal to the hemodynamical severity of CAD. Meijboom et al showed that there was agreement between significant stenosis (a reduction in diameter of ≥50%) on quantitative coronary angiography and positive FFR (<0.80) on only 65% of occasions (
      • Meijboom W.B.
      • Van Mieghem C.A.
      • van Pelt N.
      • et al.
      Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina.
      ). Hence, in addition to coronary stenosis severity, other indices have been necessary for accurate prediction of hemodynamically significant CAD. The coronary artery calcium score is known to be associated with the severity of CAD (
      • Budoff M.J.
      • Young R.
      • Burke G.
      • et al.
      Ten-year association of coronary artery calcium with atherosclerotic cardiovascular disease (ASCVD) events: the multi-ethnic study of atherosclerosis (MESA).
      ). The characteristics of coronary artery plaque seen on CCTA are also considered indicators of hemodynamically significant CAD (
      • Gaur S.
      • Ovrehus K.A.
      • Dey D.
      • et al.
      Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions.
      ,
      • Ovrehus K.A.
      • Gaur S.
      • Leipsic J.
      • et al.
      CT-based total vessel plaque analyses improves prediction of hemodynamic significance lesions as assessed by fractional flow reserve in patients with stable angina pectoris.
      ). Indeed, the anatomical index of coronary artery plaque remodeling was important for the detection of hemodynamically significant CAD in the present study. However, plaque CT number and Agatston score had relatively small influence. In this study, we used the labeling method for the plaque characterization and quantification. Previous ex vivo and in vivo studies have reported that an opacified coronary lumen substantially increases the plaque attenuation because of partial volume effects, beam hardening, and plaque vascularity (
      • Cademartiri F.
      • Mollet N.R.
      • Runza G.
      • et al.
      Influence of intracoronary attenuation on coronary plaque measurements using multislice computed tomography: observations in an ex vivo model of coronary computed tomography angiography.
      ,
      • Kidoh M.
      • Utsunomiya D.
      • Oda S.
      • et al.
      Evaluation of the effect of intracoronary attenuation on coronary plaque measurements using a dual-phase coronary CT angiography technique on a 320-row CT Scanner–in vivo validation study.
      ). Therefore, a method for accurate characterization and volumetric quantification of coronary plaques was required. The labeling method applied in the present study was developed for 320-row CT coronary angiography to overcome the limitations of CT attenuation value-based measurements and is a novel way of assessing coronary artery plaque (
      • Fujimoto S.
      • Kondo T.
      • Kodama T.
      • et al.
      A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography.
      ). This method considers relative (as opposed to absolute) CT attenuation in addition to image noise and the three-dimensional continuity of volumes and classifies plaque components into fibrous tissue, necrotic core, and calcified areas. A previous study that compared this labeling method with CT attenuation value-based measurement of non-calcified plaques demonstrated superior correlation with virtual histology intravascular ultrasound as the reference standard. This is because of the decreased influence of the various factors associated with contrast material and scanning protocols (
      • Fujimoto S.
      • Kondo T.
      • Kodama T.
      • et al.
      A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography.
      ). We considered the labeling method to be appropriate for the assessment of coronary plaque volume and morphology. Our results showed that the remodeling was a more important factor than plaque characteristics, i.e., CT number and Agatston score. We consider that the focal accumulation of plaque, which caused higher remodeling indices and stenosis severity, was associated with hemodynamically CAD (invasive FFR < 0.8). Furthermore, our study results demonstrated that added index of CT-FFR value increased the diagnostic performance.
      The diagnostic accuracy of anatomical indices measured on CCTA was moderate (AUC = 0.738 for Model-1) for detecting hemodynamically significant CAD in spite of comprehensive assessment of multiple indices. However, the functional index of CT-FFR increased diagnostic value over the anatomical indices measured on CCTA (AUC = 0.835 for Model-2). CT-FFR has been reported to have high diagnostic accuracy for detecting hemodynamically significant CAD assessed by invasive FFR (
      • Celeng C.
      • Leiner T.
      • Maurovich-Horvat P.
      • et al.
      Anatomical and functional computed tomography for diagnosing hemodynamically significant coronary artery disease: a meta-analysis.
      ) and to be an effective gatekeeper for invasive coronary angiography in clinical settings (
      • Douglas P.S.
      • Pontone G.
      • Hlatky M.A.
      • et al.
      Clinical outcomes of fractional flow reserve by computed tomographic angiography-guided diagnostic strategies vs. usual care in patients with suspected coronary artery disease: the prospective longitudinal trial of FFR(CT): outcome and resource impacts study.
      ). The on-site CT-FFR used in the present study relied on fluid-structure interaction for calculation of patient-specific CT-FFR (
      • Ko B.S.
      • Cameron J.D.
      • Munnur R.K.
      • et al.
      Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
      ), and a previous study indicated a close correlation between CT-FFR and invasive FFR that was highly reproducible (
      • Fujimoto S.
      • Kawasaki T.
      • Kumamaru K.K.
      • et al.
      Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio.
      ). As a result, it was reasonable that the functional index of CT-FFR was more important than other anatomical indices, including anatomical stenosis, plaque characteristics, and coronary artery calcification. However, the diagnostic accuracy of CT-FFR varies markedly across the spectrum of CAD severity, and is particularly poor at invasive FFR values of 0.7 to 0.8, suggesting a larger clinical ‘grey zone’ for CT-FFR compared to invasive FFR (
      • Cook C.M.
      • Petraco R.
      • Shun-Shin M.J.
      • et al.
      Diagnostic accuracy of computed tomography-derived fractional flow reserve: a systematic review.
      ). We believe that our results indicate that comprehensive assessment of multiple indices using the machine learning approach (random forest) has the potential to overcome the limitations of assessment based on a single index. Additionally, the anatomical and functional CT indices might play complementary roles. We did not compare the diagnostic accuracy of on-site CT-FFR with that of off-site CT-FFR developed by HeartFlow (Redwood, CA) (
      • Pijls N.H.
      • van Schaardenburgh P.
      • Manoharan G.
      • et al.
      Percutaneous coronary intervention of functionally nonsignificant stenosis: 5-year follow-up of the DEFER Study.
      ,
      • Norgaard B.L.
      • Leipsic J.
      • Gaur S.
      • et al.
      Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps).
      ,
      • Min J.K.
      • Leipsic J.
      • Pencina M.J.
      • et al.
      Diagnostic accuracy of fractional flow reserve from anatomic CT angiography.
      ). However, a single-center prospective study by Ko et al reported comparative results of on-site CT-FFR in detecting invasive FFR <0.8 (
      • Ko B.S.
      • Cameron J.D.
      • Munnur R.K.
      • et al.
      Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
      ). Although further validation is required in a larger prospective study, our present findings demonstrate that machine learning analysis of anatomical CCTA indices and CT-FFR has the potential to allow for accurate evaluation of lesion-specific ischemia in a more cost-effective manner.
      Machine learning is a technique which has ability to learn using artificial intelligence (AI). Machine learning has been applied for cardiovascular disease in the fields of image interpretation, diagnostic support and outcome prediction (
      • Dey D.
      • Slomka P.J.
      • Leeson P.
      • et al.
      Artificial intelligence in cardiovascular imaging: JACC State-of-the-Art Review.
      ). Ruijsink et al reported that machine learning could automatically analyze of cardiac function from cine cardiovascular magnetic resonance (
      • Ruijsink B.
      • Puyol-Anton E.
      • Oksuz I.
      • et al.
      Fully automated, quality-controlled cardiac analysis from CMR: validation and large-scale application to characterize cardiac function.
      ). Zheng et al reported that machine learning could automatically perform aortic valve segmentation intraoperatively in real time from images taken using the C-arm during Transcatheter Aortic Valve Implantation (
      • Zheng Y.
      • John M.
      • Liao R.
      • et al.
      Automatic aorta segmentation and valve landmark detection in C-arm CT for transcatheter aortic valve implantation.
      ). Motwani et al reported that machine learning combining clinical and CCTA data significantly improved prediction to 5-year all-cause mortality compared with existing clinical or CCTA alone (
      • Motwani M.
      • Dey D.
      • Berman D.S.
      • et al.
      Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis.
      ). In future, machine learning may be integrated on existing information technology infrastructure such as medical records, reporting system, and picture archiving and communication systems, and it will allow for making clinical routine more efficient. However, physician should have knowledge about the machine learning tools to determine the appropriate diagnosis and treatment strategy.
      Random forest is an accurate, easily interpretable, and computationally efficient algorithm. Random forest is composed of a collection of many decision trees. Each decision tree in the classifier is trained using a subset of the various input variables with two thirds of the original data. The remaining one third is used to generate the out-of-bag error, which is an internal validation of the final model. The outcome of a random forest model is determined by the majority vote. Random forest works on the hypothesis that an aggregation of correctly predicted classes from a large ensemble of randomly generated individual decision trees achieves higher classification accuracy (
      • Ozigis M.S.
      • Kaduk J.D.
      • Jarvis C.H.
      • et al.
      Detection of oil pollution impacts on vegetation using multifrequency SAR, multispectral images with fuzzy forest and random forest methods.
      ). Random forest is considered less vulnerable to overfitting the training dataset given the large number of trees built, making each tree an independent model. Random forest requires little preprocessing of data; the data need not be normalised; and the approach is resilient to outliers. Random forest can select the most important features when creating decision trees. In this study, application of the random forest-based machine learning approach showed that the three most important indices for identification of ischemic lesions were the CT-FFR, severity of stenosis, and remodeling index. We could not compare the diagnostic accuracy of CT-FFR alone with that of combined assessment of anatomical CTA plus CT-FFR (Model-2) because we had used a random forest algorithm that evaluates multiple indices. However, we believe that the combined assessment (Model-2) is superior to CT-FFR alone because the stenosis severity and the remodeling index had relatively large influence on the differentiation between ischemia and nonischemia lesion (Fig 2). We did not use a deep learning approach in this study. Deep learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain, which are known as artificial neural networks. Deep learning has attracted the attention of the academic medical community in recent years because of its performance (
      • Chartrand G.
      • Cheng P.M.
      • Vorontsov E.
      • et al.
      Deep learning: a primer for radiologists.
      ). It has also shown promising results with regard to segmentation of cardiac structures and identification of CAD (
      • van Hamersvelt R.W.
      • Zreik M.
      • Voskuil M.
      • et al.
      Deep learning analysis of left ventricular myocardium in CT angiographic intermediate-degree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis.
      ). Although deep learning generates features automatically, it requires large amounts of labeled data. As larger amounts of data are added, the performance of deep learning continues to increase. However, the lack of high numbers of samples makes training difficult and causes over-fitting, resulting in decreased accuracy. Therefore, traditional machine learning performs better than deep learning when sample sizes are small (
      • Tang A.
      • Tam R.
      • Cadrin-Chenevert A.
      • et al.
      Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology.
      ). In this study, we could not collect large amounts of clinical data and apply traditional machine learning because on-site computed CT-FFR based on fluid-structure interactions is a work-in-progress algorithm and is not readily accessible by most research institutions. A multicenter study of deep learning using large-scale cardiac CT data should be performed in the future.
      This study had some limitations. First, it had a retrospective observational design and a small sample size; therefore, it is not necessarily representative of the general population. We adopted the random forest because of the tradeoff between the number of variables and population size. Although we applied cross-validation in the analysis, it would have been preferable to include training, validation, and test datasets. A prospective study is needed to establish the feasibility of comprehensive assessment of multiple CT-derived indices using random forests. Second, we excluded vessels with tandem lesions to make it easier to assess the effects of each index. Third, the accuracy of CT-FFR depends on the experience of the observer and the image quality. We did not evaluate the influence of CT image quality on the CT-FFR measurements. To our experience, it is considered that CT-FFR value may be affected by the image quality factors, e.g. contrast enhancement, motion artifacts, vessel calcification because these factors make it difficult to delineate the contour of coronary vessels in the process of CT-FFR measurement. The machine-learning or deep-learning algorithm should be necessary to perform fully automatic CT-FFR measurement to improve the accuracy and to reduce inter-operator variability. However, a previous study reported that automated CT-FFR measurement software using structural and fluid analysis displayed good reproducibility even when the post-processing was performed by an inexperienced observer given only brief training (
      • Ri K.
      • Kumamaru K.K.
      • Fujimoto S.
      • et al.
      Noninvasive computed tomography-derived fractional flow reserve based on structural and fluid analysis: reproducibility of on-site determination by unexperienced observers.
      ). Lastly, we did not compare the CCTA plus CT-FFR with other gold standard functional imaging i.e. myocardial perfusion positron emission tomography due to unavailability in our institutions.
      In conclusion, comprehensive assessment of anatomical and functional indices derived from CCTA data based on machine learning (i.e. random forests) provided higher diagnostic accuracy for detecting hemodynamically significant CAD than the anatomical CT indices alone. A functional index, the CT-FFR, was more useful than conventional anatomical indices.

      Funding

      This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

      Unblinded Ethics Statement

      The study protocol was approved by Ethics Committee of Kumamoto Medical College (approval number #1294). The need for informed consent was waived in view of the retrospective nature of the analysis and the anonymity of the data.

      Appendix

      Figure 5
      Appendix FigureThe comparison among the logistic regression, support vector machine, and random forest in the ROC analysis. The AUC to differentiate between nonischemic and ischemic lesions was 0.698, 0.827, and 0.835 for the logistic regression, support vector machine, and random forest, respectively. Note: LR, logistic regression; RF, random forest; SVM, support vector machine.

      REFERENCES

        • Nowbar A.N.
        • Gitto M.
        • Howard J.P.
        • et al.
        Mortality from ischemic heart disease.
        Circ Cardiovasc Qual Outcomes. 2019; 12e005375
        • Budoff M.J.
        • Dowe D.
        • Jollis J.G.
        • et al.
        Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial.
        J Am Coll Cardiol. 2008; 52: 1724-1732
        • National Institute for Health and Care Excellence
        Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin (update): clinical guideline 95.
        National Institute for Health and Care Excellence, London, England2016
        • Meijboom W.B.
        • Van Mieghem C.A.
        • van Pelt N.
        • et al.
        Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional flow reserve in patients with stable angina.
        J Am Coll Cardiol. 2008; 52: 636-643
        • Cury R.C.
        • Abbara S.
        • Achenbach S.
        • et al.
        CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
        J Cardiovasc Comput Tomogr. 2016; 10: 269-281
        • Tonino P.A.
        • De Bruyne B.
        • Pijls N.H.
        • et al.
        Fractional flow reserve versus angiography for guiding percutaneous coronary intervention.
        N Engl J Med. 2009; 360: 213-224
        • De Bruyne B.
        • Fearon W.F.
        • Pijls N.H.
        • et al.
        Fractional flow reserve-guided PCI for stable coronary artery disease.
        N Engl J Med. 2014; 371: 1208-1217
        • Pijls N.H.
        • van Schaardenburgh P.
        • Manoharan G.
        • et al.
        Percutaneous coronary intervention of functionally nonsignificant stenosis: 5-year follow-up of the DEFER Study.
        J Am Coll Cardiol. 2007; 49: 2105-2111
        • Norgaard B.L.
        • Leipsic J.
        • Gaur S.
        • et al.
        Diagnostic performance of noninvasive fractional flow reserve derived from coronary computed tomography angiography in suspected coronary artery disease: the NXT trial (Analysis of Coronary Blood Flow Using CT Angiography: Next Steps).
        J Am Coll Cardiol. 2014; 63: 1145-1155
        • Min J.K.
        • Leipsic J.
        • Pencina M.J.
        • et al.
        Diagnostic accuracy of fractional flow reserve from anatomic CT angiography.
        JAMA. 2012; 308: 1237-1245
        • Ko B.S.
        • Cameron J.D.
        • Munnur R.K.
        • et al.
        Noninvasive CT-derived FFR based on Structural and fluid analysis: a comparison with invasive FFR for detection of functionally significant stenosis.
        JACC Cardiovasc Imaging. 2017; 10: 663-673
        • Fujimoto S.
        • Kawasaki T.
        • Kumamaru K.K.
        • et al.
        Diagnostic performance of on-site computed CT-fractional flow reserve based on fluid structure interactions: comparison with invasive fractional flow reserve and instantaneous wave-free ratio.
        Eur Heart J Cardiovasc Imaging. 2019; 20: 343-352
        • Cook C.M.
        • Petraco R.
        • Shun-Shin M.J.
        • et al.
        Diagnostic accuracy of computed tomography-derived fractional flow reserve: a systematic review.
        JAMA Cardiol. 2017; 2: 803-810
        • Villines T.C.
        Can CT-derived FFR better inform clinical decision-making and improve outcomes in stable ischaemic heart disease?.
        Eur Heart J. 2018; 39: 3712-3714
        • Gaur S.
        • Ovrehus K.A.
        • Dey D.
        • et al.
        Coronary plaque quantification and fractional flow reserve by coronary computed tomography angiography identify ischaemia-causing lesions.
        Eur Heart J. 2016; 37: 1220-1227
        • Ovrehus K.A.
        • Gaur S.
        • Leipsic J.
        • et al.
        CT-based total vessel plaque analyses improves prediction of hemodynamic significance lesions as assessed by fractional flow reserve in patients with stable angina pectoris.
        J Cardiovasc Comput Tomogr. 2018; 12: 344-349
        • Fujimoto S.
        • Kondo T.
        • Kodama T.
        • et al.
        A novel method for non-invasive plaque morphology analysis by coronary computed tomography angiography.
        Int J Cardiovasc Imaging. 2014; 30: 1373-1382
        • Menze B.H.
        • Kelm B.M.
        • Masuch R.
        • et al.
        A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data.
        BMC Bioinformatics. 2009; 10: 213
        • Budoff M.J.
        • Young R.
        • Burke G.
        • et al.
        Ten-year association of coronary artery calcium with atherosclerotic cardiovascular disease (ASCVD) events: the multi-ethnic study of atherosclerosis (MESA).
        Eur Heart J. 2018; 39: 2401-2408
        • Cademartiri F.
        • Mollet N.R.
        • Runza G.
        • et al.
        Influence of intracoronary attenuation on coronary plaque measurements using multislice computed tomography: observations in an ex vivo model of coronary computed tomography angiography.
        Eur Radiol. 2005; 15: 1426-1431
        • Kidoh M.
        • Utsunomiya D.
        • Oda S.
        • et al.
        Evaluation of the effect of intracoronary attenuation on coronary plaque measurements using a dual-phase coronary CT angiography technique on a 320-row CT Scanner–in vivo validation study.
        Acad Radiol. 2016; 23: 315-320
        • Celeng C.
        • Leiner T.
        • Maurovich-Horvat P.
        • et al.
        Anatomical and functional computed tomography for diagnosing hemodynamically significant coronary artery disease: a meta-analysis.
        JACC Cardiovasc Imaging. 2019; 12: 1316-1325
        • Douglas P.S.
        • Pontone G.
        • Hlatky M.A.
        • et al.
        Clinical outcomes of fractional flow reserve by computed tomographic angiography-guided diagnostic strategies vs. usual care in patients with suspected coronary artery disease: the prospective longitudinal trial of FFR(CT): outcome and resource impacts study.
        Eur Heart J. 2015; 36: 3359-3367
        • Dey D.
        • Slomka P.J.
        • Leeson P.
        • et al.
        Artificial intelligence in cardiovascular imaging: JACC State-of-the-Art Review.
        J Am Coll Cardiol. 2019; 73: 1317-1335
        • Ruijsink B.
        • Puyol-Anton E.
        • Oksuz I.
        • et al.
        Fully automated, quality-controlled cardiac analysis from CMR: validation and large-scale application to characterize cardiac function.
        JACC Cardiovasc Imaging. 2019;
        • Zheng Y.
        • John M.
        • Liao R.
        • et al.
        Automatic aorta segmentation and valve landmark detection in C-arm CT for transcatheter aortic valve implantation.
        IEEE Trans Med Imaging. 2012; 31: 2307-2321
        • Motwani M.
        • Dey D.
        • Berman D.S.
        • et al.
        Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis.
        Eur Heart J. 2017; 38: 500-507
        • Ozigis M.S.
        • Kaduk J.D.
        • Jarvis C.H.
        • et al.
        Detection of oil pollution impacts on vegetation using multifrequency SAR, multispectral images with fuzzy forest and random forest methods.
        Environ Pollut. 2020; 256113360
        • Chartrand G.
        • Cheng P.M.
        • Vorontsov E.
        • et al.
        Deep learning: a primer for radiologists.
        Radiographics. 2017; 37: 2113-2131
        • van Hamersvelt R.W.
        • Zreik M.
        • Voskuil M.
        • et al.
        Deep learning analysis of left ventricular myocardium in CT angiographic intermediate-degree coronary stenosis improves the diagnostic accuracy for identification of functionally significant stenosis.
        Eur Radiol. 2019; 29: 2350-2359
        • Tang A.
        • Tam R.
        • Cadrin-Chenevert A.
        • et al.
        Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology.
        Can Assoc Radiol J. 2018; 69: 120-135
        • Ri K.
        • Kumamaru K.K.
        • Fujimoto S.
        • et al.
        Noninvasive computed tomography-derived fractional flow reserve based on structural and fluid analysis: reproducibility of on-site determination by unexperienced observers.
        J Comput Assist Tomogr. 2018; 42: 256-262