Advertisement

Clinical Evaluation of the Automatic Coronary Artery Disease Reporting and Data System (CAD-RADS) in Coronary Computed Tomography Angiography Using Convolutional Neural Networks

  • Author Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Zengfa Huang
    Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Jianwei Xiao
    Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Xi Wang
    Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Zuoqin Li
    Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Ning Guo
    Affiliations
    Shukun (Beijing) Technology Co., Ltd., Jinhui Building, Qiyang Road, 100102 Beijing, China
    Search for articles by this author
  • Yun Hu
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Xiang Li
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Xiang Wang
    Correspondence
    Address correspondence to: X.W.
    Affiliations
    Department of Radiology, The Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    # Zengfa Huang, Jianwei Xiao, Xi Wang, and Zuoqin Li contributed equally to this work.
Open AccessPublished:June 23, 2022DOI:https://doi.org/10.1016/j.acra.2022.05.015

      Rationale and Objectives

      The coronary artery disease reporting and data system (CAD-RADS™) was recently introduced to standardise reporting. We aimed to evaluate the utility of an automatic postprocessing and reporting system based on CAD-RADS™ in suspected coronary artery disease (CAD) patients.

      Materials and Methods

      Clinical evaluation was performed in 346 patients who underwent coronary computed tomography angiography (CCTA). We compared deep learning (DL)-based CCTA with human readers for evaluation of CAD-RADS™ with commercially-available automated segmentation and manual postprocessing in a retrospective validation cohort.

      Results

      Compared with invasive coronary angiography, the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the DL model for diagnosis of CAD were 79.02%, 86.52%, 89.50%, 73.94%, and 82.08%, respectively. There was no significant difference between the DL-based and the reader-based CAD-RADS™ grading of CCTA results. Consistency testing showed that the Kappa value between the model and the readers was 0.775 (95% confidence interval [CI]: 0.728–0.823, p < 0.001), 0.802 (95% CI: 0.756–0.847, p < 0.001), and 0.796 (95% CI: 0.750–0.843, p < 0.001), respectively. This system reduces the time taken from 14.97 ± 1.80 min to 5.02 ± 0.8 min (p < 0.001).

      Conclusion

      The standardised reporting of DL-based CAD-RADS™ in CCTA can accurately and rapidly evaluate suspected CAD patients, and has good consistency with grading by radiologists.

      Key Words

      INTRODUCTION

      Coronary artery disease (CAD) is a global medical problem and a leading cause of morbidity and mortality (
      • Benjamin EJ
      • Blaha MJ
      • Chiuve SE
      • et al.
      Heart disease and stroke statistics-2017 update: a report from the American Heart Association.
      ). Patients with CAD are at risk of ischaemic stroke, myocardial infarction, and cardiovascular death (
      • Connolly SJ
      • Eikelboom JW
      • Bosch J
      • et al.
      Rivaroxaban with or without aspirin in patients with stable coronary artery disease: an international, randomised, double-blind, placebo-controlled trial.
      ). Coronary computed tomography angiography (CCTA) is increasingly being used as a noninvasive evaluation method to assess patients with chest pain because it has high sensitivity and specificity for the detection of CAD (
      • Investigators S-H
      • Newby DE
      • Adamson PD
      • et al.
      Coronary CT angiography and 5-year risk of myocardial infarction.
      ,
      • Adamson PD
      • Williams MC
      • Dweck MR
      • et al.
      Guiding therapy by coronary CT angiography improves outcomes in patients with stable chest pain.
      ). Moreover, unlike functional stress testing, CCTA provides an anatomical assessment of the coronary arteries, allowing for the detection and quantification of obstructive and non-obstructive atherosclerotic plaques (
      • Bittencourt MS
      • Hulten EA
      • Veeranna V
      • et al.
      Coronary computed tomography angiography in the evaluation of chest pain of suspected cardiac origin.
      ).
      The coronary artery disease reporting and data system (CAD-RADS™) was introduced in 2016 to standardise reporting and decision-making (
      • Cury RC
      • Abbara S
      • Achenbach S
      • et al.
      CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
      ). Unlike prior classifications of CAD that were based on the number of affected vessels, CAD-RADS™ classifies patients based on the highest grade of coronary artery stenosis. Xie et al. recently reported that CAD-RADS™ effectively identified patients at risk of myocardial infarctions and all-cause mortality to a similar degree as the CAD extent classification and the Duke CAD Index (
      • Xie JX
      • Cury RC
      • Leipsic J
      • et al.
      The Coronary Artery Disease-Reporting and Data System (CAD-RADS): prognostic and clinical implications associated with standardized coronary computed tomography angiography reporting.
      ).
      Computer-aided diagnosis has been rapidly growing and is increasingly used in radiological assessment (
      • Lynch CJ
      • Liston C
      New machine-learning technologies for computer-aided diagnosis.
      ,
      • Ebigbo A
      • Mendel R
      • Probst A
      • et al.
      Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma.
      ). Deep convolutional neural networks (CNN) are a newly-emerging form of computer-aided diagnostic analysis that allows the automatic extraction of features and supervised learning of large amounts of data to make quantitative decisions (
      • Song Y
      • Zhang YD
      • Yan X
      • et al.
      Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI.
      ). Accumulated evidence suggests that deep learning (DL) analysis might be a potential alternative to conventional hand-crafted methodologies for solving pattern-recognition and imaging-classification problems (
      • Schmidhuber J
      Deep learning in neural networks: an overview.
      ,
      • Alipanahi B
      • Delong A
      • Weirauch MT
      • et al.
      Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning.
      ). To date, there has been no research focusing on the combination of CNN and CCTA to solve lesion detection and classification problems based on CAD-RADS™.
      Thus, we aimed to assess the utility of an automatic post-processing and reporting system based on CAD-RADS™ in CAD or suspected CAD patients. We also sought to investigate the performance of DL-based CCTA to assist radiologists in their daily work and establish a time-saving work process.

      MATERIALS AND METHODS

      The present study was approved by the local institutional review board and was conducted in compliance with the Health Insurance Portability and Accountability Act of 1996.

      Patients

      We retrospectively searched a database of CCTA data from patients who were examined between July 2017 and December 2019. The study population consisted of 346 consecutive patients who were confirmed or suspected to have CAD (Fig 1). Scans affected by stair-step artifacts, respiratory motion artifacts, excessive image noise, or artifacts caused by arrhythmia, or poor quality images were excluded from analysis. We also excluded patients without complete records and those who previously underwent percutaneous coronary intervention or coronary artery bypass grafting or other cardio surgery from analysis.
      Figure 1
      Figure 1Flowchart illustrating exclusion criteria and final study population.

      CCTA Image Acquisition and Analysis

      Multidetector row computed tomography (CT) imaging was performed with a dual-source CT scanner (Somatom Definition, Siemens Medical Solutions, Forchheim, Germany) and a 256-slice CT scanner (Brilliance iCT; Philips Healthcare, Cleveland, OH). Heart rate (HR) control (≥ 65 beats/min) was performed with beta-blockers before the scan. For contrast enhancement, 60–80 mL of iopromide (370 mgI/mL, Bayer Schering Pharma, Leverkusen, Germany) followed by 30–40 mL of pure saline was administered with a flow rate of 4–5 mL/s. The iodine contrast agent was automatically triggered into the descending aorta of 100 Hounsfield units threshold units. Then the scanning was performed during an inspiratory breath hold of 8–14 s after a delay of 2 s. The reconstructed images were automatically sent to a workstation (CoronaryDoc, Shukun technology, Beijing, China) equipped with a coronary analysis software tool (Computer Aided Diagnosis of Coronary Artery, Version 1.8, Shukun technology).

      Deep Learning (DL)

      The patients in the test set were examined by invasive coronary angiography (ICA) with an interval of fewer than 30 days after the CCTA procedure. We have previously reported the validation of our DL system (
      • Chen M
      • Wang X
      • Hao G
      • et al.
      Diagnostic performance of deep learning-based vascular extraction and stenosis detection technique for coronary artery disease.
      ,
      • Huang Z
      • Xiao J
      • Xie Y
      • et al.
      The correlation of deep learning-based CAD-RADS evaluated by coronary computed tomography angiography with breast arterial calcification on mammography.
      ). Before training, the aorta, coronary artery and plaques were labeled on each image using a multi-layer manual annotation system consisting of multiple layers of trained graders. The first layer of graders consisted of radiologists who had knowledge of medical imaging and coronary anatomy. The second layer of graders comprised radiologists with more than 3 years of work experience in radiology, who performed a preliminary inspection of the accuracy of the label. The third and final layer of graders was made up of experienced experts with over 5 years of work experience who verified the correctness of the label of each image. In this study, we adopted an improved three-dimensional (3D) U-Net architecture and added a bottle-neck model for segmentation of coronary arteries and aorta, then a growing iterative prediction network model was developed to solve the problem of vascular segmentation fracture, and finally full coronary tree segmentation was obtained (
      • Yang G
      • Kitslaar P
      • Frenay M
      • et al.
      Automatic centerline extraction of coronary arteries in coronary computed tomographic angiography.
      ). Based on coronary tree segmentation, multiple planar reformat (MPR), curve planar reformat (CPR), maximum intensity projection, and volume rendering images were reconstructed. To detect stenosis, we developed a 3D segmentation neural network and a one-dimensional sequence checking hybrid technique (

      Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International Publishing, 2015.

      ). Firstly, a 3D segmentation neural network was applied to MPR and CPR images to detect stenosis, and then a one-dimensional sequence-checking algorithm was used to reduce false-positive results (Fig 2).
      Figure 2
      Figure 2Flowchart of the process of the model; convolutional neural networks (CNN) and deep learning (DL)-based coronary computed tomography angiography (CCTA) assistant diagnosis system. (a) Diagram showing complete workflow of the aorta, coronary artery and plaque labelling and segmentation network. Each blue box corresponds to a convolutional layer, with matrix size and depth indicated at each layer. Colored arrows indicate different operations: yellow = convolution and rectified linear unit (ReLU), orange = copy and concatenate from down-sampling path to up-sampling path, green = maximum pooling, blue = up-convolution and purple = convolution. The input was images from CCTA acquisition, whereas the output was the aorta, coronary artery and plaque contours if these are detected on the image. (b) CNN networks for coronary artery disease reporting and data system (CAD-RADS™) classification. Plaque segmentation was applied by a 3D UNET network on a curve planar reformat (CPR) image and a straightened image respectively, and then the output results were modified and classified by a one-dimensional back-detection model. Finally, the CAD-RADS™ classification output was determined according to the degree of stenosis. C&D, DL-based CCTA assistant diagnostic system. (Color version of figure is available online.)
      Three readers with 5 years of experience in cardiac CT imaging diagnosis recorded the CAD-RADS™ classification based on the degree of coronary stenosis (CAD-RADS 0: 0%, CAD-RADS 1: 1–24%, CAD-RADS 2: 25–49%, CAD-RADS 3: 50–69%, CAD-RADS 4A: 70–99%, CAD-RADS 4B: Left main > 50% or 3-vessel disease, 70–99%, CAD-RADS 5: 100%) according to the CAD-RADS™ consensus document (
      • Cury RC
      • Abbara S
      • Achenbach S
      • et al.
      CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
      ). A structured report including CAD-RADS™ category was created based on the model independently. All the readers and the model provided CAD-RADS™ scores independently, which remained hidden to each. Coronary artery lesions were evaluated for stenosis on a diameter-based approach. The readers, who were blinded to the results of each other and to the model, manually measured the relevant lesions independently. One hundred patients were randomly selected for time consumption (including postprocessing, report-writing, typesetting and print) analysis. CAD was defined as stenosis > 50% in a coronary artery segment ≥ 2mm in diameter.

      Statistical Analysis

      Continuous variables are presented as mean ± SD. Categorical variables are presented as percentages or absolute values. We used either the chi-square test, or Fisher's exact test as appropriate for categorical variables. The diagnostic performance of CAD with DL-based CCTA was determined with regard to sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy by comparison with ICA as the standard of reference. The agreement of the CAD-RADS™ categories was compared between readers and the model using the linear weighted kappa identity test. p < 0.05 was considered to represent statistical significance. All statistical analysis was performed using SPSS version 18 (SPSS, Inc., Chicago, IL, USA) and MedCalc Statistical Software version 16.8.4.0 (Ostend, Belgium).

      RESULTS

      Table 1 shows the baseline characteristics of the 346 study participants (mean age: 62.9 years; range from 33 to 85 years). Of the 346 patients, 181 (52.3%), 181 (52.3%), 180 (52.0%), and 180 (52.0%) were diagnosed with CAD by the DL model, Reader 1, Reader 2, and Reader 3, respectively. There were no significant differences between readers and the DL model in the detection of CAD. The PPV, NPV, sensitivity, specificity, and accuracy of the DL model for diagnosis of CAD were 79.02%, 86.52%, 89.50%, 73.94%, and 82.08%, respectively when using ICA as the reference standard (Table 2). The area under the receiver operating characteristic curve (AUC) of the DL model for CAD diagnosis was 0.83, which was similar to those of Reader 1 (0.84), Reader 2 (0.82) and Reader 3 (0.84) (all p > 0.05, Fig 3). In addition, the diagnostic performance of CAD between readers was similar (all p > 0.05, Fig 3).
      Table 1Basic Characteristics of the Included Patients
      CharacteristicsTotal (N = 346)
      Age62.9 ± 9.1
      Male, no. (%)191 (55)
      BMI (kg/m2)25.2 ± 7.3
      Hypertension210 (60.7)
      DM93 (26.9)
      CKD37 (10.7)
      TG2.0 ± 1.7
      TC4.7 ± 1.1
      HDL-C1.2 ± 0.3
      LDL-C2.8 ± 0.9
      SCR71.4 ± 38
      BUN5.6 ± 1.9
      EF (%)59.6 ± 5.8
      BMI, body mass index; BUN, blood urea nitrogen; CKD, chronic kidney disease; LDL-C, low density lipoprotein-C; DM, diabetes mellitus; EF, ejection fraction; HDL-C, high density lipoprotein-C; SCR, serum creatinine; TC, total cholesterol; TG, triglyceride.
      Table 2Diagnostic Performance of DL for Detecting CAD Compared with Readers
      VariableTPTNFPFNSensitivitySpecificityPPVNPVAccuracyAUC
      DL162122194379.02 (72.68-84.25)86.52

      (79.51-91.49)
      89.50

      (83.87-93.40)
      73.94

      (66.43-80.31)
      82.08 (77.62-86.54)0.83
      DL vs. Reader 1 (p = 0.960).
      ,
      DL vs. Reader 2 (p = 0. 169).
      ,
      DL vs. Reader 3 (p = 0. 625).
      (0.78-0.87)
      Reader 1164124174180.00 (73.73-85.12)87.94

      (81.13-92.61)
      90.61

      (85.15-94.27)
      75.15

      (67.71-81.39)
      83.24 (78.92-87.55)0.84
      Reader 1 vs. Reader 2 (p = 0.25)
      ,
      Reader 1 vs. Reader 3(p = 0.898).


      (0.80-0.88)
      Reader 2160121204578.05 (71.63-83.39)85.82

      (78.70-90.91)
      88.89

      (83.14-92.91)
      72.89

      (65.35-79.35)
      81.21 (76.64-85.78)0.82
      Reader 2 vs. Reader 3(p = 0.327).
      (0.78-0.86)
      Reader 3163124174279.51 (73.21-84.69)87.94

      (81.13-92.61)
      90.56

      (85.07-94.23)
      74.70

      (67.26-80.97)
      83.95 (78.60-87.30)0.84 (0.80-0.88)
      AUC, area under the curve; CAD, coronary artery disease; DL, deep learning; FN, false negative; FP, false positive; NPV, negative predictive value; PPV, positive predictive value; TN, true negative; TP, true positive.
      low asterisk DL vs. Reader 1 (p = 0.960).
      # DL vs. Reader 2 (p = 0. 169).
      & DL vs. Reader 3 (p = 0. 625).
      ^ Reader 1 vs. Reader 2 (p = 0.25)
      $ Reader 1 vs. Reader 3(p = 0.898).
      ! Reader 2 vs. Reader 3(p = 0.327).
      Figure 3
      Figure 3Receiver operating characteristic (ROC) curves for coronary artery disease (CAD) diagnosis by DL and Readers.
      The distribution classification of CAD-RADS™ assessed by the DL model was 38 as CAD-RADS 0, 19 as CAD-RADS 1, 108 as CAD-RADS 2, 90 as CAD-RADS 3, 58 as CAD-RADS 4A, 24 as CAD-RADS 4B and 9 as CAD-RADS 5 (Fig 4). As shown in Table 3, the agreement between the DL model and Reader 1 for CAD-RADS™ classification was good (Kappa = 0.775; 95% confidence interval [CI]: 0.782–0.823). The agreement between the DL model and Reader 2 working in consensus was excellent (Kappa = 0.802; 95% CI: 0.756–0.847). The agreement between the DL model and Reader 3 was also good (Kappa = 0.796; 95% CI: 0.750–0.843). Moreover, the agreement between readers for CAD-RADS™ showed excellent results (Supplementary file; Table S1).
      Figure 4
      Figure 4The distribution classification of CAD-RADS by DL and Readers.
      Table 3Agreement Between the DL Model and Readers for CAD-RADS Classification
      ComparisonLinear Weighted Kappa Valuep Value
      DL vs. Reader 10.775 (0.728-0.823)<0.001
      DL vs. Reader 20.802 (0.756-0.847)<0.001
      DL vs. Reader 30.796 (0.750-0.843)<0.001
      CAD-RADS, Coronary Artery Disease-Reporting and Data System; DL, deep learning.
      In comparison with ICA, a total of 62 patients were misclassified by the DL model. The DL model underestimated 43 patients with CAD including five patients with a DL-based CAD-RADS 1 classification and 38 patients with a DL-based CAD-RADS 2 classification. In contrast, the DL model overestimated 19 patients without CAD who were classified as CAD-RADS 2. Moreover, the clinical workup would be changed in 60 (17.3%), 46 (13.3%), and 49 (14.2%) patients if the DL model alone was used when compared with Reader 1, Reader 2, And Reader 3, respectively. In addition, the clinical workup of 62 (17.9%), 58 (16.8%), 65 (18.8%), and 59 (17.1%) patients would be changed if using the DL model, Reader 1, Reader 2, And Reader 3, respectively when compared with ICA.
      The time taken (including postprocessing, report-writing, typesetting, and printing) of the three readers was 14.51 ± 1.92 min, 15.28 ± 1.64 min and 15.11 ± 1.83 min, respectively. In comparison, the average time taken by DL was reduced from 14.97 ± 1.80 min to 5.02 ± 0.8 min (p < 0.001, Fig 5). We also found that the time taken differed between manually-recognised CAD patients and non-CAD patients, while there was no significant difference when DL was used.
      Figure 5
      Figure 5The comparison of consumed time between the DL and Readers.

      DISCUSSION

      In this study, we first developed a novel CNN model using CCTA images and evaluated its performance in categorization of coronary artery stenosis based on CAD-RADS™. Then, the utilisation of an automatic post-processing and reporting system based on CAD-RADS was assessed in suspected CAD patients.
      As a standardised reporting system of CCTA, the primary aim of CAD-RADS™ is to facilitate consistent assessment among physicians, including recommendations for further investigations and management (
      • Cury RC
      • Abbara S
      • Achenbach S
      • et al.
      CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
      ,
      • Chandrashekhar Y
      • Min JK
      • Hecht H
      • et al.
      CAD-RADS: a giant first step toward a common lexicon?.
      ). Moreover, the common language of coronary artery stenosis provided by this reporting system increases the clarity between radiologists and clinicians in the diagnosis and treatment planning of suspected CAD patients. In addition, the achieved standardisation of reporting will be of benefit in education, research, peer review, and quality assurance, ultimately resulting in improved quality of care (
      • Nieman K
      • Hoffmann U
      Cardiac computed tomography in patients with acute chest pain.
      ).
      Cancer imaging standardised reporting systems, such as Lung-RADS and BI-RADS have been widely accepted by offering reliable and consistent assessment categories to radiologists and physicians who can then supply appropriate management recommendations for specific patients. Structured reporting tools have been applied in cardiac imaging to improve imaging data integrity and then establish standard databases for education, patient care and research purposes (
      • Faggioni L
      • Coppola F
      • Ferrari R
      • et al.
      Usage of structured reporting in radiological practice: results from an Italian online survey.
      ). Structured reporting platforms based on automated CAD-RADS™ calculations have been proposed by Dewey et al. (
      • Dewey M
      Structure or entropy in reporting cardiac CT findings.
      ), and Szilveszter et al. have revealed good agreement between manual and automated CAD-RADS™ classifications using a structured reporting platform (
      • Szilveszter B
      • Kolossvary M
      • Karady J
      • et al.
      Structured reporting platform improves CAD-RADS assessment.
      ). Previous methods for automated CAD or CAD-RADS™ assessment showed strong agreement and inter-observer reproducibility (
      • Maroules CD
      • Hamilton-Craig C
      • Branch K
      • et al.
      Coronary artery disease reporting and data system (CAD-RADS(TM)): inter-observer agreement for assessment categories and modifiers.
      ,
      • Abdel-Razek AAK
      • Elrakhawy MM
      • Yossof MM
      • et al.
      Inter-observer agreement of the Coronary Artery Disease Reporting and Data System (CAD-RADS(TM)) in patients with stable chest pain.
      ). However, no machine learning or DL models have been used in these studies.
      Recently, promising applications of machine learning or artificial intelligence (AI) have been used in cardiovascular imaging and cardiology to process several algorithms that are already in routine clinical use (
      • Henglin M
      • Stein G
      • Hushcha PV
      • et al.
      Machine learning approaches in cardiovascular imaging.
      ,
      • Dey D
      • Slomka PJ
      • Leeson P
      • et al.
      artificial intelligence in cardiovascular imaging: JACC state-of-the-art review.
      ). These algorithms are likely to provide a process to improve the quality and speed of acquisition, reduce manual postprocessing, measurement and manual report writing time, and allow prompt diagnoses, which in turn would improve workflow and patient care. Moreover, these could facilitate screening programs and promote establishment of more effective referral mechanisms in medicine, particularly in remote or low-resource areas, leading to a broad, positive clinical, and public health impact (
      • Kermany DS
      • Goldbaum M
      • Cai W
      • et al.
      Identifying medical diagnoses and treatable diseases by image-based deep learning.
      ). In addition, they encourage researchers to improve the performance of future models and help drive the field forward.
      Machine learning algorithms have been extensively used for the optimisation of information extraction from CCTA. A two-step support vector machine based on learning with CCTA was established by Kang et al. (
      • Kang D
      • Dey D
      • Slomka PJ
      • et al.
      Structured learning algorithm for detection of nonobstructive and obstructive coronary plaque lesions from computed tomography angiography.
      ) for detecting nonobstructive and obstructive CAD, and resulted in an accuracy of 94% and an AUC of 0.94 and thus had a higher accuracy than obtained in the present study. However, only two classifications were used in the study and obstructive CAD was defined as lesion with stenosis ≥25%. However in our study the CAD-RADS classification system was applied and we used the widely-accepted definition of CAD (lesion with stenosis ≥50%). Moreover, a small CCTA dataset with only 42 patients were included in the study by Kang et al., while our study included a relatively larger sample size of 346 patients. In addition, our DL platform was trained and validated using the Digital Imaging and Communications in Medicine (DICOM) standards, which makes CCTA images from different manufacturers (dual-source CT and Philips ICT) reasonably consistent. Finally, our study developed a DL model which was able to automatically extract coronary arteries and detect stenosis.
      Accumulated researches revealed good automatic detection and quantification with CCTA images. Schuhbaeck et al. evaluated the interscan reproducibility of quantitative measurements of coronary plaque volumes using a standardised automated method (
      • Schuhbaeck A
      • Dey D
      • Otaki Y
      • et al.
      Interscan reproducibility of quantitative coronary plaque volume and composition from CT coronary angiography using an automated method.
      ). A recent study showed that CCTA-derived plaque markers combined with a DL-based CT-FFR had a good predictive value in identifying lesion-specific ischaemia when compared to CCTA stenosis grading alone (
      • von Knebel Doeberitz PL
      • De Cecco CN
      • Schoepf UJ
      • et al.
      Coronary CT angiography-derived plaque quantification with artificial intelligence CT fractional flow reserve for the identification of lesion-specific ischemia.
      ). Quantification of coronary artery calcium score derived from CCTA with a DL algorithm has also been validated to be accurate (
      • Schuhbaeck A
      • Otaki Y
      • Achenbach S
      • et al.
      Coronary calcium scoring from contrast coronary CT angiography using a semiautomated standardized method.
      ,
      • Wang W
      • Wang H
      • Chen Q
      • et al.
      Coronary artery calcium score quantification using a deep-learning algorithm.
      ). A previous study demonstrated that a machine learning algorithm-based CT-FFR value was able to assess the functional significance of CAD and make a therapeutic decision. This algorithm showed excellent performance (sensitivity of 97%, specificity of 100%, PPV 100%, NPV of 97%, and accuracy of 99%) of ML-based CT-FFR with CCTA in determining an appropriate treatment strategy (
      • Tesche C
      • Vliegenthart R
      • Duguay TM
      • et al.
      Coronary computed tomographic angiography-derived fractional flow reserve for therapeutic decision making.
      ). However, few studies have focussed on DL-based CCTA using CAD-RADS™ for stratifying the risk of suspected CAD patients. The present study was designed to evaluate the feasibility of CAD-RADS™ classifications by using DL based on CCTA images (Fig 6).
      Figure 6
      Figure 6DL-based CAD-RADS™ classification. (a and b) CAD-RADS 0; (c) CAD-RADS 1; (d) CAD-RADS 2; (e) CAD-RADS 3; (f) CAD-RADS 4A; (g) CAD-RADS 4B; (h) CAD-RADS 5.
      Some limitations of this study should be addressed. First, this was a retrospective analysis of a relatively small sample size from a single centre. Prospective multicentre studies of large samples will enhance the application of CAD-RADS™ among cardiologists and radiologists. Second, in order to restrict our analysis, we did not include CAD-RADS™ modifiers to describe patients with stents (modifier S), vulnerable plaque features (modifier V), or grafts (modifier G). Further studies should include more patients in the training set and testing set of DL to improve the CAD-RADS™ classification scheme. Third, the DL model slightly underestimated CAD-RADS 4A and CAD-RADS 5 patients compared to the readers as shown in Figure 4. As a result, this would impact clinical decision-making as the consideration of ICA would be reduced in a portion of patients that might need them. It should be noted that the sensitivity of the DL model for diagnosing CAD is 79%, which is lower than previous studies with a sensitivity of 84–94% using CCTA (
      • Marano R
      • Rovere G
      • Savino G
      • et al.
      CCTA in the diagnosis of coronary artery disease.
      ). Further improvement of this model is still needed to reduce the possibility of underdiagnosing CAD. In addition, visual evaluation by ICA may overestimate stenosis grade, as has been demonstrated in previous research (
      • Nallamothu BK
      • Spertus JA
      • Lansky AJ
      • et al.
      Comparison of clinical interpretation with visual assessment and quantitative coronary angiography in patients undergoing percutaneous coronary intervention in contemporary practice: the Assessing Angiography (A2) project.
      ).

      CONCLUSION

      The standardised reporting of DL-based CAD-RADS™ in CCTA images can accurately evaluate suspected CAD patients while saving time, and has excellent consistency with radiologists.

      ACKNOWLEDGMENTS

      None.

      FUNDING

      Not applicable.

      ETHICS APPROVAL AND CONSENT TO PARTICIPATE

      The research was approved by the Institutional Review Board of the Clinical Research Institute at The Central Hospital of Wuhan.

      Appendix. Supplementary materials

      REFERENCES

        • Benjamin EJ
        • Blaha MJ
        • Chiuve SE
        • et al.
        Heart disease and stroke statistics-2017 update: a report from the American Heart Association.
        Circulation. 2017; 135: e146-e603
        • Connolly SJ
        • Eikelboom JW
        • Bosch J
        • et al.
        Rivaroxaban with or without aspirin in patients with stable coronary artery disease: an international, randomised, double-blind, placebo-controlled trial.
        Lancet. 2018; 391: 205-218
        • Investigators S-H
        • Newby DE
        • Adamson PD
        • et al.
        Coronary CT angiography and 5-year risk of myocardial infarction.
        N Engl J Med. 2018; 379: 924-933
        • Adamson PD
        • Williams MC
        • Dweck MR
        • et al.
        Guiding therapy by coronary CT angiography improves outcomes in patients with stable chest pain.
        J Am Coll Cardiol. 2019; 74: 2058-2070
        • Bittencourt MS
        • Hulten EA
        • Veeranna V
        • et al.
        Coronary computed tomography angiography in the evaluation of chest pain of suspected cardiac origin.
        Circulation. 2016; 133: 1963-1968
        • Cury RC
        • Abbara S
        • Achenbach S
        • et al.
        CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology.
        J Cardiovasc Comput Tomogr. 2016; 10: 269-281
        • Xie JX
        • Cury RC
        • Leipsic J
        • et al.
        The Coronary Artery Disease-Reporting and Data System (CAD-RADS): prognostic and clinical implications associated with standardized coronary computed tomography angiography reporting.
        JACC Cardiovasc Imaging. 2018; 11: 78-89
        • Lynch CJ
        • Liston C
        New machine-learning technologies for computer-aided diagnosis.
        Nat Med. 2018; 24: 1304-1305
        • Ebigbo A
        • Mendel R
        • Probst A
        • et al.
        Computer-aided diagnosis using deep learning in the evaluation of early oesophageal adenocarcinoma.
        Gut. 2019; 68: 1143-1145
        • Song Y
        • Zhang YD
        • Yan X
        • et al.
        Computer-aided diagnosis of prostate cancer using a deep convolutional neural network from multiparametric MRI.
        J Magn Reson Imaging. 2018; 48: 1570-1577
        • Schmidhuber J
        Deep learning in neural networks: an overview.
        Neural Netw. 2015; 61: 85-117
        • Alipanahi B
        • Delong A
        • Weirauch MT
        • et al.
        Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning.
        Nat Biotechnol. 2015; 33: 831-838
        • Chen M
        • Wang X
        • Hao G
        • et al.
        Diagnostic performance of deep learning-based vascular extraction and stenosis detection technique for coronary artery disease.
        Br J Radiol. 2020; 9320191028
        • Huang Z
        • Xiao J
        • Xie Y
        • et al.
        The correlation of deep learning-based CAD-RADS evaluated by coronary computed tomography angiography with breast arterial calcification on mammography.
        Sci Rep. 2020; 10: 11532
        • Yang G
        • Kitslaar P
        • Frenay M
        • et al.
        Automatic centerline extraction of coronary arteries in coronary computed tomographic angiography.
        Int J Cardiovasc Imaging. 2012; 28: 921-933
      1. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer International Publishing, 2015.

        • Chandrashekhar Y
        • Min JK
        • Hecht H
        • et al.
        CAD-RADS: a giant first step toward a common lexicon?.
        JACC Cardiovasc Imaging. 2016; 9: 1125-1129
        • Nieman K
        • Hoffmann U
        Cardiac computed tomography in patients with acute chest pain.
        Eur Heart J. 2015; 36: 906-914
        • Faggioni L
        • Coppola F
        • Ferrari R
        • et al.
        Usage of structured reporting in radiological practice: results from an Italian online survey.
        Eur Radiol. 2017; 27: 1934-1943
        • Dewey M
        Structure or entropy in reporting cardiac CT findings.
        Int J Cardiovasc Imaging. 2016; 32: 1657-1658
        • Szilveszter B
        • Kolossvary M
        • Karady J
        • et al.
        Structured reporting platform improves CAD-RADS assessment.
        J Cardiovasc Comput Tomogr. 2017; 11: 449-454
        • Maroules CD
        • Hamilton-Craig C
        • Branch K
        • et al.
        Coronary artery disease reporting and data system (CAD-RADS(TM)): inter-observer agreement for assessment categories and modifiers.
        J Cardiovasc Comput Tomogr. 2018; 12: 125-130
        • Abdel-Razek AAK
        • Elrakhawy MM
        • Yossof MM
        • et al.
        Inter-observer agreement of the Coronary Artery Disease Reporting and Data System (CAD-RADS(TM)) in patients with stable chest pain.
        Pol J Radiol. 2018; 83 (-e9): e151
        • Henglin M
        • Stein G
        • Hushcha PV
        • et al.
        Machine learning approaches in cardiovascular imaging.
        Circ Cardiovasc Imaging. 2017; 10
        • Dey D
        • Slomka PJ
        • Leeson P
        • et al.
        artificial intelligence in cardiovascular imaging: JACC state-of-the-art review.
        J Am Coll Cardiol. 2019; 73: 1317-1335
        • Kermany DS
        • Goldbaum M
        • Cai W
        • et al.
        Identifying medical diagnoses and treatable diseases by image-based deep learning.
        Cell. 2018; 172 (-e9): 1122-1131
        • Kang D
        • Dey D
        • Slomka PJ
        • et al.
        Structured learning algorithm for detection of nonobstructive and obstructive coronary plaque lesions from computed tomography angiography.
        J Med Imaging (Bellingham). 2015; 2014003
        • Schuhbaeck A
        • Dey D
        • Otaki Y
        • et al.
        Interscan reproducibility of quantitative coronary plaque volume and composition from CT coronary angiography using an automated method.
        Eur Radiol. 2014; 24: 2300-2308
        • von Knebel Doeberitz PL
        • De Cecco CN
        • Schoepf UJ
        • et al.
        Coronary CT angiography-derived plaque quantification with artificial intelligence CT fractional flow reserve for the identification of lesion-specific ischemia.
        Eur Radiol. 2019; 29: 2378-2387
        • Schuhbaeck A
        • Otaki Y
        • Achenbach S
        • et al.
        Coronary calcium scoring from contrast coronary CT angiography using a semiautomated standardized method.
        J Cardiovasc Comput Tomogr. 2015; 9: 446-453
        • Wang W
        • Wang H
        • Chen Q
        • et al.
        Coronary artery calcium score quantification using a deep-learning algorithm.
        Clin Radiol. 2019;
        • Tesche C
        • Vliegenthart R
        • Duguay TM
        • et al.
        Coronary computed tomographic angiography-derived fractional flow reserve for therapeutic decision making.
        Am J Cardiol. 2017; 120: 2121-2127
        • Marano R
        • Rovere G
        • Savino G
        • et al.
        CCTA in the diagnosis of coronary artery disease.
        La Radiologia medica. 2020; 125: 1102-1113
        • Nallamothu BK
        • Spertus JA
        • Lansky AJ
        • et al.
        Comparison of clinical interpretation with visual assessment and quantitative coronary angiography in patients undergoing percutaneous coronary intervention in contemporary practice: the Assessing Angiography (A2) project.
        Circulation. 2013; 127: 1793-1800