Advertisement
Original Investigation| Volume 29, ISSUE 12, P1792-1801, December 2022

Download started.

Ok

A Combination of Radiomic Features, Imaging Characteristics, and Serum Tumor Biomarkers to Predict the Possibility of the High-Grade Subtypes of Lung Adenocarcinoma

  • Author Footnotes
    # These authors contributed equally to this work
    Yuanqing Liu
    Footnotes
    # These authors contributed equally to this work
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Yue Chang
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Xinyi Zha
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Jiayi Bao
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Qian Wu
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Author Footnotes
    # These authors contributed equally to this work
    Hui Dai
    Footnotes
    # These authors contributed equally to this work
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China

    Institute of Medical Imaging, Soochow University, Suzhou, Jiangsu Province, P.R. China

    Suzhou Key Laboratory of Intelligent Medicine and Equipment, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Chunhong Hu
    Correspondence
    Address correspondence to:
    Affiliations
    Department of Radiology, The First Affiliated Hospital of Soochow University, Suzhou, Jiangsu Province, P.R. China

    Institute of Medical Imaging, Soochow University, Suzhou, Jiangsu Province, P.R. China

    Suzhou Key Laboratory of Intelligent Medicine and Equipment, Suzhou, Jiangsu Province, P.R. China
    Search for articles by this author
  • Author Footnotes
    # These authors contributed equally to this work
Open AccessPublished:March 26, 2022DOI:https://doi.org/10.1016/j.acra.2022.02.024

      Rationale and Objectives

      Lung adenocarcinomas (LADC) containing high-grade subtypes have a poorer prognosis. And some studies have shown that high-grade subtypes have been identified as an independent predictor of local recurrence in patients treated with limited resection. The aim of this study was to construct a combined model based on radiomic features, imaging characteristics and serum tumor biomarkers to predict the possibility of preoperative high-grade subtypes.

      Materials and Methods

      156 patients with LADC were retrospectively recruited in this study. These patients were randomly divided into training and validation cohorts. Radiomics features and imaging characteristics were extracted from plain CT images. A nomogram was developed in a training cohort by univariate and multivariate logistic analysis, and its performance was evaluated by receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA) in the training and validation cohorts.

      Results

      A total of 1316 radiomic features were extracted from the lesions in plain chest CT images. After applying the mRMR algorithm and the LASSO regression, 4 features were retained. Based on these radiomic features, Radiomic score (Radscore) was calculated for each patient. Spiculation, air bronchogram sign, CYFRA 21-1 and Radscore had been used in the construction of the combined model. The AUC of the combined model was respectively 0.88 (95% CI, 0.82-0.95) and 0.94 (95% CI, 0.86-1.00) in the training and validation cohorts.

      Conclusion

      The combined model based on CT images and serum tumor biomarkers, can predict the high-grade subtypes of LADC in a non-invasive manner, which may influence individual treatment planning, such as the choice of surgical approach and postoperative adjuvant therapy.

      Key Words

      Abbreviations:

      LADC (lung adenocarcinoma), CT (computed tomography), DCA (decision curve analysis), ROI (region of interesting, Radscore radiomic score), AUC (area under curve), ROC (receiver operating characteristic), mRMR (minimum redundancy), LASSO (least absolute shrinkage and selection operator)

      INTRODUCTION

      Lung cancer remains the leading cause of cancer-related deaths, with lung adenocarcinoma (LADC) being the most common type of pathology (
      • Bray F
      • Ferlay J
      • Soerjomataram I
      • et al.
      Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
      ). As a result of the presence of multiple histological subtypes of LADC, different histological subtypes are also highly heterogeneous in terms of molecular, pathology, prognosis and treatment modalities Therefore, the International Association for the Study of Lung Cancer (IASLC), the American Thoracic Society (ATS), and the European Respiratory Society (ERS) released the new histological classification of LADC (
      • Travis WD
      • Brambilla E
      • Noguchi M
      • et al.
      International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
      ). In this classification, invasive LADCs can be stratified according to malignant aggressiveness as low grade (lepidic predominant invasive LADC), intermediate grade (papillary and acinar predominant invasive LADC), and high-grade (solid and micropapillary predominant invasive LADC) based on a study of prognosis according to histological subtypes (
      • Russell PA
      • Wainer Z
      • Wright GM
      • et al.
      does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new international association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary lung adenocarcinoma classification.
      ,
      • Yoshizawa A
      • Motoi N
      • Riely GJ
      • et al.
      Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases.
      ).
      Those tumors classified as a predominant of high-grade subtypes have a poorer prognosis with a widely demonstrated propensity for recurrence and metastasis (
      • Yuan Y
      • Zhu L
      • Jiang M
      • et al.
      Clinical impacts of a micropapillary pattern in lung adenocarcinoma: a review.
      ,
      • Leeman JE
      • Rimner A
      • Montecalvo J
      • et al.
      Histologic subtype in core lung biopsies of early-stage lung adenocarcinoma is a prognostic factor for treatment response and failure patterns after stereotactic body radiation therapy.
      ,
      • Qian F
      • Yang W
      • Wang R
      • et al.
      Prognostic significance and adjuvant chemotherapy survival benefits of a solid or micropapillary pattern in patients with resected stage IB lung adenocarcinoma.
      ). Even a small percentage of high-grade subtypes have been reported to have a significant prognostic impact on survival (
      • Lee G
      • Lee HY
      • Jeong JY
      • et al.
      Clinical impact of minimal micropapillary pattern in invasive lung adenocarcinoma.
      ,
      • Wang Y
      • Zheng D
      • Zheng J
      • et al.
      Predictors of recurrence and survival of pathological T1N0M0 invasive adenocarcinoma following lobectomy.
      ). This poor prognostic impact of high-grade subtypes can influence treatment planning. As in patients treated with limited resection, the high-grade subtypes has been identified as an independent predictor of local recurrence (
      • Nitadori J
      • Bograd AJ
      • Kadota K
      • et al.
      Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2cm or smaller.
      ,
      • Hung J
      • Yeh Y
      • Jeng W
      • et al.
      Prognostic factors of survival after recurrence in patients with resected lung adenocarcinoma.
      ). Therefore, preoperative diagnosis of LADC with high-grade subtypes is crucial for appropriate surgical planning. However, preoperative histology using biopsy is only a fraction of the heterogeneous tumor obtained. And the lesions are not fully characterized (
      • Huang KY
      • Ko PZ
      • Yao CW
      • et al.
      Inaccuracy of lung adenocarcinoma subtyping using preoperative biopsy specimens.
      ). Therefore, preoperative CT data may be useful to stratify patients and thus guide further adjuvant therapy or monitoring of patients with LADC. Some previous imaging characteristics have been proposed to help identify different pathological subtypes and stratify patients (
      • Lederlin M
      • Puderbach M
      • Muley T
      • et al.
      Correlation of radio- and histomorphological pattern of pulmonary adenocarcinoma.
      ). However, these radiological characteristics, such as marginal configuration and solidity, still show a large overlap in subtypes, the performance of subtyping based on these features has not been fully evaluated, and there is a large interobserver variability. In addition, serum tumor biomarkers are non-invasive diagnostic tools for the identification of malignancies and are commonly used in the screening of cancer as an indicator of prognostic factors and therapeutic efficacy (
      • Nakamura H
      • Nishimura T.
      History, molecular features, and clinical importance of conventional serum biomarkers in lung cancer.
      ). Radiomics refers to the extraction and analysis of large numbers of advanced quantitative imaging features from medical images that reflect the spatial distribution of radioactivity within a tumor (
      • Lambin P
      • Rios-Velazquez E
      • Leijenaar R
      • et al.
      Radiomics: extracting more information from medical images using advanced feature analysis.
      ). These features can be used in clinical decision support systems to improve the accuracy of diagnosis, prognosis prediction, especially in LADC (
      • Kumar V
      • Gu Y
      • Basu S
      • et al.
      Radiomics: the process and the challenges.
      ). Developing such a quantitative approach and testing its effectiveness could provide a new non-invasive and convenient way to better define treatment strategies, resulting in optimized clinical and economic benefits for patients (
      • Weng Q
      • Hui J
      • Wang H
      • et al.
      Radiomic feature-based nomogram: a novel technique to predict EGFR-activating mutations for EGFR Tyrosin kinase inhibitor therapy.
      ).
      In this study, a combined model based on radiomic features, imaging characteristics and serum tumor biomarkers was constructed to predict the possibility of the presence of preoperative high-grade subtypes based on selected risk factors. Self-sampling internal validation and calibration curves were used to validate the combined model. Decision curve analysis (DCA) was applied to evaluate the value of the model in clinical practice.

      MATERIALS AND METHODS

      Patients

      This study retrospectively recruited 156 patients who underwent resection for invasive LADC at the First Affiliated Hospital of soochow University from January 2016 to December 2020. All patients with LADC were completely randomized in a 7:3 ratio into a training cohort (n=109) and a validation cohort (n=46). The study was performed in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the institutional review board of the First Affiliated Hospital of soochow University. Due to its retrospective nature, the requirement for informed consent was waived.
      The inclusion criteria were (
      • Bray F
      • Ferlay J
      • Soerjomataram I
      • et al.
      Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
      ) patients who had undergone chest CT within 7 days before surgery; (
      • Travis WD
      • Brambilla E
      • Noguchi M
      • et al.
      International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
      ) available CT with 1 mm or 1.25 mm slice thickness; (
      • Russell PA
      • Wainer Z
      • Wright GM
      • et al.
      does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new international association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary lung adenocarcinoma classification.
      ) complete pathology report describing the pathological subtype with pathologically and confirmed invasive LADC; (
      • Yoshizawa A
      • Motoi N
      • Riely GJ
      • et al.
      Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases.
      ) no history of chemotherapy or radiotherapy; and (
      • Yuan Y
      • Zhu L
      • Jiang M
      • et al.
      Clinical impacts of a micropapillary pattern in lung adenocarcinoma: a review.
      ) clinical data containing CEA, CA724 and CYFRA 21-1 tumor indicators within 7 days before surgery. And the exclusion criteria were (
      • Bray F
      • Ferlay J
      • Soerjomataram I
      • et al.
      Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
      ) patients with multifocal lesions; (
      • Travis WD
      • Brambilla E
      • Noguchi M
      • et al.
      International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
      ) tissue sample obtained by biopsy rather than surgery; and (
      • Russell PA
      • Wainer Z
      • Wright GM
      • et al.
      does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new international association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary lung adenocarcinoma classification.
      ) the concomitant presence of other malignancies.

      Histologic Evaluation

      Surgically resected specimens were fixed in formalin, embedded in paraffin, sectioned with a microtome, and stained with hematoxylin and eosin (H&E). All available H&E-stained tumor slides were independently reviewed by two different pathologists. The extent of all five growth patterns were recorded according to the 2015 World Health Organization classification of lung tumors. Histopathologically, lepidic pattern consists of tumor cells with lepidic growth along alveolar walls and with no evidence of stromal, vascular, or pleural invasion. Acinar pattern consists of cuboidal or columnar tumor cells arranged in acini and tubules. Papillary pattern consists of papillae structures with a fibrovascular core and complicated secondary and tertiary branches. Micropapillary pattern consists of small papillary tufts composed of tumor cells with peripheral nuclei and no fibrovascular core. Solid pattern consists of polygonal tumor cells that lack acini, tubules, and papillae with mucin production (
      • Austin JHM
      • Garg K
      • Aberle D
      • et al.
      Radiologic implications of the 2011 classification of adenocarcinoma of the lung.
      ).

      Image Acquisition

      The chest CT images were obtained with multi-row spiral CT scanners (GE, Siemens, and Philips Healthcare systems). Details regarding the acquisition parameters were set as follows: detector collimation, 1 to 1.25 mm; field of view, 20 to 38 cm; beam pitch, 0.800 to 1.396; beam width, 10 to 40 mm; gantry speed, 0.5 or 0.8 seconds per rotation; 100 to 130 kV; 47 to 351 mA; reconstruction interval, 0.39 to 0.6 mm; and matrix, 512 × 512 mm. All CT data were acquired in the supine position at full inspiration. The scans ranged from the lung base to the level of the thoracic inlet. The patients’ CT data were downloaded from the Picture Archiving and Communication Systems (PACS).

      Patient Data

      The plain CT images were analyzed independently by 2 radiologists with 5 and 10 years of diagnostic experience, respectively, blinded to the clinical and histological findings. Disagreements were resolved by the third radiologist who had 20 years of experience. The image features included the following (
      • Bray F
      • Ferlay J
      • Soerjomataram I
      • et al.
      Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
      ): air bronchogram sign (present/absent) (
      • Travis WD
      • Brambilla E
      • Noguchi M
      • et al.
      International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
      ): pleural retraction (present/absent) (
      • Russell PA
      • Wainer Z
      • Wright GM
      • et al.
      does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new international association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary lung adenocarcinoma classification.
      ): lobulation (present/absent) and (
      • Yoshizawa A
      • Motoi N
      • Riely GJ
      • et al.
      Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases.
      ): spiculation (present/absent). In addition, clinical indicators and serum tumor biomarkers were included, including age, sex, smoking status, CEA, CYFRA 21-1 and CA724.

      Tumor Segmentation and Feature Extraction

      Three-dimensional region of interesting (3d-ROI) of lesion was obtained semi-manually by ITK-snap software (version 3.4.0, www.itksnap.org) for each patient. All images were segmented and adjusted by a radiologist with 10 years of experience blinded to the clinical and histological findings.
      Radiomic features were extracted from the 3D-ROI by commercial software FeAture Explorer (FAE) which developed by GE Healthcare (
      • Song Y
      • Zhang J
      • Zhang Y
      • et al.
      FeAture Explorer (FAE): A tool for developing and comparing radiomics models.
      ). Radiomics features, including features types and transform types, were extracted from each subject. The features types included first order, shape, gray-level cooccurrence matrix (GLCM), gray-level size zone matrix (GLSZM), gray-level run length matrix (GLRLM), gray-level dependence matrix (GLDM), and neighboring gray tone difference matrix (NGTDM). And the transform types included log, wavelet, and local binary pattern (LBP). With the exception of shape, all radiomic features are a combination of features types and transform types. The framework is presented in Figure 1.
      Figure 1
      Figure 1Framework for the construction of prediction model of the high-grade subtypes (micropapillary and solid) in patients with LADC. (Color version of figure is available online.)

      Statistical Analysis & Prediction Model Construction

      All radiomic feature-based prediction model construction and statistical analysis were performed using R (Version 4.1.0). The Kolmogorov-Smirnov test was used to test the normality of the continuously distributed data, and those that conformed to the normal distribution were expressed as mean (SD). The t-test for independent samples was used for comparison between two groups. The data that did not conform to the normal distribution were expressed as median [IQR], and the Mann-Whitney U test was used for the comparison between the two groups. The categorical variables were expressed as the number of cases, and the χ² test or Fisher's exact probability method was used for comparison between the two groups. The area under curve (AUC), the sensitivity and specificity of the receiver operating characteristic (ROC) curves were calculated according to the maximum value of Youden index. p<0.05 was considered a statistically significant difference.
      A radiomics feature model was built based on features selected from the training cohort. Z-score was applied to feature normalization before feature selection. Two methods, maximum relevance minimum redundancy (mRMR) and least absolute shrinkage and selection operator (LASSO), were used to select the features. At first, mRMR was performed to eliminate the redundant and irrelevant features, 20 features were retained. Then LASSO was conducted to choose the optimized subset of features to construct the final model. After the number of features was determined, the most predictive subset of features was selected and the corresponding coefficients were evaluated. Next, we built a model with the selected radiomic features. Radiomic score (Radscore) was calculated for each patient by linearly combining the selected features, weighted by their respective coefficients. Radscore=1iβiXi, Xi is the radiomic feature selected by LASSO, βi is the corresponding coefficient.
      The multivariate logistic regression was used to fit the Radscore, clinical characteristics and imaging characteristics of the training cohort to construct a combined model. In addition, a clinical model was developed. The discriminative performance of the model was analyzed using ROC curves. The calibration curve of the nomogram was used to assess how closely the nomogram predicted the high-grade subtypes relative to the actual probability. The Hosmer-Lemeshow test was used to evaluate the goodness-of-fit of the calibration curve. Finally, decision curve analysis (DCA) was used to evaluate the clinical usefulness of the model. DCA are used to derive the net benefit of the model at different threshold probabilities obtained from information weighting the relative harms of false positive and false negative predictions (
      • Vickers AJ
      • Elkin EB.
      Decision curve analysis: a novel method for evaluating prediction models.
      ).

      RESULTS

      Basic Data

      In this study, 66 patients (42.3%) were diagnosed as LADC with high-grade subtypes. Two group (0 for those without high-grade subtypes and 1 for those with high-grade subtypes) have been designed and evaluated. There was no statistical difference between the training and control cohort for each variable (Table 1).
      Table 1Baseline Characteristics of Patients with And Without the High-Grade Subtypes.
      VariableTraining (n=109)Test (n=46)p-value
      age60.6 (
      • Nitadori J
      • Bograd AJ
      • Kadota K
      • et al.
      Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2cm or smaller.
      )
      60.3 (11.5)0.867
      sex (female /male)66/4328/181.000
      smoking

      (never/ active)
      88/2140/60.483
      lobulation

      (absent/ presence)
      33/768/380.144
      spiculation

      (absent/ presence)
      40/6920/260.541
      pleural retraction

      (absent/ presence)
      44/6521/250.666
      encapsulated air

      (absent/ presence)
      54/5518/280.312
      CYFRA 2112.1 (1.5)2.3 (2.2)0.474
      CEA14.1 (78.2)8.3 (12.6)0.618
      CA7242.6 (4.4)2.5 (2.5)0.94
      Radscore-0.2 [-1.4, 0.8]0.3 [-1.3, 0.8]0.542
      CYFRA21-1, Cytokeratin 19-fragments; CEA, Carcinoembryonic antigen; CA724, carbohydrate antigen 724.

      Univariate Analysis & Multivariate Logistic Analysis

      After univariate logistic regression analysis of the patients’ clinical data, the following factors were proved to be significantly associated with the high-grade subtypes of LADC in training cohort (p < 0.05, Table 2): spiculation, air bronchogram sign, and CYFRA 21-1. No significant relationships were found between age, sex, smoking status, lobulation, pleural retraction, CEA and CA724 with the high-grade subtypes in LADC (p > 0.05). Multivariate logistic regression analysis was applied to the risk factors selected by univariate logistic regression. The results showed that spiculation (OR= 1.46; 95% CI: 0.55∼2.45), air bronchogram sign (OR=-1.21; 95% CI: -2.14∼-0.34) and CYFRA 21-1 (OR=0.64; 95% CI: 0.18-1.21) were independent risk factors for the high-grade subtypes of LADC in the training cohort (Fig 2).
      Table 2Univariate Analysis of Clinical Features, Imaging Characteristics and Serum Tumor Biomarkers That Related to The High-Grade Subtypes.
      VariableLabel = 0 (n=61)Label = 1 (n=48)p-value
      age59.3 (10.5)62.2 (9.1)0.125
      sex (female/male)39/2227/210.537
      smoking

      (never/ active)
      52/936/120.271
      lobulation

      (absent/ presence)
      20/4113/350.665
      spiculation

      (absent/ presence)
      31/309/390.001*
      pleural retraction

      (absent/ presence)
      27/3417/310.461
      encapsulated air

      (absent/ presence)
      24/3730/180.027*
      CYFRA 2111.7 (0.8)2.6 (
      • Travis WD
      • Brambilla E
      • Noguchi M
      • et al.
      International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
      )
      0.002*
      CEA6.4 (24.2)24 (114.6)0.242
      CA7242.6 (3.3)2.6 (5.6)0.924
      Radscore-1.1 [-2.1, -0.3]0.8 [0.3, 1.2]< 0.001*
      Figure 2
      Figure 2Multivariate analysis of imaging characteristics and serum tumor biomarkers that related to the high-grade subtypes in patients with LADC (Color version of figure is available online.)

      Radiomics Model

      A total of 1,316 radiomic features were extracted from the lesions in plain CT of the lungs, divided into 7 groups: 252 first order, 14 shape, 336 GLCM, 224 GLSZM, 224 GLRLM, 196 GLDM, and 70 NGTDM. After applying the mRMR algorithm, 20 features were retained. To reduce the appearance of overfitting of the results, logistic regression was used to determine an optimal λ value. In logistic regression, after the value of λ reaches a certain size, even changing the value of λ does not improve the model performance significantly, and the value of λ corresponds to a model with good performance and the least number of independent variables. Then, the obtained λ values were put into the LASSO regression and four non-zero coefficient features (squareroot_firstorder_Sk, square_glrlm_RP, square_firstorder_Sk and squareroot_gldm_LDHGLE) were selected to construct the radiomics model. The final formula for the Radscore was as follows: Radscore = -0.815 * squareroot_firstorder_Sk + -0.163 * square_glrlm_RP + 0.243 * square_firstorder_Sk + 0.283 * squareroot_gldm_LDHGLE + -0.373 (Fig 3). The Radscore showed a significant difference between LADC patients present of high-grade subtypes and absent of high-grade subtypes in the training (p<0.001) and validation cohorts (p<0.001). Patients with high-grade subtype generally showed a higher Radscore (Fig 4). As shown in Figure 5 a&b, the model with only radiomic feature achieved an AUC of 0.88 in the training cohort and 0.92 in the validation cohort.
      Figure 3
      Figure 3(a) Logistic regression was used to determine an optimal λ value. The red dot is the mean of the deviations and the gray bar indicates the confidence interval of the deviations. The smaller the mean square error the better the model. The number above indicates the number of independent features still present in the model. The first dashed line shows the minimum value of the mean square error; the second dashed line marks the location of the lowest point of the doubled standard deviation, indicating the simplest model that can be obtained at the expense of the doubled standard deviation. (b) Each curve represents the trajectory of each independent variable coefficient, with the vertical coordinate being the value of the coefficient, the lower horizontal coordinate being log(λ), and the upper horizontal coordinate being the number of non-zero coefficients at that point in the model. The optimal λ value (0.06) selected by logistic regression was put into the LASSO regression, and 4 non-zero coefficients were selected to be used to construct the radiomic model. c Histogram showing the contribution of the selected features with their regression coefficients in the radiomic model construction. (Color version of figure is available online.)
      Figure 4
      Figure 4Difference in the Radscore between LADC patients with the high-grade subtypes and without the high-grade subtypes in training cohort (a) and validation cohort (b). p-values between with the high-grade subtypes group and without the high-grade subtypes group are placed in top of each image. The light green areas indicate the shape of the distribution of the data. Wider sections of the violin plot represent a higher probability that members of Radscore will take on the given value; the skinnier sections represent a lower probability. The bold horizontal in the middle of the box represents the median, the area inside the box represents the interquartile range, the thin black line outside the box represents the 95% confidence interval, and the external points are indicated as outliers. (Color version of figure is available online.)
      Figure 5
      Figure 5(a & b) Comparison of performance among the three developed models for the prediction of the high-grade subtypes in patients with LADC. ROC curves of clinical features alone, radiomic features alone and combined features in the training (a) and validation (b) cohorts. ‘Combined’ indicates combined features, ‘Radiomic’ indicates radiomic features alone, and ‘Clinical’ indicates clinical features alone. c&d The calibration curve for combined model evaluation. e The combined model chart of the factors selected by Multivariate analysis. The corresponding point of the feature is found on the corresponding axis in the figure, a vertical line is made at that point, and the intersection of the points axis is the score of the feature in the model. The scores are summed to get the total score, and then a vertical line is made on the total points axis in the same way, and the corresponding intersection point on the risk axis is the possibility of the existence of high-grade subtypes. (Color version of figure is available online.)

      Combined Model & DCA Validation

      Spiculation, air bronchogram sign, CYFRA 21-1 and Radscore had been used in the construction of the combined model. The contribution of the above variables to the combined model was shown in Figure 5e, with the length of the line segment reflecting the magnitude of the factor's contribution to the combined model. The combined model yielded an AUC of 0.88 (95% CI, 0.82-0.95 in the training cohort (Fig 5a) and an AUC of 0.94 (95% CI, 0.86-1.00) in the validation cohort (Fig. 5b). In the training cohort and the validation cohort, the differences between the AUC of the combined model and the clinical model were statistically significant. The calibration curve of the combine model was shown in Figure 5 C&D. In the training cohort, the predicting curve and the actual curve showed the consistency. The model was proved to be in compliance. Finally, The DCA analysis (Fig 6) showed that the nomograph had a higher overall net benefit in differentiating between without high-grade subtype and with high-grade subtype to the clinical and radiomic models within a reasonable threshold probability range.
      Figure 6
      Figure 6DCA for predicting the high-grade subtypes in patients with LADC for each model. 'Combined' indicates combined features, 'Radiomic' indicates radiomic features alone, and 'Clinical ' indicates clinical features alone. The horizontal line (black) indicates that all samples are negative, all samples are untreated, and the net benefit is 0. The diagonal line (purple) indicates that all samples are positive, all samples are treated, and the net benefit is backslash with a negative slope. (Color version of figure is available online.)

      DISCUSSION

      In studies related to disease-free survival (DFS) and overall survival (OS) in patients with LADC, micropapillary-dominant and solid-dominant patterns were predictive markers of benefit from adjuvant chemotherapy (
      • Wang C
      • Yang J
      • Lu M
      Micropapillary predominant lung adenocarcinoma in stage ia benefits from adjuvant chemotherapy.
      ,
      • Hung J
      • Wu Y
      • Chou T
      • et al.
      Adjuvant chemotherapy improves the probability of freedom from recurrence in patients with resected stage IB lung adenocarcinoma.
      ). This incentive behavior of high-grade LADC may also influence the surgical approach. Further studies have also reported that patients treated with limited resection have a higher risk of recurrence than similar patients treated with lobectomy when LADC contains high-grade subtype, suggesting that limited resection may not be the optimal surgical approach for LADC high-grade subtype (
      • Nitadori J
      • Bograd AJ
      • Kadota K
      • et al.
      Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2cm or smaller.
      ). However, in patients with poor cardiopulmonary function, elderly patients or patients with multiple lesions requiring multiple resections, the priority should be to preserve pulmonary function and avoiding unnecessary over-resection of the lung parenchyma. Therefore, preoperative knowledge of the high-grade subtype in LADC is important for optimal surgical planning and early selection of aggressive postoperative adjuvant therapy in patients with LADC with micropapillary and solid. It is important to determine the high-grade subtype even in inoperable LADC, as biopsy samples may not reflect all subtype characteristics of heterogeneous tumors (
      • Huang KY
      • Ko PZ
      • Yao CW
      • et al.
      Inaccuracy of lung adenocarcinoma subtyping using preoperative biopsy specimens.
      ). The full histological subtype can only be determined by evaluating the entire tumor specimen. Therefore, the diagnosis of the presence or absence of high-grade subtypes is essential for determining individualized treatment strategies. Recently, researchers have been searching for a way to identify pathological subtypes in LADC preoperatively. One potential approach to overcome this obstacle is the analysis of preoperatively obtained medical images.
      In this study, a model-based joint nomogram was developed and validated for preoperative individualized prediction of whether LADC contains a high-grade subtype. The nomogram integrates 3 clinical features, namely spiculation, air bronchogram and CYFRA 21-1, and four radiomic features. This nomogram combines imaging histology and clinical models to provide a visual presentation of the results and achieve a high AUC. According to this nomogram, LADC can be classified as having high-grade subtypes and no high-grade subtypes.
      In the limited studies on radiomic differentiation of lung cancer subtypes, Yang et al showed LADC with high purity pathological subtypes (≥ 70%) demonstrates strong stratification of radiomic values, with validation set accuracies of 83% and 94% in five subtypes and three pathological grades (
      • Yang S
      • Chen L
      • Wang H
      • et al.
      Extraction of radiomic values from lung adenocarcinoma with near-pure subtypes in the International Association for the Study of Lung Cancer/the American Thoracic Society/the European Respiratory Society (IASLC/ATS/ERS) classification.
      ). Park et al. demonstrated that radiomics could differentiate three pathological grades of the predominant subtypes in adenocarcinoma with comparable performance to radiologists (
      • Park S
      • Lee SM
      • Noh HN
      • et al.
      Differentiation of predominant subtypes of lung adenocarcinoma using a quantitative radiomics approach on CT.
      ). Not only is the high purity pathological subtypes or the predominant subtypes important for prognosis, but the presence of a small portion of high-grade subtypes also has a significant impact on DFS and OS in patients with LADC. In the current study, different proportions of the respective histological subtypes of LADC were included, which is more relevant to the clinical use scenario. Song et al reported that two radiomics features, lower value of the minimum of the whole pixel value and lower value of the variance of the positive pixel value, were determined to be predictive of a micropapillary component within LADC (
      • Song SH
      • Park H
      • Lee G
      • et al.
      Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma.
      ). Bae et al showed that radiomic features extracted from dual-energy CT data differentiated subtype groups with AUC values calculated from cross-validation for discriminating indolent, intermediate, and high-grade subtypes of 0.9307, 0.8610, and 0.8394, respectively (
      • Bae JM
      • Jeong JY
      • Lee HY
      • et al.
      Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy CT images.
      ). In contrast, our study not only used a radiomic approach but also included serum tumor biomarkers, clinical feature and imaging features, such as the spiculation, air bronchogram and CYFRA 21-1.
      Spiculation refers to the radiating, unbranched burr-like protrusions on the edge of the tumor, which is formed by the edema of the interlobular interval around the tumor or the infiltration of cancer cells around the small blood vessels, small lymphatic vessels, and small bronchial tubes around the tumor. The presence of the spiculation sign reflects active tumor growth with significant infiltrative growth. It can be used as a means of differentiating benign and malignant pulmonary nodules and has moderate diagnostic value (
      • Li Y
      • Wang T
      • Fu YF
      • et al.
      Computed tomography-based spiculated sign for prediction of malignancy in lung nodules: A meta-analysis.
      ). The air bronchogram sign is the result of tumor cells spreading along the wall of fine bronchus and alveolar wall in a volvulus-like growth pattern without destroying the lung scaffold structure, and the residual gas in the bronchus and alveoli is visualized (
      • Lederlin M
      • Puderbach M
      • Muley T
      • et al.
      Correlation of radio- and histomorphological pattern of pulmonary adenocarcinoma.
      ). In previous studies, the air bronchogram sign was correlated with low-grade subtypes and was useful for differentiating histological subtypes with different prognosis of LADC (
      • Li Q
      • Li X
      • Li XY
      • et al.
      Histological subtypes of solid-dominant invasive lung adenocarcinoma: differentiation using dual-energy spectral CT.
      ). CYFRA 21-1, a fragment of cytokeratin 19 which is responsible for the structural integrity of epithelial cells is overexpressed and detectable in the cytoplasm of several epithelial tumors including lung cancer. These are released in the blood because of cell lysis and tumor necrosis. It is positively correlated with disease stage, performance status, different pathological types of lung cancer, and a high CYFRA 21-1 level indicates worse prognosis (
      • Muley T
      • Dienemann H
      • Ebert W.
      Increased CYFRA 21-1 and CEA levels are negative predictors of outcome in p-stage I NSCLC.
      ). The preoperative serum CYFRA 21-1 is associated with pathological features such as tumor stage, tumor size and poor differentiation, and is an important independent prognostic factor for lung cancer. For example, elevated serum CYFRA 21-1 may reflect tumor necrosis, which is often caused by invasive growth (

      Chen H, Fu F, Zhao Y, et al. The prognostic value of preoperative serum tumor markers in non-small cell lung cancer varies with radiological features and histological types. FRONT ONCOL. 2021 2021-06-11;11:645159. doi:10.3389/fonc.2021.645159

      ,
      • Park SY
      • Lee JG
      • Kim J
      • et al.
      Preoperative serum CYFRA 21-1 level as a prognostic factor in surgically treated adenocarcinoma of lung.
      ,
      • Hanagiri T
      • Sugaya M
      • Takenaka M
      • et al.
      Preoperative CYFRA 21-1 and CEA as prognostic factors in patients with stage I non-small cell lung cancer.
      ) .Four radiomic features were incorporated into the final model, namely firstorder_Sk, glrlm_RP and gldm_LDHGLE. Sk is denoted as skewness and is used to measure the asymmetry of the distribution of values about the mean value (
      • van Griethuysen JJM
      • Fedorov A
      • Parmar C
      • et al.
      Computational radiomics system to decode the radiographic phenotype.
      ). RP stands for Run Percentage. RP measures the coarseness of the texture by the ratio of the number of runs to the number of voxels in the ROI (
      • van Griethuysen JJM
      • Fedorov A
      • Parmar C
      • et al.
      Computational radiomics system to decode the radiographic phenotype.
      ). This may be because the high-grade subtype of lung cancer exhibits active tumor cell growth and marked infiltration, and the high-grade subtype is often mixed with other tissue types, i.e., there is significant histologic heterogeneity (
      • Lederlin M
      • Puderbach M
      • Muley T
      • et al.
      Correlation of radio- and histomorphological pattern of pulmonary adenocarcinoma.
      ,
      • Yanagawa N
      • Shiono S
      • Abiko M
      • et al.
      New IASLC/ATS/ERS classification and invasive tumor size are predictive of disease recurrence in stage I lung adenocarcinoma.
      ). LDHGLE stands for Large Dependence High Gray Level Emphasis, which reflects the gray level relationship between the central voxel and its neighboring voxels, in other words it reflects the heterogeneity on the medical image (
      • van Griethuysen JJM
      • Fedorov A
      • Parmar C
      • et al.
      Computational radiomics system to decode the radiographic phenotype.
      ). This can be explained by the fact that greater CT imaging heterogeneity is often associated with more aggressiveness and poorer prognosis (
      • Bashir U
      • Siddique MM
      • Mclean E
      • et al.
      Imaging heterogeneity in lung cancer: techniques, applications, and challenges.
      ). In the present study, the diagnosis of high-grade subtypes of LADC had a high sensitivity in nomogram. This means that prior to resection of LADC, combining clinical features and imaging histological features extracted from preoperative CT data to calculate the presence of high-grade subtypes, the surgeon might change the surgical plan and choose the appropriate extent of resection (
      • Nitadori J
      • Bograd AJ
      • Kadota K
      • et al.
      Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2cm or smaller.
      ). Pathologists may be vigilant and work to find evidence of high-grade subtype during cryopathology analysis. This is because heterogeneity and misdiagnosis still occur occasionally in pathological practice despite clear criteria for subtype differentiation, as bias in tissue sampling leads to low diagnostic accuracy of the micropapillary type (
      • Trejo Bittar HE
      • Incharoen P
      • Althouse AD
      • et al.
      Accuracy of the IASLC/ATS/ERS histological subtyping of stage I lung adenocarcinoma on intraoperative frozen sections.
      ,
      • Warth A
      • Stenzinger A
      • von Brünneck A
      • et al.
      Interobserver variability in the application of the novel IASLC/ATS/ERS classification for pulmonary adenocarcinomas.
      ).
      Although a highly sensitive combined model was constructed, there are several limitations. First, this was a single-center retrospective study, and incomplete patient data collection resulted in a relatively small number of patients. In future work, the predictive model of this study should be validated in a large sample, prospective, multicenter study. Second, CT images with different acquisition protocols were used, and in further studies the impact of CT scanner variability and inconsistent acquisition parameters on radiomics feature extraction needs to be considered. Third, external validation using an independent population was not performed. However, cross-validation was used as the method of internal validation for this study.
      In conclusion, we developed a model based on CT images, combining clinical indicators and serum tumor biomarkers. This model can identify pathological subtypes of LADC in a non-invasive manner to predict the micropapillary and solid subtypes, which can be used to distinguish them from other LADC pathologies. The nomogram may provide additional diagnostic value in identifying the micropapillary and solid subtype, which may influence individual treatment planning, such as the choice of surgical approach and postoperative adjuvant therapy.

      References

        • Bray F
        • Ferlay J
        • Soerjomataram I
        • et al.
        Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
        CA Cancer J Clin. 2018; 68 (2018-11-01): 394-424https://doi.org/10.3322/caac.21492
        • Travis WD
        • Brambilla E
        • Noguchi M
        • et al.
        International association for the study of lung cancer/american thoracic society/european respiratory society international multidisciplinary classification of lung adenocarcinoma.
        J thorac oncol. 2011; 6: 244-285https://doi.org/10.1097/JTO.0b013e318206a221
        • Russell PA
        • Wainer Z
        • Wright GM
        • et al.
        does lung adenocarcinoma subtype predict patient survival?: A clinicopathologic study based on the new international association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary lung adenocarcinoma classification.
        J THORAC ONCOL. 2011; 6: 1496-1504https://doi.org/10.1097/JTO.0b013e318221f701
        • Yoshizawa A
        • Motoi N
        • Riely GJ
        • et al.
        Impact of proposed IASLC/ATS/ERS classification of lung adenocarcinoma: prognostic subgroups and implications for further revision of staging based on analysis of 514 stage I cases.
        MODERN PATHOL. 2011; 24: 653-664https://doi.org/10.1038/modpathol.2010.232
        • Yuan Y
        • Zhu L
        • Jiang M
        • et al.
        Clinical impacts of a micropapillary pattern in lung adenocarcinoma: a review.
        ONCOTARGETS THER. 2016; 9: 149https://doi.org/10.2147/OTT.S94747
        • Leeman JE
        • Rimner A
        • Montecalvo J
        • et al.
        Histologic subtype in core lung biopsies of early-stage lung adenocarcinoma is a prognostic factor for treatment response and failure patterns after stereotactic body radiation therapy.
        Int J Radiat Oncol*Biol*Physic. 2017; 97: 138-145https://doi.org/10.1016/j.ijrobp.2016.09.037
        • Qian F
        • Yang W
        • Wang R
        • et al.
        Prognostic significance and adjuvant chemotherapy survival benefits of a solid or micropapillary pattern in patients with resected stage IB lung adenocarcinoma.
        J Thorac Cardiovasc Surg. 2018; 155: 1227-1235https://doi.org/10.1016/j.jtcvs.2017.09.143
        • Lee G
        • Lee HY
        • Jeong JY
        • et al.
        Clinical impact of minimal micropapillary pattern in invasive lung adenocarcinoma.
        AM J SURG PATHOL. 2015; 39: 660-666https://doi.org/10.1097/PAS.0000000000000399
        • Wang Y
        • Zheng D
        • Zheng J
        • et al.
        Predictors of recurrence and survival of pathological T1N0M0 invasive adenocarcinoma following lobectomy.
        J Cancer Res Clin Oncol. 2018; 144 (2018-06-01): 1015-1023https://doi.org/10.1007/s00432-018-2622-8
        • Nitadori J
        • Bograd AJ
        • Kadota K
        • et al.
        Impact of micropapillary histologic subtype in selecting limited resection vs lobectomy for lung adenocarcinoma of 2cm or smaller.
        JNCI: J Nat Cancer Inst. 2013; 105 (2013-01-01): 1212-1220https://doi.org/10.1093/jnci/djt166
        • Hung J
        • Yeh Y
        • Jeng W
        • et al.
        Prognostic factors of survival after recurrence in patients with resected lung adenocarcinoma.
        J THORAC ONCOL. 2015; 10 (2015-01-01): 1328-1336https://doi.org/10.1097/JTO.0000000000000618
        • Huang KY
        • Ko PZ
        • Yao CW
        • et al.
        Inaccuracy of lung adenocarcinoma subtyping using preoperative biopsy specimens.
        J Thorac Cardiovasc Surg. 2017; 154 (2017-07-01): 332-339https://doi.org/10.1016/j.jtcvs.2017.02.059
        • Lederlin M
        • Puderbach M
        • Muley T
        • et al.
        Correlation of radio- and histomorphological pattern of pulmonary adenocarcinoma.
        EUR RESPIR J. 2013; 41: 943-951https://doi.org/10.1183/09031936.00056612
        • Nakamura H
        • Nishimura T.
        History, molecular features, and clinical importance of conventional serum biomarkers in lung cancer.
        Surgery today (Tokyo, Japan). 2017; 47 (2017-01-01): 1037-1059https://doi.org/10.1007/s00595-017-1477-y
        • Lambin P
        • Rios-Velazquez E
        • Leijenaar R
        • et al.
        Radiomics: extracting more information from medical images using advanced feature analysis.
        EUR J CANCER. 2012; 48 (2012-03-01): 441-446https://doi.org/10.1016/j.ejca.2011.11.036
        • Kumar V
        • Gu Y
        • Basu S
        • et al.
        Radiomics: the process and the challenges.
        MAGN RESON IMAGING. 2012; 30 (2012-11-01): 1234-1248https://doi.org/10.1016/j.mri.2012.06.010
        • Weng Q
        • Hui J
        • Wang H
        • et al.
        Radiomic feature-based nomogram: a novel technique to predict EGFR-activating mutations for EGFR Tyrosin kinase inhibitor therapy.
        FRONT ONCOL. 2021; 11 (2021-01-20)590937https://doi.org/10.3389/fonc.2021.590937
        • Austin JHM
        • Garg K
        • Aberle D
        • et al.
        Radiologic implications of the 2011 classification of adenocarcinoma of the lung.
        RADIOLOGY. 2013; 266: 62-71https://doi.org/10.1148/radiol.12120240
        • Song Y
        • Zhang J
        • Zhang Y
        • et al.
        FeAture Explorer (FAE): A tool for developing and comparing radiomics models.
        PLOS ONE. 2020; 15e237587https://doi.org/10.1371/journal.pone.0237587
        • Vickers AJ
        • Elkin EB.
        Decision curve analysis: a novel method for evaluating prediction models.
        MED DECIS MAKING. 2006; 26 (2006-01-01): 565-574https://doi.org/10.1177/0272989X06295361
        • Wang C
        • Yang J
        • Lu M
        Micropapillary predominant lung adenocarcinoma in stage ia benefits from adjuvant chemotherapy.
        ANN SURG ONCOL. 2020; 27 (2020-06-01): 2051-2060https://doi.org/10.1245/s10434-019-08113-0
        • Hung J
        • Wu Y
        • Chou T
        • et al.
        Adjuvant chemotherapy improves the probability of freedom from recurrence in patients with resected stage IB lung adenocarcinoma.
        Ann Thorac Surg. 2016; 101: 1346-1353https://doi.org/10.1016/j.athoracsur.2015.10.075
        • Yang S
        • Chen L
        • Wang H
        • et al.
        Extraction of radiomic values from lung adenocarcinoma with near-pure subtypes in the International Association for the Study of Lung Cancer/the American Thoracic Society/the European Respiratory Society (IASLC/ATS/ERS) classification.
        LUNG CANCER. 2018; 119: 56-63https://doi.org/10.1016/j.lungcan.2018.03.004
        • Park S
        • Lee SM
        • Noh HN
        • et al.
        Differentiation of predominant subtypes of lung adenocarcinoma using a quantitative radiomics approach on CT.
        EUR RADIOL. 2020; 30 (2020-01-01): 4883-4892https://doi.org/10.1007/s00330-020-06805-w
        • Song SH
        • Park H
        • Lee G
        • et al.
        Imaging phenotyping using radiomics to predict micropapillary pattern within lung adenocarcinoma.
        J THORAC ONCOL. 2017; 12: 624-632https://doi.org/10.1016/j.jtho.2016.11.2230
        • Bae JM
        • Jeong JY
        • Lee HY
        • et al.
        Pathologic stratification of operable lung adenocarcinoma using radiomics features extracted from dual energy CT images.
        Oncotarget. 2017; 8 (2017-01-03): 523-535https://doi.org/10.18632/oncotarget.13476
        • Li Y
        • Wang T
        • Fu YF
        • et al.
        Computed tomography-based spiculated sign for prediction of malignancy in lung nodules: A meta-analysis.
        Clin Respirat J. 2020; 14: 1113-1121https://doi.org/10.1111/crj.13258
        • Li Q
        • Li X
        • Li XY
        • et al.
        Histological subtypes of solid-dominant invasive lung adenocarcinoma: differentiation using dual-energy spectral CT.
        CLIN RADIOL. 2021; 76: 71-77https://doi.org/10.1016/j.crad.2020.08.034
        • Muley T
        • Dienemann H
        • Ebert W.
        Increased CYFRA 21-1 and CEA levels are negative predictors of outcome in p-stage I NSCLC.
        ANTICANCER RES. 2003; 23 (2003-01-01): 4085
      1. Chen H, Fu F, Zhao Y, et al. The prognostic value of preoperative serum tumor markers in non-small cell lung cancer varies with radiological features and histological types. FRONT ONCOL. 2021 2021-06-11;11:645159. doi:10.3389/fonc.2021.645159

        • Park SY
        • Lee JG
        • Kim J
        • et al.
        Preoperative serum CYFRA 21-1 level as a prognostic factor in surgically treated adenocarcinoma of lung.
        Lung cancer (Amsterdam, Netherlands). 2012; 79: 156-160https://doi.org/10.1016/j.lungcan.2012.11.006
        • Hanagiri T
        • Sugaya M
        • Takenaka M
        • et al.
        Preoperative CYFRA 21-1 and CEA as prognostic factors in patients with stage I non-small cell lung cancer.
        LUNG CANCER. 2011; 74: 112-117https://doi.org/10.1016/j.lungcan.2011.02.001
        • van Griethuysen JJM
        • Fedorov A
        • Parmar C
        • et al.
        Computational radiomics system to decode the radiographic phenotype.
        CANCER RES. 2017; 77: e104-e107https://doi.org/10.1158/0008-5472.CAN-17-0339
        • Yanagawa N
        • Shiono S
        • Abiko M
        • et al.
        New IASLC/ATS/ERS classification and invasive tumor size are predictive of disease recurrence in stage I lung adenocarcinoma.
        J thorac oncol. 2013; 8: 612-618https://doi.org/10.1097/JTO.0b013e318287c3eb
        • Bashir U
        • Siddique MM
        • Mclean E
        • et al.
        Imaging heterogeneity in lung cancer: techniques, applications, and challenges.
        AJR. Am J roentgenol. 2016; 207: 534-543https://doi.org/10.2214/AJR.15.15864
        • Trejo Bittar HE
        • Incharoen P
        • Althouse AD
        • et al.
        Accuracy of the IASLC/ATS/ERS histological subtyping of stage I lung adenocarcinoma on intraoperative frozen sections.
        Modern pathol. 2015; 28: 1058-1063https://doi.org/10.1038/modpathol.2015.71
        • Warth A
        • Stenzinger A
        • von Brünneck A
        • et al.
        Interobserver variability in the application of the novel IASLC/ATS/ERS classification for pulmonary adenocarcinomas.
        Euro respiratory J. 2012; 40: 1221-1227https://doi.org/10.1183/09031936.00219211