Multiparametric quantitative imaging biomarkers (QIBs) offer distinct advantages over
single, univariate descriptors because they provide a more complete measure of complex,
multidimensional biological systems. In disease, where structural and functional disturbances
occur across a multitude of subsystems, multivariate QIBs are needed to measure the
extent of system malfunction. This paper, the first Use Case in a series of articles
on multiparameter imaging biomarkers, considers multiple QIBs as a multidimensional
vector to represent all relevant disease constructs more completely. The approach
proposed offers several advantages over QIBs as multiple endpoints and avoids combining
them into a single composite that obscures the medical meaning of the individual measurements.
We focus on establishing statistically rigorous methods to create a single, simultaneous
measure from multiple QIBs that preserves the sensitivity of each univariate QIB while
incorporating the correlation among QIBs. Details are provided for metrological methods
to quantify the technical performance. Methods to reduce the set of QIBs, test the
superiority of the mp-QIB model to any univariate QIB model, and design study strategies
for generating precision and validity claims are also provided. QIBs of Alzheimer's
Disease from the ADNI merge data set are used as a case study to illustrate the methods
described.
Key Words
Abbreviations:
AD (Alzheimer's disease), ADNI (Alzheimer's Disease Neuroimaging Initiative), AI (Artificial intelligence), BLOQ (Below the lower limit of quantification), CCC (Concordance correlation coefficient), CFA (Confirmatory factor analysis), CN (Cognitively normal), CT (Computed tomography), DCE-MRI (Dynamic contrast enhanced MRI), DECT (Dual energy computed tomography), DFn (Kurtosis corrected degrees of freedom), DM (Mahalanobis distance), DM2 (Mahalanobis distance squared), DSC (Dynamic susceptibility contrast imaging), DTI (Diffusion tensor imaging), DX (X2-X1), E[X] (The expected value of X), EFA (Exploratory factor analysis), EMCI (Early mild cognitive impairment), FDA (Food and Drug Administration), FDG (Flurodeoxyglucose), GFI (Goodness of fit index), LLOQ (Lower limit of quantification), LMCI (Late mild cognitive impairment), LOA (Limits of Agreement), LOQ (Limit of quantification), MANOVA (Multivariate analysis of variance), MANOVA-RM (MANOVA Repeated measures), MAR (Missing at random), MCAR (Missing completely at random), MCCC (Multivariate CCC), MCD (Minimum covariance determinant), MCI (Mild cognitive impairment), mp-QIB (Multiparametric QIB), MRI (Magnetic resonance imaging), PET (Positron emission tomography), QIB (Quantitative Imaging Biomarker), QIBA (Quantitative Imaging Biomarker Alliance), RC (Repeatability coefficient), RCmp (Multiparametric RC), RDCmp (Multiparametric reproducibility coefficient), RMSEA (Root mean square error of approximation), S (Sample variance estimate), SAS® (Statistical analysis software)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Academic RadiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- QIBA Terminology Working Group. The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions.Stat Methods Med Res. 2015; 24: 9-26
- Quantitative Imaging Biomarkers Alliance (QIBA).2021 (Available at) (Accessed July 6, 2021)
- Quantitative imaging biomarkers: a review of statistical methods for technical performance assessment.Stat Methods Med Res. 2015; 24: 27-67
- Metrology standards for quantitative imaging biomarkers.Radiology. 2015; 277: 813-825
- Quantitative imaging biomarkers: a review of statistical methods for computer algorithm comparisons.Stat Methods Med Res. 2015; 24: 68-106
- Meta-analysis of the technical performance of an imaging procedure: guidelines and statistical methodology.Stat Methods Med Res. 2015; 24: 141-174
Committee QM. https://qibawiki.rsna.org/index.php/Metrology_Committee. Accessed June 22, 2022
- Multiparametric quantitative imaging in risk prediction: recommendations for data acquisition, technical performance assessment, and model development and validation.Acad Radiol. 2022; https://doi.org/10.1016/j.acra.2022.09.018
- Multiparametric quantitative imaging biomarkers for phenotype classification: a framework for development and validation.Acad Radiol. 2022; https://doi.org/10.1016/j.acra.2022.09.004
- A framework for evaluating the technical performance of multiparameter quantitative imaging biomarkers (mp-QIBs).Acad Radiol. 2022; https://doi.org/10.1016/j.acra.2022.08.031
- Multiparametric data-driven imaging markers: guidelines for development, application reporting of model outputs in radiomics.Acad Radiol. 2022; https://doi.org/10.1016/j.acra.2022.10.001
- Biomarkers and surrogate endpoints: preferred definitions and conceptual framework.Clin Pharmacol Ther. 2001; 69: 89-95
- Procedures for comparing samples with multiple endpoints.Biometrics. 1984; : 1079-1087
- Multiparametric assessment of changes in renal tissue after kidney transplantation with quantitative mr relaxometry and diffusion-tensor imaging at 3 T.J Clin Med. 2020; (9 2020/05/28)https://doi.org/10.3390/jcm9051551
- Imaging biomarker analysis of advanced multiparametric MRI for glioma grading.Phys Med. 2019; 60 (2019/03/27): 188-198https://doi.org/10.1016/j.ejmp.2019.03.014
- Efficacy endpoint selection and multiplicity adjustment methods in clinical trials with inherent multiple endpoint issues.Stat Med. 2003; 22: 3133-3150
- Recommendations for initial evaluation, staging, and response assessment of Hodgkin and non-Hodgkin lymphoma: the Lugano classification.J Clin Oncol. 2014; 32: 3059
- Effects of 5, 6-dimethylxanthenone-4-acetic acid on human tumor microcirculation assessed by dynamic contrast-enhanced magnetic resonance imaging.J Clin Oncol. 2002; 20: 3826-3840
- Report of the 6th International Workshop on PET in lymphoma.Leuk Lymphoma. 2017; 58: 2298-2303
- Multiparametric imaging of tumor response to therapy.Radiology. 2010; 256: 348-364
Bosca RJ. Methodological development of a multi-parametric quantitative imaging biomarker framework for assessing treatment response with MRI. 2014.
- Cardiac function after multimodal breast cancer therapy assessed with functional magnetic resonance imaging and echocardiography imaging.Int J Radiat Oncol* Biol* Phys. 2015; 93: 836-844
- Radiomics of coronary artery calcium in the framingham heart study.Radiol: Cardiothor Imag. 2020; 2e190119
- Assessing contractile function when ejection fraction is normal: a case for strain imaging.Am Heart Assoc. 2015; : 1-3
- Derivation and evaluation of age-specific multivariate reference regions to aid in identification of abnormal filling patterns: the HUNT and VaMIS studies.JACC: Cardiovasc Imag. 2018; 11: 400-408
- Carotid intima-media thickness measurements in intervention studies: design options, progression rates, and sample size considerations: a point of view.Stroke. 2003; 34: 2985-2994
- Spatio-temporal texture (SpTeT) for distinguishing vulnerable from stable atherosclerotic plaque on dynamic contrast enhancement (DCE) MRI in a rabbit model.Med Phys. 2014; 41042303
- Measurement accuracy of atherosclerotic plaque structure on CT using phantoms to establish ground truth.Acad Radiol. 2017; 24: 1203-1215
- Effectiveness of regional DTI measures in distinguishing Alzheimer's disease, MCI, and normal aging.NeuroImage: clin. 2013; 3: 180-195
- White matter hyperintensities correlate to cognition and fiber tract integrity in older adults with HIV.J Neurovirol. 2017; 23: 422-429
- Correlation of computed tomography and positron emission tomography in patients with metastatic gastrointestinal stromal tumor treated at a single institution with imatinib mesylate: proposal of new computed tomography response criteria.J Clin Oncol. 2007; 25: 1753-1759
- Intra-articular primatised anti-CD4: efficacy in resistant rheumatoid knees. a study of combined arthroscopy, magnetic resonance imaging, and histology.Ann Rheum Dis. 1999; 58: 342-349
- Multidimensional texture characterization: on analysis for brain tumor tissues using MRS and MRI.J Digit Imaging. 2014; 27: 496-506
- Plaque imaging volume analysis: technique and application.Cardiovasc Diagnos Ther. 2020; 10: 1032
- Atherosclerotic plaque tissue: noninvasive quantitative assessment of characteristics with software-aided measurements from conventional CT angiography.Radiology. 2018; 286: 622-631
- Role of imaging in the staging and response assessment of lymphoma: consensus of the International Conference on Malignant Lymphomas Imaging Working Group.J Clin Oncol. 2014; 32 (2014/08/11): 3048-3058https://doi.org/10.1200/JCO.2013.53.5229
- Clinical imaging of bone microarchitecture with HR-pQCT.Curr osteoporos rep. 2013; 11: 147-155
- Multidimensional imaging provides evidence for down-regulation of T cell effector function by MDSC in human cancer tissue.Sci immunol. 2019; 4 (p.eaaw9159)
- Multiple endpoints in clinical trials guidance for industry draft guidance.Washington, DC, FDA Maryland2017
- Biomarkers and surrogate endpoints in clinical trials.Stat Med. 2012; 31: 2973-2984
- Multiple biomarkers predict disease severity, progression and mortality in COPD.Respir Res. 2017; 18: 117https://doi.org/10.1186/s12931-017-0597-7
- Multiple co-primary endpoints: medical and statistical solutions a report from the multiple endpoints expert team of the pharmaceutical research and manufacturers of America.Drug Inf J. 2007; 41: 31
QIBA Wiki. Available at: http://qibawiki.rsna.org/index.php/Claim_Guidance. Accessed January 7, 2022.
- Multiple comparison adjustments when two or more co-primary endpoints must all be statistically significant.in: Proceedings of the Annual Meeting of the American Statistical Association, Chicago, IL. 1986
- Challenge of multiple co-primary endpoints: a new approach.Stat Med. 2007; 26: 1181-1192
- Discussion of “some controversial multiple testing problems in regulatory applications.J Biopharm Stat. 2009; 19: 14-21https://doi.org/10.1080/10543400802541719
- Multiple co-primary endpoints: medical and statistical solutions: a report from the multiple endpoints expert team of the Pharmaceutical Research and Manufacturers of America.Drug Inf J. 2007; 41: 31-46
- Gatekeeping procedures with clinical trial applications.Pharmaceut Stat. 2007; 6: 171-180
- The analysis of multiple endpoints in clinical trials.Biometrics. 1987; : 487-498
- Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review.BMJ. 2010; 18: 341
- World Health Organization and International League of Associations for Rheumatology core endpoints for symptom modifying antirheumatic drugs in rheumatoid arthritis clinical trials.J Rheumatol Suppl. 1994; 41: 86-89
- Quantifying skin disease burden in mycosis fungoides–type cutaneous T-cell lymphomas: the Severity-Weighted Assessment Tool (SWAT).Arch Dermatol. 2002; 138: 42-48
- Phase IIb multicenter trial of vorinostat in patients with persistent, progressive, or treatment refractory cutaneous T-cell lymphoma.J Clin Oncol. 2007; 25: 3109-3115
- Composite end points in clinical research: a time for reappraisal.Circulation. 2017; 135: 2299-2307
- Statistical evaluation of surrogate endpoints with examples from cancer clinical trials.Biomet J. 2016; 58: 104-132
- Desirability of outcome ranking (DOOR) and response adjusted for duration of antibiotic risk (RADAR).Clin Infect Dis. 2015; 61: 800-806
- Combining mortality and longitudinal measures in clinical trials.Stat Med. 1999; 18: 1341-1354
- Generalized pairwise comparisons of prioritized outcomes in the two-sample problem.Stat Med. 2010; 29: 3245-3257
- The win ratio: a new approach to the analysis of composite endpoints in clinical trials based on clinical priorities.Eur Heart J. 2012; 33: 176-182
- Evaluating treatment efficacy by multiple end points in phase II acute heart failure clinical trials: analyzing data using a global method.Circul: Heart Fail. 2012; 5: 742-749
- The Combined Assessment of Function and Survival (CAFS): a new endpoint for ALS clinical trials.Amyo lat scleros front deg. 2013; 14: 162-168
- Good studies evaluate the disease while great studies evaluate the patient: development and application of a desirability of outcome ranking endpoint for Staphylococcus aureus bloodstream infection.Clin Infect Dis. 2019; 68: 1691-1698
- DOOR/RADAR: A gateway into the unknown?.Clin Infect Dis. 2016; 62: 814-815
- Analysis of ordered composite endpoints.Stat Med. 2020; 39: 602-616
- Applied multivariate statistical analysis.Pearson London, UK2014
- Agreement of CAD features with expert observer ratings for characterization of pulmonary nodules in CT using the LIDC-IDRI database.Medical imaging 2009: computer-aided diagnosis. H. International Society for Optics and Photonics, Bellingham, WA2009: 72600
- Using radiomics-based modelling to predict individual progression from mild cognitive impairment to Alzheimer's disease.Eur J Nucl Med Mol Imaging. 2022; 15: 1-11
Little R, D'Agostino R, Dickersin K, et al. The prevention and treatment of missing data in clinical trials. pannel on handling missing data in clinical trials. 2010.
- Avoid the perils of using rounded data.J Pharm Biomed Anal. 2015; 115: 502-508
- An analysis of transformations.J Royal Stat Soc: Series B (Methodol). 1964; 26: 211-243
- CLSI Evaluation of detection capability for clinical laboratory measurement procedures; approved guideline. CLSI document EP17-A2.Clinical and Laboratory Standards Institute, Wayne, PA2012
- Missing value imputation in high-dimensional phenomic data: imputable or not, and how?.BMC Bioinf. 2014; 15 (2014/11/06): 346https://doi.org/10.1186/s12859-014-0346-6
- Multiple imputation: a primer.Stat Methods Med Res. 1999; 8: 3-15
- Statistical analysis with missing data.John Wiley & Sons, Hoboken, NJ2019
Wang Z, Akande O, Poulos J, et al. Are deep learning models superior for missing data imputation in large surveys? Evidence from an empirical comparison. arXiv preprint arXiv:210309316 2021.
- Lower limit of iron quantification using dual-energy CT—a phantom study.J Applied Clin Med Phys. 2021; 22: 299-307
- Sonography principles and instruments E-Book.Elsevier Health Sciences, St. Louis, MO2019
- Evaluation of post wall filter for doppler ultrasound systems. acoustical imaging.Springer, Dordrecht, Netherlands2008: 133-138
- How well quantified is the limit of quantification?.Epidemiology. 2010; : S10-S16
- Correlation coefficient estimation involving a left censored laboratory assay variable.Stat Med. 2001; 20: 2921-2933
- Methods for non-compartmental pharmacokinetic analysis with observations below the limit of quantification.Stat Biopharmaceut Res. 2021; 13: 59-70
- A bayesian approach for PK/PD modeling with PD data below limit of quantification.J Biopharm Stat. 2012; 22: 1220-1243https://doi.org/10.1080/10543406.2011.585441
- The ghosts of departed quantities: approaches to dealing with observations below the limit of quantitation.Stat Med. 2012; 31: 4280-4295
- How to deal with non-detectable and outlying values in biomarker research: Best practices and recommendations for univariate imputation approaches.Comprehens Psychoneuroendocrinol. 2021; 7100052
- The use of multiple imputation for data subject to limits of detection.Sri Lankan J Appl Stat. 2014; 5 (2014/12/15): 227-246https://doi.org/10.4038/sljastats.v5i4.7792
Nassiri V, Barnett H, Geys H, et al. BLOQ: impute and analyze data with observations below the limit of quantification, https://cran.r-project.org/web/packages/BLOQ/(2018). Accessed June 22, 2022
- Modeling observations with a detection limit using a truncated normal distribution with censoring.BMC Med Res Method. 2020; 20: 170https://doi.org/10.1186/s12874-020-01032-9
- Technical performance assessment of quantitative imaging in radiological device premarket submissions guidance for industry and food and drug administration staff.Center for devices and radiological health. Silver Spring, MD, FDA2022 (June 16)
- Advanced MRI methods for assessment of chronic liver disease.Am J Roentgenol. 2009; 193: 14-27https://doi.org/10.2214/AJR.09.2601
- Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis.Springer, New York, NY2015
Madukaife MS. Use of the theory of Euclidean distance in testing for multivariate normality with application to breast cancer diagnostic data. 2020.
- Normface: L2 hypersphere embedding for face verification.in: Proceedings of the 25th ACM international conference on Multimedia. 2017: 1041-1049
- On information and sufficiency.Ann Mathemat Stat. 1951; 22 (78): 79-86
- Tests of significance in multivariate analysis.Biometrika. 1948; 35: 58-79
- Mahalanobis distance.Wiley Stats Ref: Stat Ref Online. 2014; : 1110-1111
- Cont univariat distribut. 1994; 1: 451
- Art of computer programming, volume 2: Seminumerical algorithms.Addison-Wesley Professional, Boston, MA2014
- Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome.J Med imag. 2018; 5011018
- Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method.Endoscopy. 2021; 53: 893-901
- Technique standards for skin lesion imaging: a delphi consensus statement.JAMA Dermatol. 2017; 153: 207-213https://doi.org/10.1001/jamadermatol.2016.3949
- Utilization of multiparametric prostate magnetic resonance imaging in clinical practice and focal therapy: report from a Delphi consensus project.World J Urol. 2017; 35: 695-701
- Scale development: theory and applications.Sage publications, Thousand Oaks, CA2016
- Principal component analysis and exploratory factor analysis.Stat Methods Med Res. 1992; 1: 69-95
- A step-by-step approach to using SAS for factor analysis and structural equation modeling.Sas Institute, Cary, NC2013
- Structural equation modeling in practice: a review and recommended two-step approach.Psychol Bull. 1988; 103: 411
- A third moment adjusted test statistic for small sample factor analysis.Multi Behav Res. 2012; 47: 448-462https://doi.org/10.1080/00273171.2012.673948
- Asymptotically distribution-free methods for the analysis of covariance structures.Br J Math Stat Psychol. 1984; 37: 62-83
- On selecting a multiple comparison procedure for analysis of a clinical trial: fallback, fixed sequence, and related procedures.Stat Biopharmaceut Res. 2010; 2: 22-32
- Some controversial multiple testing problems in regulatory applications.J Biopharm Stat. 2009; 19: 1-11
- Nonparametric statistical methods.John Wiley & Sons, New York, NY2013
- Alzheimer's Disease Neuroimaging Initiative (ADNI): clinical characterization.Neurology. 2010; 74 (2009/12/30): 201-209https://doi.org/10.1212/WNL.0b013e3181cb3e25
- Sample size in factor analysis.Psychol Methods. 1999; 4: 84
Kohl M, Kolampally S. mpe: Multiple Primary Endpoints. 2017.
- Power analysis for cluster randomized trials with continuous co-primary endpoints.Biometrics. 2022; : 1-13
- A review of “methods of multivariate analysis”.Taylor & Francis, New York, NY2005: 558-559
- Minimum covariance determinant and extensions.Wiley Interdiscip Rev Comput Stat. 2018; 10: e1421
- Generalized inverse of a matrix and its applications. 1. University of California Press, Berkley, CA1972: 601-620 (Theory of Statistics)
- Generalized inverses: theory and applications.Springer Science & Business Media, New York, NY2003
- A fast algorithm for the minimum covariance determinant estimator.Technometrics. 1999; 41: 212-223
- Association of habitual physical activity with cardiovascular disease risk.Circ Res. 2020; : 1253-1260
- Matrix-based concordance correlation coefficient for repeated measures.Biometrics. 2011; 67: 1007-1016
- Robust tests for the equality of variances.J Am Statist Assoc. 1974; 69: 364-367
- Robust procedures for testing equality of covariance matrices.Biometrics. 1992; 1: 819-827
- Testing the equality of variance-covariance matrices the robust way.Commun Theory Methods. 1985; 14: 3033-3051
- 3rd Edition. The international vocabulary of metrology—basic and general concepts and associated terms (VIM. 200. JCGM, 2012: 2012
- Some tests for the equality of covariance matrices.J Stat Plan Inference. 2001; 94: 25-36
- A simple procedure for the comparison of covariance matrices.BMC Evol Biol. 2012; 12: 222https://doi.org/10.1186/1471-2148-12-222
- Variance of sample variance.Sec Survey Res Methods–JSM. 2008; 2: 1291-1293
- Some useful moment results in sampling problems.Am Stat. 2014; 68: 282-296
- Mahalanobis distance tractometry (MaD-Tract)–a framework for personalized white matter anomaly detection applied to TBI.Neuroimage. 2022; 260119475
- A joint model for multivariate longitudinal and survival data to discover the conversion to Alzheimer's disease.Stat Med. 2022; 41: 356-373
- Functional survival forests for multivariate longitudinal outcomes: dynamic prediction of Alzheimer's disease progression.Stat Methods Med Res. 2021; 30: 99-111
- BACE inhibition causes rapid, regional, and non-progressive volume reduction in Alzheimer's disease brain.Brain. 2020; 143: 3816-3826
- Magnetic resonance imaging measures of brain atrophy from the EXPEDITION3 trial in mild Alzheimer's disease.Alzheimer's Dement: Translat Res Clin Intervent. 2019; 5: 328-337
- Tracking pathophysiological processes in Alzheimer's disease: an updated hypothetical model of dynamic biomarkers.lancet neurol. 2013; 12: 207-216
Imaging & Data Archive at the Laboratory of Neuroimging (LONI). ADNI study data: ADNIMERGE, Available at: https://ida.loni.usc.edu/pages/access/studyData.jsp (Accessed June 23, 2022).
Article info
Publication history
Published online: December 01, 2022
Accepted:
October 29,
2022
Received in revised form:
October 24,
2022
Received:
June 2,
2022
Identification
Copyright
© 2022 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.
