Estimating the Precision of Quantitative Imaging Biomarkers without Test-Retest Studies

      Rationale and Objectives

      A critical performance metric for any quantitative imaging biomarker is its ability to reliably generate similar values on repeat testing. This is known as the repeatability of the biomarker, and it is used to determine the minimum detectable change needed in order to show that a change over time is real change and not just due to measurement error. Test-retest studies are the classic approach for estimating repeatability; however, these studies can be infeasible when the imaging is expensive, time-consuming, invasive, or requires contrast agents. The objective of this study was to develop and test a method for estimating repeatability without a test-retest study.

      Materials and Methods

      We present a statistical method for estimating repeatability and testing whether an imaging method meets a specified criterion for repeatability in the absence of a test-retest study. The new method is applicable for the particular situation where a reference standard is available. A Monte Carlo simulation study was conducted to evaluate the performance of the new method.


      The proposed estimator is unbiased, and hypothesis tests with the new estimator have nominal type I error rate and power similar to a test-retest study. We considered the situation where the reference standard provides the true value, as well as when the reference standard itself has various magnitudes of measurement error. An example from CT imaging biomarkers of atherosclerosis illustrates the new method.


      Precision of a QIB can be measured without a test-retest study in the situation where a reference standard is available.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Kessler LG
        • Barnhart HX
        • Buckler AJ
        The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions.
        Stat Methods Med Res. 2015; 24: 9-26
        • Raunig DL
        • McShane LM
        • Pennello G
        • et al.
        Quantitative imaging biomarkers: a review of statistical methods for technical performance assessment.
        Stat Methods Med Res. 2015; 24: 27-67
        • Obuchowski NA
        • Reeves AP
        • Huang EP
        • et al.
        Quantitative imaging biomarkers: a review of statistical methods for computer algorithm comparisons.
        Stat Methods Med Res. 2015; 24: 68-106
        • Huang EP
        • Wang XF
        • Choudhury KR
        Meta-analysis of the technical performance of an imaging procedure: guidelines and statistical methodology.
        Stat Methods Med Res. 2015; 24: 141-174
        • Shukla-Dave A
        • Obuchowski NA
        • Chenevert TL
        • et al.
        Quantitative imaging biomarkers alliance (QIBA) recommendations for improved precision of DWI and DCE-MRI derived biomarkers in multicenter oncology trials.
        J Magn Reson Imaging. 2019; 49: 101-121
        • Obuchowski NA
        • Buckler A
        • Kinahan P
        Statistical issues in testing conformance with the quantitative imaging biomarker alliance (QIBA) profile claims.
        Acad Radiol. 2016; 4: 496-506
        • Obuchowski NA.
        Interpreting change in quantitative imaging biomarkers.
        Acad Radiol. 2018; 25: 372-379
        • Obuchowski NA
        • Bullen J.
        Quantitative imaging biomarkers: effect of sample size and bias on confidence interval coverage.
        Stat Methods Med Rese. 2018; 27: 3139-3150
      1. Quantitative Imaging Biomarker Alliance (QIBA) Wiki page Available at Accessed 07/12/2021.

        • Sheahan M
        • Ma X
        • Paik D
        • et al.
        Noninvasive quantitative assessment of characteristics with software-aided measurements from conventional CT angiography.
        Radiology. 2018; 286: 622-631
        • Barnhart HX
        • Barboriak DP.
        Applications of the repeatability of quantitative imaging biomarkers: a review of statistical analysis of repeat data sets.
        Transl Oncol. 2009; 2: 231-235
        • Obuchowski NA
        • Mozley DP
        • Matthews D
        • et al.
        Statistical considerations for planning clinical trials with quantitative imaging biomarkers.
        J Natl Cancer Inst. 2019; 111: 19-26
        • Matheson GJ.
        We need to talk about reliability: making better use of test-retest studies for study design and interpretation.
        Peer J. 2019; 24;7 ( PMID: 31179173; PMCID: PMC6536112): e6918