Metrology Committee

Chairs: Nancy Obuchowski, PhD (Co-Chair); Erich Huang, PhD (Co-chair)

• Roster

Scope

The Metrology Committee supports QIBA Biomarker Committees by providing expertise in bio-statistics and metrology.

• Develop guidance documents
• Respond to committee questions

The Metrology Committee operated as the Multi-parametric Metrology Task Force under the QIBA Steering Committee.

QIBA Ground Truth Task Force

• 
• Report_GroundTruthTaskForce_March2021 -- recommendations provided
• Report_GroundTruthTaskForce_March2021 (slides)

Series 1:

Metrology papers

Under the direction of RSNA, three QIBA Metrology Working Groups were formed to discuss terminology for metrology concepts, algorithm comparisons and technical performance of an imaging assay.

The all-volunteer groups embraced the challenge to standardize across these concepts and reduce ambiguity regarding the terms and methodology used for measuring, describing and comparing various components of imaging tests.

The efforts of the QIBA Metrology Working Groups establish standardized terminology for use in research and clinical practice to meet the alliance’s overall goal to improve the value and practicality of quantitative imaging biomarkers by reducing variability across devices, patients, and time.

Based on the committees’ work, the following six manuscripts were published in either Radiology or Statistical Methods in Medical Research:

Sullivan DC, Obuchowski NA, Kessler LG, et al. Metrology Standards for Quantitative Imaging Biomarkers. Radiology. 2015 Aug 12. Epub ahead of print. doi: 10.1148/radiol.2015142202.

Kessler, LG, et. al., The Emerging Science of Quantitative Imaging Biomarkers Terminology and Definitions for Scientific Studies and Regulatory Submissions, Stat Methods Med Res 0962280214537333, first published on June 11, 2014 as doi:10.1177/0962280214537333

Raunig, DL, et. al., Quantitative Imaging Biomarkers: A Review of Statistical Methods for Technical Performance Assessment, Stat Methods Med Res 0962280214537344, first published on June 11, 2014 as doi:10.1177/0962280214537344

Obuchowski, NA, et. al., Quantitative Imaging Biomarkers: A Review of Statistical Methods for Computer Algorithm Comparisons, Stat Methods Med Res 0962280214537390, first published on June 11, 2014 as doi:10.1177/0962280214537390

Obuchowski, NA, et. al., Statistical Issues in the Comparison of Quantitative Imaging Biomarker Algorithms Using Pulmonary Nodule Volume as an Example, Stat Methods Med Res 0962280214537392, first published on June 11, 2014 as doi:10.1177/0962280214537392

Huang, EP, et. al., Meta-analysis of the Technical Performance of an Imaging Procedure: Guidelines and Statistical Methodology, Stat Methods Med Res 0962280214537394, first published on May 28, 2014 as doi:10.1177/0962280214537394

Series 2:

Abstract: Combinations of multiple quantitative imaging biomarkers (QIBs) are often able to predict the likelihood of an event of interest such as death or disease recurrence more effectively than single imaging measurements can alone. The development of such multiparametric quantitative imaging and evaluation of its fitness of use differs from the analogous processes for individual QIBs in several key aspects. A computational procedure to combine the QIB values into a model output must be specified. The output must also be reproducible and be shown to have reasonably strong ability to predict the risk of an event of interest. Attention must be paid to statistical issues not often encountered in the single QIB scenario, including overfitting and bias in the estimates of model performance.

These papers are a five-part series on statistical methodology for assessing the technical performance of multiparametric quantitative imaging. Considerations for data acquisition are discussed and recommendations from the literature on methodology to construct and evaluate QIB-based models for risk prediction are summarized. The findings in the literature upon which these recommendations are based are demonstrated through simulation studies. Concepts are applied to a real-life example involving prediction of major adverse cardiac events using automated plaque analysis.

1- Obuchowski, NA, Hall, TJ, Introduction to Multiparametric QIB Series, (2023). Academic Radiology, 30(2) 145-146. ISSN 1076-6332, https://doi.org/10.1016/j.acra.2022.07.008. (https://www.sciencedirect.com/science/article/pii/S1076633222004044)

2-Raunig, DL, Pennello, GA, Delfino, JG, Buckler, AJ, Hall, TJ, Guimaraes, AR, Obuchowski, NA. (2023). Multiparametric Quantitative Imaging Biomarker as a Multivariate Descriptor of Health: A Roadmap. Academic Radiology, 30(2), 159-182.

3- Delfino, JG, Pennello, GA, Barnhart, HX, Buckler, AJ, Wang, X, Huang EP, Raunig, DL, Guimaraes, AR, Hall, TJ, deSouza, NM, Obuchowski, NA. (2023) Multiparametric Quantitative Imaging Biomarkers for Phenotype Classification: A Framework for Development and Validation, Academic Radiology, 30(2) 183-195. ISSN 1076-6332, https://doi.org/10.1016/j.acra.2022.09.004. (https://www.sciencedirect.com/science/article/pii/S1076633222004950)

4- Huang, EP, Pennello, GA, deSouza, NM, Wang, X, Buckler, AJ, Kinahan, PE, Barnhart, HX, Delfino, JG, Hall, TJ, Raunig, DL, Guimaraes, AR, Obuchowski, NA. (2023). Multiparametric Quantitative Imaging in Risk Prediction: Recommendations for Data Acquisition, Technical Performance Assessment, and Model Development and Validation, Academic Radiology, 30(2), 196-214. ISSN 1076-6332, https://doi.org/10.1016/j.acra.2022.09.018. (https://www.sciencedirect.com/science/article/pii/S1076633222005098)

5-Wang, X, Pennello, GA, deSouza, NM, Huang, EP, Buckler, AJ, Barnhart, HX, Delfino, JG, Raunig, DL, Wang, L, Guimaraes, AR, Hall, TJ, Obuchowski, NA. (2023). Multiparametric Data-driven Imaging Markers: Guidelines for Development, Application and Reporting of Model Outputs in Radiomics, (2023) Academic Radiology, 30(2), 215-229, ISSN 1076-6332, https://doi.org/10.1016/j.acra.2022.10.001. (https://www.sciencedirect.com/science/article/pii/S1076633222005153)

Working Documents

• Claim Guidance
• Assessment Procedure Guidance

Current Work

• Multi-parametric biomarkers

Overview Slides

Meetings/Call Summaries

Metrology Committee Call Summaries Archive