ࡱ >
5- bjbjWW 3 5 5 A " M M [ [ [ ` d ` D 6a 6a 6a a < f 6a j o o o o }q ]{ L ~ } b Q j[ t Ç [q " }q ݈ ` [ [ o o e v < [ 8 o j[ o Ç & h T R[ L o p" 5 t d ` ,
L L ^ * Q 6 D _ d Q Q Q P Q Q Q Ç Ç Ç Ç Q Q Q Q Q Q Q Q Q M Y :
QIBA Profile:
Lung Nodule Volume Assessment in Evaluating Primary Lung Cancerand Monitoring in Lung Cancer Screening
Version 1.0
Date: Oct 16, 2013Status: draft
Table of Contents
TOC \o "1-3" \h \z \u HYPERLINK \l "_Toc323911085" 1. Executive Summary PAGEREF _Toc323911085 \h 6
HYPERLINK \l "_Toc323911086" 2. Clinical Context and Claims PAGEREF _Toc323911086 \h 7
HYPERLINK \l "_Toc323911089" 3. Profile Details PAGEREF _Toc323911089 \h 7
HYPERLINK \l "_Toc323911090" 3.1. Subject Handling PAGEREF _Toc323911090 \h 9
HYPERLINK \l "_Toc323911097" 3.2. Image Data Acquisition PAGEREF _Toc323911097 \h 12
HYPERLINK \l "_Toc323911098" 3.3. Image Data Reconstruction PAGEREF _Toc323911098 \h 15
HYPERLINK \l "_Toc323911099" 3.4. Image Analysis PAGEREF _Toc323911099 \h 17
HYPERLINK \l "_Toc323911100" 4. Compliance PAGEREF _Toc323911100 \h 21
HYPERLINK \l "_Toc323911101" 4.1. Performance Assessment: Tumor Volume Change Variability PAGEREF _Toc323911101 \h 21
HYPERLINK \l "_Toc323911102" 4.2. Performance Assessment: Image Acquisition Site PAGEREF _Toc323911102 \h 23
HYPERLINK \l "_Toc323911103" References PAGEREF _Toc323911103 \h 26
HYPERLINK \l "_Toc323911104" Appendices PAGEREF _Toc323911104 \h 29
HYPERLINK \l "_Toc323911105" Appendix A: Acknowledgements and Attributions PAGEREF _Toc323911105 \h 29
HYPERLINK \l "_Toc323911106" Appendix B: Background Information PAGEREF _Toc323911106 \h 30
HYPERLINK \l "_Toc323911107" Appendix C: Conventions and Definitions PAGEREF _Toc323911107 \h 46
HYPERLINK \l "_Toc323911108" Appendix D: Model-specific Instructions and Parameters PAGEREF _Toc323911108 \h 47
1. Executive Summary
X-ray computed tomography provides an effective means of detecting and monitoring pulmonary nodules, and can lead to a reduction in mortality in individuals at high risk for lung cancer. Size quantification on serial imaging is helpful in evaluating whether a pulmonary nodule is benign or malignant. Currently, pulmonary nodules most commonly are measured in two dimensions on axial slices. Investigators have suggested that quantifying whole nodule volume could solve some of the limitations of diameter measures [1-2] and many studies have explored the value of volumetry [3-12]. This document proposes standardized methods for performing repeatable volume measurements on CT images of pulmonary nodules in the setting of lung cancer screening and post-screening surveillance.
Lung cancer CT screening presents an additional challenge in developing an optimized protocol that balances the benefit of detecting and accurately characterizing lung nodules against the potential risk of radiation exposure in this asymptomatic population who may undergo annual screening for several decades. Our understanding of the extent to which performing scans at the lowest dose possible with the associated increase in noise impacts our ability to accurately measure these small nodules, is rapidly evolving. Therefore, any protocol will represent a compromise between these competing needsCT screening presents an additional challenge in developing an optimized protocol in that there is an imperative to balance the risks and harms in this asymptomatic population and in particular regarding performing scans at the lowest dose possible while still being able to detect the small nodules which make screening worthwhile. However, the extent to which the increased noise associated with the lower dose affects our ability to accurately measure these small nodules is rapidly evolving. Therefore, any protocol will represent a compromise between these various competing needs when performing screening
This QIBA Profile makes claims about the confidence with which changes in pulmonary nodule volume can be measured under a set of defined image acquisition, processing, and analysis conditions, and provides specifications that may be adopted by users and equipment developers to meet targeted levels of clinical performance in identified settings. While this Profile focuses on the lung cancer screening setting and the low radiation doses used therein, it also may be applied to nodules detected incidentally on CT examinations performed for other clinical indications, and nodules evaluated in the course of staging other cancers.
An additional area of focus that QIBA will make in regard to screening extends beyond the quantitative aspects of nodule measurements but also extends to developing a protocol that optimizes our ability to detect small nodules, both by the radiologist and using computer assisted methods.
The intended audiences of this document includes healthcare professionals and all other stakeholders invested in lung cancer screening, including but not limited to radiologists, technologists, and physicists designing CT acquisition protocols
Radiologists, technologists, and administrators at healthcare institutions considering specifications for procuring new CT equipment
Technical staff of software and device manufacturers who create products for this purpose
Biopharmaceutical companies
Oncologists
Clinicians engaged in screening process
Clinical trialists
Radiologists and other physicians health care providers making quantitative measurements on CT images
Regulators, oncologists, and others making decisions based on quantitative image measurements
Note that specifications stated as requirements in this document are only requirements to achieve the claim, not requirements on standard of care. Specifically, meeting the goals of this Profile is secondary to properly caring for the patient.
2. Clinical Context and Claims
These specifications are appropriate for performing low-dose CT screening with a view towards balancing the need of the radiologist to detect small nodules using low-dose technique and understanding the extent that these techniques influence our ability to measure small nodules. The primary objective is to evaluate their growth or regressionchange in volume over time with serially acquired CT scans and image processing techniques. Compliance with this Profile by relevant staff and equipment supports the following claim(s):
Claim: Measure Change in Nodule Volume
Suggest that first set of claims relates to being able to visualize nodules >= to 3mm and slice thickness necessary.
My concern (JLM) is that the lower we go the more variance. Why would we go below 5mm as this would not effect clinical management and it gives us better variance management?
Claim 2 relates to additional reconstruction series that should be made. This includes a series for improved radiologist visualization, and perhaps an additional series to allow optimized image processing.
CLAIM 1: Measure Volume Change in Small Nodules
The primary focus here is on small nodules which we define as having a diameter of < or = to 10 mm, and in the context of CT screening down to as small as 3(5) mm. This is the domain where other types of evaluation become challenging, in particular PET scan evaluation or semi-invasive procedures such as navigational bronchoscopy of fine needle aspiration. In addition, as there is an inverse relationship between nodule diameter and proportional change in volume, the proportional change in measurement error will therefore also follow an inverse relationship with diameter. In the lower size ranges, an increase in diameter of even 1 mm can result in the doubling and any error in measurement will therefore result in large proportional changes as well.
Based on a review of the available literature, which is somewhat limited, as well as being informed by a modeling study based on a calibration device that has been tested within a clinical trial and through a series of simulations, the following change thresholds for volume are recommended.
_ 5 m m ( 1 0 0 % c h a n g e )
_ 8 m m ( 3 0 3 5 % c h a n g e )
_ 1 0 1 1 m m ( 2 0 2 3 % c h a n g e )
_ A n y l e s i o n c h a n g e b e y o n d t h e s e p e r c e n t a g e s w o u l d r e f l e c t t r u e b i o l o g i c a l c h a n g e
a t a 9 5 % c o n f i d e n c e i n t e r v a l A m e a s u r e d v o l u m e c h a n g e g r e a t e r t h a n t h o s e p r o v i d e d a b o v e i m p l i e s at least a 95% probability that there is a true volume change; P (true volume change > 0% | measured volume change >___%) > 95%.
This claim holds when the margins of the nodule are sufficiently distinct from surrounding structures and geometrically simple enough to be segmented using automated software without manual correction.
For both claims, volume change refers to proportional change, where the percentage change is the difference in the two volume measurements divided by the average of the two measurements. By using the average instead of one of the measurements as the denominator, asymmetries in percentage change values are avoided.(Need to have a full discussion around this critical proposalJLM)
Procedures for claiming compliance to the Image Data Acquisition and Image Data Reconstruction activities have been provided (See Section 4). Procedures for claiming compliance to the Image Analysis activity are proposed in draft form and will be revised in the future.
For details on the derivation and implications of the Claim, refer to Appendix B.
While the claim has been informed by an extensive review of the literature as well as results from a modeling study, it is currently a proposedor workingconsensus claim that has not yet been fully substantiated by studies that strictly conform to the specifications given here. A standard utilized by a sufficient number of studies does not exist to date. The expectation is that during field test, data on the actual field performance will be collected and changes made to the claim or the details accordingly. At that point, this caveat may be removed or re-stated.
3. Profile Details
The Profile is documented in terms of Actors performing Activities.
Equipment, software, staff or sites may claim conformance to this Profile as one or more of the Actors in the following table. Compliant Actors shall support the listed Activities by meeting all requirements in the referenced Section. Failing to comply with a shall is a protocol deviation. Although deviations invalidate the Profile Claim, such deviations may be reasonable and unavoidable as discussed below.
Table 1: Actors and Required Activities
ActorActivitySectionAcquisition DeviceSubject Handling3.1.Image Data Acquisition3.2.TechnologistSubject Handling3.1.Image Data Acquisition3.2.Image Data Reconstruction3.3.RadiologistSubject Handling3.1.Image Analysis3.4.Reconstruction SoftwareImage Data Reconstruction3.3.Image Analysis ToolImage Analysis3.4.The sequencing of the Activities specified in this Profile is shown in Figure 1:
Figure SEQ Figure \* ARABIC 1: CT Tumor Volumetry - Activity Sequence
The method for measuring change in tumor volume may be described as a pipeline. Subjects are prepared for scanning, raw image data is acquired, images are reconstructed and possibly post-processed. Such images are obtained at two (or more) time points. Image analysis assesses the degree of change between two time points for each evaluable target nodule by calculating absolute volume at each time point and subtracting. Volume change is expressed as a percentage (volume difference between the two time points divided by the average of the volume at time point 1 and time point tvolume at the earlier time point).
The change may be interpreted according to a variety of different response criteria. These response criteria are beyond the scope of this document. Detection and classification of nodules are also beyond the scope of this document.
This initial e Profile, is expect to need revision as further innovation and validation data emerge. The above pipeline provides a reference model. Algorithms which achieve the same or a better result as compared to the reference model but use different methods are expected , for example by changing the size threshold for defining a significant nodule.directly measuring the change between two image sets rather than measuring the absolute volumes separately. The profile specifications requirements included herein are intended to establish a baseline level of capabilities. Providing higher performance or advanced capabilities is both allowed and encouraged. The Profile does not intend to limit how equipment suppliers meet these requirements.
This Profile is nodule-oriented. The Profile requires that images of a given nodule be acquired and processed the same way each time and all efforts should be made in the same fashionto achieve this goal. The requirements in this Profile do not codify a Standard of Care; they onlyprovide guidance intended to achieve the stated volumetric CT Claim. Although deviating from the specifications in this Profile may invalidate the Profile Claims, the radiologist or supervising physician is expected to do so when required by the best interest of the patient or research subject. How study sponsors and others decide to handle deviations for their own purposes is entirely up to them.
Since much of this Profile emphasizes performing subsequent scans consistent with the baseline scan of the subject, the parameter values chosen for the baseline scan are particularly significant and should be considered carefully documentedimportant. In some scenarios, the baseline might be defined as a reference point that is not necessarily the first scan of the patient.
3.1. Subject Handling
This Profile will refer primarilyto asymptomatic persons participating in a CT screening and surveillance program for lung cancer. If thTheis profileIn theory, this Profile also may be is also may be appliedcable to patients with known or incidentally-detected pulmonary nodules in whom quantitative volumetric assessment is used for characterization or response to therapyto determine volume change. , it is not clear that the claim statements will be accurate.
Subject handling guidelines are intended to reduce the likelihood that lung nodules will be obscured by surrounding disease or image artifacts, which could alter quantitative measurements, and to promote consistency of image quality on serial scans.
3.1.1 Timing of Scan
3.1.1.1 Timing Relative to Acute Cardiopulmonary Symptoms
Profile claims require the absence of acute or subacute abnormalities in the lungs that could interfere with or alter pulmonary nodule volume measurements, and the ability to cooperate fully with breath-holding instructions for scanning. Therefore, for initial screening, subjects should be asymptomatic, or at baseline if symptomatic, with respect to cardiac and pulmonary symptoms. If they are not asymptomatic or at baseline, postponement of initial screening until the subject returns to clinical baseline is recommendedpreferred. The screening setting mandates a Absence of symptoms or baseline clinical status as well as also are preferred atThese conditions also apply to the time of CT follow-up for a previous screen-detected abnormality. If these clinical status conditions cannot be met, such as due to the time-dependent nature of follow-up, the Profile claims may not be valid. Chronic abnormalities such as pulmonary fibrosis also may invalidate Profile claims if they affect nodule volume measurement accuracy.
3.1.1.2 Timing of Scan Relative to Other Procedures
Recent diagnostic or therapeutic procedures may result in parenchymal lung abnormalities that invalidate the claims of this Profile. Examples include bronchoscopy, thoracic or abdominal surgery, and radiation therapy. To meet Profile claims, scans should be performed prior to or at an appropriate time following such procedures.
Oral contrast administered for unrelated gastrointestinal imaging studies or abdominal CT that remains in the esophagus, stomach, or bowel may cause artifacts in certain areas of the lungs that interfere with quantitative nodule assessment. If oral contrast is present in the same transverse plane as a quantififiable lung nodule, the Profile claims may not be valid.
3.1.1.3 Specification
ParameterSpecificationPulmonary Symptoms If acute pulmonary symptoms are present, scanning should shall be delayed for a time period that allows resolution of potential reversible CT abnormalities. If scanning is necessary to avoid an excessive delay in follow-up of a known nodule or to evaluate new symptoms,then this is not lung cancer screening and should be considered a routine lung cancer diagnostic work-up, so and the nodule is obscured or an adequate level of inspiration is not achieved, measurements will not be subject to the Profile claims.
Medical ProceduresScanning should shall be performed prior to or at an appropriate time following procedures that could alter the attenuation of the lung nodule or surrounding lung tissue. If this specification is not met, and the attenuation of the lung or nodule is altered, Profile claims will not be valid.
3.1.2 Use of Intravenous Contrast
3.1.2.1 Discussion
Intravenous contrast is should not be used not recommended for CT screening. Because of the inherently high contrast between lung nodules and the surrounding parenchyma, contrast is unnecessary for nodule detection and quantification. Its use incurs additional cost, the potential for renal toxicity, and complicatesmay affect volumetric quantitation. In addition, contrast may alter the measured volume and other quantitative characteristics of a nodule, reducing the accuracy of comparison to non-contrast images. If contrast is administered, nodule measurements will not be subject to the Profile claims.
3.1.2.2 Specification
ParameterSpecificationUse of intravenous or oral contrast Intravenous contrast is not indicated recommendedfor lung cancer screening or follow-up of screen-detected nodules.
If the contrast is administered, quantitative nodule measurements will not be subject to the Profile claims.3.1.3 Subject Preparation
3.1.3.1 Discussion
It is recommended that subjects cough several times prior to CT scanning. This may help open small areas of atelectasis and improve the ability to inflate the lungs during breath holding. Coughing also may help clear mucus from the central airways, which may be difficult to distinguish from an endobronchial lesion.
Metallic objects on or within the thorax or upper abdomen may produce artifacts that reduce the conspicuity of pulmonary nodules or alter their attenuation, and. Radiodense metallic objects should be removed prior to scanning. including Examples include metal-containing shirts, bras, pants, or belts, necklaces and other jewelry, pins, EKG leads, and any other removable metallic objects. The topogram should be inspected, and if any previously unidentified metallic objects are present, they should be removed.
Internal metallic objects, such as pacemakers and spinal instrumentation, if in or near the scanned plane of a pulmonary nodule, also may produce artifacts that reduce the conspicuity of pulmonary nodules or alter their attenuation. If such artifacts occur, screening may still be performed, but the Claims of this Profile will not be met for nodules affected by metal artifacts, and the sensitivity for nodule detection may be reduced.
The effects of bismuth breast shields (used by some to reduce radiation exposure in the diagnostic CT setting but which increase image noise) on lung nodule quantification are unknown, but are likely to be magnified in the lung cancer screening setting due to the lower radiation dose used for screening. Their effects on image quality may vary depending on the model and their positioning on the chest, and their use could introduce another variable when assessing nodules for quantitative changes over time. The American Association of Physicists in Medicine currently does not endorse the use of breast shields, recommending the use of other dose reduction methods instead (ref). Thus, the use of breast shields is not consistent compatible with the Profile Claims and is not recommended for lung cancer screening.
3.1.3.2 Specification
ParameterSpecificationForced CoughingThe Technologist shall instruct the subject to cough forcefully several times before lying on the CT scanner table.Metallic ObjectsMetallic objects on or underneath the chest and abdomen shall be removed prior to scanning, and breast shields should not be used. The technologist shall inspect the topogram and remove any metal objects forgotten by the subject. Scanning may be performed if internal metallic objects are present, but resulting artifacts may invalidate Profile measurement claims.
3.1.4 Subject Positioning
3.1.4.1 Discussion
Consistent patient positioning is essentialimportant, to especially to avoidsreduce changes in attenuation due to changes in gravity induced shape and fluid distributionvariation in x-ray beam hardening and scatter and in anatomic nodule orientation. Ensuring that the chest (excluding the breasts) is in the center of the gantry throughout its length improves the consistency of relative attenuation values in different regions of the lung, and avoids unnecessary scan-to-scan variation in the behavior of dose modulation algorithms. The subject should be made comfortable, to reduce the potential for motion artifacts and to facilitate compliance with breath holding instructions.
To achieve these goals, subjects should be positioned supine with arms overhead, in keeping with standard clinical practice. Prone positioning creates the potential for unacceptable variance in volume quantitation. is not recommended. The chest, shoulders, and hips should be centered along the length of the table. The table height should be adjusted so that the midaxillary line is at the widest part of the gantry. The use of positioning wedges under the knees and head is recommended so that the lumbar lordosis is straightened and the scapulae are both in contact with the table. It is expected that local clinical practice and patient physical capabilities and limitations will influence patient positioning; an approach that promotes scan-to-scan consistency is essential recommended.
3.1.34.2 Specification
ParameterSpecificationMetallic ObjectsMetallic objects on or underneath the chest and abdomen shall be removed prior to scanning, and breast shields shall not be used. The technologist shall inspect the topogram and remove any metal objects forgotten by the subject.
Scanning may be performed if internal metallic objects are present, but resulting artifacts may invalidate Profile measurement claims.Subject PositioningThe Technologist shall position the subject supine with arms overhead, and the sternum centered along the length of the table. with use of devices such as positioning wedges as described above.Table Height & CenteringThe Technologist shall adjust the table height for the mid-axillary plane to pass through the isocenter of the gantry.
The Technologist shall position the patient such that the sagittal laser line lies along the sternum (e.g. from the suprasternal notch to the xiphoid process).
3.1.45 Instructions to Subject During Acquisition
3.1.45.1 Discussion
Scans should be performed during breath holding at maximal inspiration, for several reasons. Breath holding greatly reduces motion artifacts, which impair the quantitative assessment of lung nodules. Incomplete lung expansion can artificially increase the measured nodule volume (refs). Maximizing inspiratory volume also separates structures, making nodules more conspicuous, and minimizes atelectasis in the dependent portions of the lungs which can obscure lung nodules. Scanning at maximal inspiration also provides CT image data suitable for quantitative assessment of emphysema (see COPD/Asthma Profile).
To minimize measurement variability, efforts should be made to obtain consistent, reproducible, maximal inspiratory lung volume on all scans. To achieve this, the use of live breathing instructions given at a pace easily tolerated by the patient is strongly recommended. However, depending on local practice preference and expertise, the use of prerecorded breathing instructions may provide acceptable results. Regardless of whether live or recorded breathing instructions are used, compliance with breathing instructions should be monitored by carefully observing the movement of the chest wall and abdomen to insure that the breathing cycle stays in phase with the verbal instructions. The scan should not be initiated until maximal inspiratory volume is reached and all movement has ceased.
To promote patient compliance, performing a practice round of the breathing instructions prior to moving the patient into the scanner also is strongly recommended. This will make the subject familiar with the procedure, make the technologist familiar with the subjects breathing rate, and allow the technologist to address any subject difficulties in following the instructions.
Sample breathing instructions:
Take in a deep breath (watch anterior chest rise)
Breathe all the way out (watch anterior chest fall)
Now take a deep breath in..inin..in all the way as far as you can
When chest and abdomen stop rising, say Now hold your breath.
Initiate the scan when the chest and abdomen stop moving, allowing for the moment it takes for the diaphragm to relax after the glottis is closed.
When scan is completed, say You can breathe normally
Breath holding during CT scanning greatly reduces motion artifacts, and is essential for quantitative assessment of lung nodules. The inspiratory volume achieved at the time of breath holding influences quantitative lung nodule volume measurements, such that incomplete lung expansion can artificially increase the measured nodule volume (refs). Maximizing inspiratory volume also serves to separate structures, making nodules more conspicuous, and minimizes atelectasis in the dependent portions of the lungs which can impair the detection and assessment of lung nodules. Furthermore, scanning at full inspiration provides CT image data suitable for quantitative assessment of emphysema (see COPD/Asthma Profile). Therefore, scans should be performed at full inspiration.
To minimize measurement variability, efforts should be made to maximize the consistency of inspiratory lung volume. Devices that trigger the CT scan at a preset inspiratory level have been developed, but are not currently recommended due to the additional time, expense, and technologist training that would be required, uncertainty regarding subject acceptance and compliance, and lack of availability across scanner vendors and models. A system that automatically monitors and coordinates automated breathing instructions with subject breathing movements to trigger scanning at maximal breath-hold volume, and is uniform among scanner models, is a conceivable solution but does not currently exist. At this time, nonautomated methods requiring technologist involvement are needed.
Therefore, adherence to the use of specific breathing instructions designed to maximize inspiratory lung volume and consistency during scanning is necessary. Subject compliance should be monitored by carefully observing the movement of the chest wall and abdomen, to insure that the breathing cycle stays in phase with the verbal instructions. The scan should not be initiated until full inspiratory volume is reached and all movement has ceased.
Individual verbal coaching and monitoring using live breathing instructions is strongly recommended; the use of automated, pre-recorded breathing instructions is discouraged. To promote patient compliance, performing a practice round of the breathing instructions prior to moving the patient into the scanner also is strongly recommended. This will make the subject familiar with the procedure, make the technologist familiar with the subjects breathing rate, and allow the technologist to address any subject difficulties in following the instructions.
Sample breathing instructions:
Take in a deep breath (watch anterior chest rise)
Breathe all the way out (watch anterior chest fall)
Now take a deep breath in..inin..in all the way
When chest and abdomen stop rising, say Now hold your breath.
Initiate the scan when the chest and abdomen stop moving, allowing for the moment it takes for the diaphragm to relax after the glottis is closed.
When scan is completed, say You can breathe normally
3.1.45.2 Specification
ParameterSpecificationBreath holdThe Technologist shall instruct the subject in proper breath-hold procedures to achieve maximal inspiration. Providing live voice breath-holding instructions is preferred, and coaching with close visual monitoring for compliance with instructions is strongly recommended to achieve maximum lung volume during scanning.
3.2. Image Data Acquisition
3.2.1 Discussion
CT scans for nodule volumetric analysis can be performed on any equipment that complies with the specifications set out in this Profile. However, we strongly encourage performing all CT scans for an individual subject on the same platform (manufacturer, model and version) which we expect willis expected to further reduce variation and is strongly recommended.
Many scan parameters can have direct or indirect effects on identifying, segmenting and measuring nodules. To reduce this potential source of variance, all efforts should be made to have as many of the scan parameters as possible consistent with the baseline.
Consistency with the baseline implies a need for a method to record and communicate the baseline settings and make that information available at the time and place that subsequent scans are performed. Although it is conceivable that the scanner could retrieve prior/baseline images and extract acquisition parameters to encourage consistency, such interoperability mechanisms are not defined or mandated here and cannot be depended on to be present or used. Similarly, managing and forwarding the data files when multiple sites are involved may exceed the practical capabilities of the participating sites. Sites should be prepared to use manual methods instead.
The goal of parameter consistency is to achieve consistent performance. Parameter consistency when using the same scanner makebrand/model generally means using the same values. Parameter consistency when the baseline was acquired on a different makebrand/model may require some interpretation to achieve consistent performance since the same values may produce different behavior on different models. The parameter sets in Appendix D may be helpful in this task.
The approach of the specifications here, and in the reconstruction section, is to focus as much as possible on the characteristics of the resulting dataset, rather than one particular technique for achieving those characteristics. This is intended to allow as much flexibility as possible for product innovation and reasonable adjustments for patient size (such as increasing acquisition mAs and reconstruction DFOV for larger patients), while reaching the performance targets. Again, the technique parameter sets in Appendix D may be helpful for those looking for more guidance.
The purpose of the minimum scan speed requirement is to permit acquisition of an anatomic region in a single breath-hold, thereby preventing respiratory motion artifacts or anatomic gaps between breath-holds. Anatomic coverage should include the entire volume of the lungs, minimizing the volume scanned above and below the lungs to avoid unnecessary radiation exposure.
Use of CT scanners with a minimum of 16 detectors is expected to allow the claims of this profile to be met consistently. The primary consideration leading to this requirement is the desire to scan the entire length of the lungs in a single breath-hold to minimize motion artifacts, at a pitch that provides adequate z-axis resolution. Published investigations have demonstrated accuracy levels of CT nodule volumetry meeting the claims of this Profile using 16-detector scanners with pitch up to X. The limited data available indicate that z-axis resolution may be inadequate for nodule volumetry in some patients using scanners with <16 detectors and pitch high enough to allow the entire lung to be scanned in a single breath hold.
In CT screening, the choice of scan acquisition parameters is strongly influenced by the desire to minimize radiation dose. The radiation dose delivered by volumetric CT scanning is indicated by the volume CT dose index (CTDIvol), and determined by the interaction of multiple parameters, including the tube voltage (kV), tube current (mA), tube rotation speed, pitch, and the image reconstruction method. The CTDIvol should be chosen to provide the lowest radiation dose that maintains acceptable image quality for detecting pulmonary nodules. Typical CT parameter settings used in lung cancer screening trials translate to CTDIvol in the range of __ to __. The use of iterative reconstruction techniques allows CTDIvol to be reduced even further. Settings for kV, mA, rotation time, and pitch may be varied as needed to achieve the desired CTDIvol.
Pitch is chosen so as to allow completion of the scan in a single breath hold with adequate spatial resolution along the subject z-axis. It is recommended that pitch does not exceed 2.0 for CT acquisitions obtained with a single x-ray tube, or the equivalent for acquisitions with dual-source technology.
Automatic exposure control aims to achieve consistent noise levels throughout the lungs by varying the tube current during scan acquisition. Use of automatic exposure control is expected to have little effect on Profile claims and is considered optional, though as with other acquisition parameters its use should be consistent with baseline. This scanner feature may be a useful tool for reducing unnecessary radiation exposure in certain patients, but it also can increase radiation exposure depending on the target noise level, patient size and anatomy, and the method employed by the vendor. These factors should be kept in mind when deciding whether to use automatic exposure control in an individual patient.
For subjects needing two or more breath-holds to fully cover an anatomic region, different nodules may be acquired on different breath-holds. It is still necessary that each nodule be fully included in images acquired within a single breath-hold to avoid discontinuities or gaps that would affect the measurement.
Scan Plane (transaxial is preferred) may differ between subjects due to the need to position for physical deformities or external hardware. For an individual subject, a consistent scan plane will reduce unnecessary differences in the appearance of the nodule.
Total Collimation Width (defined as the total nominal beam width, NxT, for example 64x1.25mm) is often not directly visible in the scanner interface. Manufacturer reference materials typically explain how to determine this for a particular scanner make, model and operating mode. Wider collimation widths can increase coverage and shorten acquisition, but can introduce cone beam artifacts which may degrade image quality. Imaging protocols will seek to strike a balance to preserve image quality while providing sufficient coverage to keep acquisition times short.
Nominal Tomographic Section Thickness (T), the term preferred by the International Electrotechnical Commission (IEC), is sometimes also called the Single Collimation Width. Choices depend on the detector geometry which varies with different scanner models. It The Nominal Tomographic Section Thickness affects the spatial resolution along the subject z-axis and the available options for reconstructed section thickness.
Thinner sections with sSmaller voxels are preferable, to reduce partial volume effects and provide higher accuracy due to higher spatial resolution. The resolution/voxel size that reaches the analysis software is affected by both acquisition parameters and reconstruction parameters (discussed below).
X-ray CT uses ionizing radiation. Exposure to ionizing radiation from CT can pose risks; however, as the radiation dose is reduced, image quality can be degraded. The imaging in the NLST involved the exclusive used of LDCT for nodule involved the use of imaging techniquesuse of lowered radiation doses that required on average 1.5 mSvi per low dose imagescreening examination. and this is all of the radiation required on average for any LDCT screening. It is expected that health care professionals will balance the need for good image quality with the risks of radiation exposure on a case-by-case basis. It is not within the scope of this document to describe how these trade-offs should be resolved.
3.2.2 Specification
The Acquisition Device shall be capable of performing scans with all the parameters set as described in the following table. The Technologist shall set up the scan to achieve the requirements in the following table.
ParameterSpecificationDICOM TagScan Duration for ThoraxScan duration should be less than 10 seconds. The necessary table speed will depend on the detector configuration, patient size, and pitch requirements for the scanner model (see Pitch).Single breath hold Table Speed
(0018,9309)Anatomic CoverageThe entirety of the lungs shall be included from the apices through the bases.
Apex to through bBase of lungsAnatomic Region Sequence
(0008,2218)Scan Plane (Image Orientation)Number of detectorsTransaxial (Is this needed?)16 or greaterGantry/Detector Tilt (0018,1120)Total Collimation WidthGreater than or equal to 16mm.Total Collimation Width
(0018,9307)CTDIvolIEC PitchAs close to 1.0 as is achievable by the scanner. Not higher than 1.2 or lower than 0.95 (needs verification)No greater than 2.0 for single source scanners, or the equivalent for dual source scanners..Spiral Pitch Factor
(0018,9311)Tube Potential (kVp)120 kVp for cross-sectional imaging. For scout views, reduce below 120 to the greatest extent possible while maintaining adequate image quality to recognize relevant anatomic landmarks. 120 kVp or lessAdjust to achieve appropriate CTDIvol.
For scout view, use lowest needed to view anatomic landmarks.KVP
(0018,0060)mAsAdjust to achieve appropriate CTDIvol.
For scout view, use lowest needed to view anatomic landmarks.For cross-sectinal imaging: No more than 40, ideally 20 or less using iterative reconstruction. (Guideline to increase >40 for marked obesity? Using noise index <40 HU SD?)
For scout view, reduce to the greatest extent possible while maintaining adequate image quality to recognize relevant anatomic landmarks.Automatic exposure controlOptionalRotation timeMay vary as needed to a c h i e v e o t h e r s e t t i n g s . G e n e r a l l y d"0 . 5 s e c . A u t o m a t i c e x p o s u r e c o n t r o l N e e d s f u l l d i s c u s s i o n R o t a t i o n t i m e N o m i n a l T o m o g r a p h i c S e c t i o n T h i c k n e s s ( T ) A d j u s t t o a c h i e v e r e c o n s t r u c t e d s l i c e t h i c k n e s s d"1 . 2 5 m m L e s s t h a n o r e q u a l t o 1 . 2 5 m m . S i n g l e C o l l imation Width
(0018,9306)Acquisition Field of View (FOV)Consistent with baseline. Guidelines for baseline?Image HeaderThe Acquisition Device shall record actual Field of View, Scan Duration, Scan Plane, Total Collimation Width, Single Collimation Width, Scan Pitch, Tube Potential, Tube Current, Rotation Time, Exposure and Slice Width in the DICOM image header.
So we say nothing here? JLM3.3. Image Data Reconstruction
3.3.1 Discussion
Image reconstruction is modeled as a separate Activity in the QIBA Profile. Although it is closely related to image acquisition, and is usually performed on the Acquisition Device, reconstruction may be performed, or re-performed, separate from the acquisition. Many reconstruction parameters will be influenced or constrained by related acquisition parameters. This specification is the result of discussions to allow a degree of separation in their consideration without suggesting they are totally independent.
Many reconstruction parameters can have direct or indirect effects on identifying, segmenting, and measuring nodules. To reduce this potential source of variance, all efforts should be made to have as many of the parameters as possible consistent with the baseline.
Consistency with the baseline implies a need for a method to record and communicate the baseline settings and make that information available at the time and place that subsequent reconstructions are performed. Although it is conceivable that the scanner could retrieve prior/baseline images and extract reconstruction parameters to encourage consistency, such interoperability mechanisms are not defined or mandated here and cannot be depended on to be present or used. Similarly, managing and forwarding the data files when multiple sites are involved may exceed the practical capabilities of the participating sites. Sites should be prepared to use manual methods instead.
Spatial Resolution quantifies the ability to resolve spatial details. Lower spatial resolution can make it difficult to accurately determine the borders of nodules, and as a consequence, decreases the precision of volume measurements. Increased spatial resolution typically comes with an increase in noise which may degrade segmentation and quantification of nodules. Therefore, the choice of factors that affect spatial resolution typically represent a balance between the need to accurately represent fine spatial details of objects (such as the boundaries of nodules) and the noise within the image. Maximum spatial resolution is mostly determined by the scanner geometry (which is not usually under user control) and the reconstruction kernel (over which the user has some choice). Resolution is stated in terms of the number of line-pairs per cm that can be resolved in a scan of resolution phantom (such as the synthetic model provided by the American College of Radiology and other professional organizations). If a followup scan has a significantly different resolution than the baseline, it is likely that the exposure characteristics will change which can affect repeatability. The impact of partial volume effects can also change, so reasonable consistency of resolution within a given patient is desirable.
Noise Metrics quantify the magnitude of the random variation in reconstructed CT numbers. Increased levels of noise can make it difficult to identify the boundary of nodules by humans and automated algorithms.
Some properties of the noise can be characterized by the standard deviation of reconstructed CT numbers over a uniform region in phantom. Voxel Noise (pixel standard deviation in a region of interest) can be reduced by reconstructing images with greater thickness for a given mAs. A constant value for the noise metric might be achieved by increasing mAs for thinner reconstructed images and reducing mAs for thicker reconstructed images. The use of a standard deviation metric has limitations since it can vary with different reconstruction kernels, which will also impact the spatial resolution. While the Noise-Power Spectrum would be a more comprehensive metric, it is not practical to calculate (and interpret) at this time. Therefore, the standard deviation metric is the preferred measure for Voxel Noise. It is not expected that the Voxel Noise be measured for each subject scan, but rather the Acquisition Device and Reconstruction Software be qualified for the expected acquisition and reconstruction parameters.
Reconstruction Field of View interacts with image matrix size (512x512 for most reconstruction algorithms) to determine the affects reconstructed pixel size because the fixed image matrix size of most reconstruction algorithms is 512x512. Pixel size directly affects voxel size in the x-y plane. Smaller voxels are preferable to reduce partial volume effects that can blur the edges of nodules and reduce measurement accuracy and precision. Pixel size in each dimension is not the same as spatial resolution in each dimension. The spatial resolution of the reconstructed image depends on a number of additional factors including the section thickness and reconstruction kernel. Targeted reconstructions with a small field of view minimize partial volume effects, but have limited effect on the accuracy of nodule volumetry compared to a standard field of view that encompasses all of the lungs. A reconstructed field of view set to the widest diameter of the lungs, and consistent with baseline, is sufficient to meet the claims of this Profile. If it is necessary to expand the field of view to encompass more anatomy, the resulting larger pixels may be insufficient to achieve the claim. A targeted reconstruction with a smaller field of view may be necessary, but a reconstruction with that field of view would need to be performed for every time point. Pixel Size directly affects voxel size along the subject x-axis and y-axis. Smaller voxels are preferable to reduce partial volume effects and provide higher measurement precision. Pixel size in each dimension is not the same as spatial resolution in each dimension. The spatial resolution of the reconstructed image depends on a number of additional factors including a strong dependence on the reconstruction kernel.
The reconstructed slice thickness should be small relative to the size of the smallest nodules detected and followed by CT screening, to minimize partial volume averaging. A thickness of 1.25 mm or less is required to meet the Profile claims.
Reconstruction Interval (a.k.a. Slice spacing) that results in discontiguous data is unacceptable as it may truncate the spatial extent of the nodule, degrade the identification of nodule boundaries, confound the precision of measurement for total nodule volumes, etc. Images should be reconstructed either contiguously or in an overlapping Decisions about overlap manner (i.e. having with an interval that is less than the nominal reconstructed slice thickness). need to consider the technical requirements of the clinical trial, including effects on measurement, throughput, image analysis time, and storage requirements.Either method will be consistent with the Profile claims, though overlap on the order of 33-50% may provide better accuracy and precision compared to contiguous slice reconstruction.
Reconstructing datasets with overlap will increase the number of images and may slow down throughput, increase reading time, and increase storage requirements. For multi-detector row CT (MDCT) scanners, creating overlapping image data sets , but has NO effect on radiation exposure; this is true because multiple reconstructions having different kernel, slice thickness and intervals can be reconstructed from the same acquisition (raw projection data) and therefore no additional radiation exposure is needed. A reconstruction Interval that results in gaps between slices is unacceptable as it may truncate the spatial extent of the nodule, degrade the identification of nodule boundaries, confound the precision of measurement for total nodule volumes, etc.
The Reconstruction Algorithm Type most commonly used for CT has been filtered back projection, which meets the claims of this Profile. More recently introduced methods of iterative reconstruction can provide reduced image noise and/or radiation exposure. Studies to date have indicated that iterative methods are at least comparable to filtered back projection for CT volumetry, and are also acceptable.
Slice thickness is nominal since the thickness is not technically the same at the middle and at the edges.
Reconstruction Kernels Characteristics influence the texture and the appearance of nodules in the reconstructed images, which may influence measurementsincluding the sharpness of the nodule edges. In general, aA softer, smoother kernel can reduces noise at the expense of spatial resolution, while a sharper, higher-frequency . An enhancing kernel can improves resolutionving power at the expense of increased noise. Kernel types may interact differently with different software segmentation algorithms. The claims of this Profile are most applicable to reconstruction kernels in the medium-smooth to medium-sharp range of those available on clinical scanners. With increasing kernel smoothness overestimation of nodule volume becomes a potential concern, while with increasing kernel sharpness image noise and segmentation errors become potential concerns. Use of a reconstruction kernel consistent with baseline therefore is particularly important for relying on the Profile claimsThe characteristics of different tissues (e.g. lung) may call for the use of different kernels, and implementers are encouraged to use kernels suitable for the anatomic region and tissue imaged. The use of multiple kernels in a single study is not prohibited by the specification below, but any given nodule must be measured on images reconstructed using consistent kernels at each time point.
The use of iterative reconstruction also may influence the texture and the appearance of nodules in the reconstructed images, which may influence measurements. Therefore the effects of iterative reconstruction on quantitative accuracy and reproducibility are not fully understood as of this writing of this Profile version so it is not currently allowed within the Profile Claim. (Disagree, I encourage the use of IR or even model based even though we dont know much about its effect. We do know it reduces noise and I believe that is sufficient. The noise reduction will ultimately be driving the screening process so that dose can be lowered.)
The stability of HU between time points and its effect on volume measurements is not fully understood as of the writing of this version of the Profile.
3.3.2 Specification
The Reconstruction Software shall be capable of producing images that meet the following specifications. The Technologist shall set up or configure the reconstruction to achieve the requirements in the following table.
ParameterSpecificationIn-plane Spatial ResolutionGreater than or equal to 6 lp/cm and consistent with baseline (i.e. within 1 lp/cm).
Should we merely specify a maximal FOV instead of results from calibration device
Should we delete this parameter since it is dictated by the Recon FOV?Voxel NoiseStandard deviation of < 18HU measured near the center of a 20cm water phantom. This sounds very low. For screening, would consider up to 100HU Needs full discussionReconstructionField of ViewSet to the widest diameter of the lungs.Slice Reconstructed Slice ThicknessLess than or equal to 1.25 mm and consistent with baseline (i.e. within 0.5mm). Reconstruction IntervalLess than or equal to slice thickness and consistent with baseline.Reconstruction OverlapGreater than or equal to 0 (i.e. no gap, and may have some overlap) and consistent with baseline.Reconstruction Algorithm TypeFiltered bBack-pProjection or iterative methods. (would recommend IR or even model based), Reconstruction Kernel CharacteristicsConsistent with baseline (i.e. the same kernel if available, otherwise the kernel most closely matching the kernel response of the baseline). Recommend a non enhancing, medium smooth to medium sharp kernel., ie: Standard or B30Image HeaderThe Reconstruction Software shall record actual Spatial Resolution, Noise, Pixel Spacing, Reconstruction Interval, Reconstruction Overlap, Reconstruction Kernel Characteristics, as well as the model-specific Reconstruction Software parameters utilized to achieve compliance with these metrics in the image header.3.4. Image Analysis
3.4.1 Discussion
This Profile characterizes each designated nodule by itsDetermining the volume change relative to prior image sets.of a nodule is typically done by defining the nodule boundary (referred to as segmentation), computing the volume of the segmented nodule, and calculating the difference from baseline. The specifications also apply to determination of the baseline volume of a nodule, which may guide management in some screening protocols.
This is typically done by determining the boundary of the nodule (referred to as segmentation), computing the volume of the segmented nodule and calculating the difference of the nodule volume in the current scan and in the baseline scan.
Volume Calculation values from a segmentation may or may not correspond to the total of all the segmented voxels. The algorithm may consider partial volumes, do surface smoothing, nodule or organ modeling, or interpolation of user sculpting of the volume. The algorithm may also pre-process the images prior to segmentation.
Segmentation typically is performed may be performed automatically by a software algorithm, manually by a human observer, or semi-automatically by an automated algorithm after with human guidance/intervention, for example to identify the user identifies a starting seed point, stroke, or region, or to edit boundaries. Segmentation boundaries should be closely inspected in three dimensions by the user for concordance with the visually-assessed nodule margins. For these tasks, the display window and level should be fixed and kept the same for all time points being compared. Use of standard window and level settings, or recording of settings used, will facilitate consistency over multiple time points.
Nodules for which the segmentation tracks the margins most accurately, without manual editing, will most closely meet the claims of this Profile. If the segmentation does not track the nodule margins accurately, manual editing or segmentation boundaries may be performed but should be kept to a minimum. If in the radiologists opinion the segmentation is unacceptable, quantitative volumetry should not be used and volume change should be assessed using standard clinical methods.
In measuring nodules, the variance would be more favorable in quantitating an isolated lesion.
If a human observer participates in the segmentation, either by determining while looking at the images the proper settings for an automated process, or by manually editing boundaries, the settings for conversion of density into display levels (window and level) should either be fixed during the segmentation process or documented so that observers can apply consistent display settings at future scans (or a different observer for the same scan, if multiple readers will read each scan, as for a clinical trial).
Nodule location and margin characteristics impact segmentation quality, which is more reliable for nodules that are isolated and well-separated from adjacent structures, and that have smooth borders. Consequently, variance in nodule measurement is more favorable for isolated nodules compared to nodules abutting pulmonary vessels or parietal pleura, and also for smooth compared to spiculated or irregularly shaped nodules.
Nodule Volume Change Variability, which is the focus of the Profile Claim, is a key performance parameter for this biomarker. The 30% target is a conservative threshold of measurement variation (the 30% change in the claim is the outside of 95% confidence interval of 15% of measurement variability when sample size is 40 or more). Based on a survey of clinical studies (See Appendix B.2) the 30% target is considered to be reasonable and achievable. In Table B.1, the range between the minimum and maximum values in the 95% CI of the measurement difference column is mostly within +/- 15%. Considering a large study from Wang et al using 2239 patients ADDIN EN.CITE ADDIN EN.CITE.DATA [ HYPERLINK \l "_ENREF_15" \o "Wang, 2008 #7906" 15], the 95% confidence interval ranged [-13.4%, 14.5%].
Methods that calculate volume changes directly without calculating volumes at individual time points are acceptable so long as the results are compliant with the specifications set out by this Profile.
When deriving the nodule volume difference between two time points, tThe Image Analysis Tooluser should be prepared to process both directly compare the current data and previous image data at the same time and support matching up the appearanceto reduce interobserver and intraobserver variation. Storing segmentations and measurement results for review at a later date is certainly a useful practice as it can save time and cost. However, segmentation results at both time points should be inspected visually in three dimensions to make sure that they are of sufficient and each nodule incomparable accuracy both data sets in order to derive volume change valuesmeet the claims of the Profile. If a previous segmentation is unavailable for viewing, or the previous segmentation is not of comparable accuracy to the current segmentation, segmentation at the comparison time point should be repeated. Although it is conceivable that they could be processed separately and the results of prior processing could be imported and a method of automated tagging and matching of the nodules could be implemented, such interoperability mechanisms are not defined or mandated here and cannot be depended on to be present or used.
Storing segmentations and measurement results that can be loaded by an Image Analysis Tool analyzing data collected at a later date is certainly a useful practice as it can save time and cost. For this to happen reliably, the stored format must be compatible and the data must be stored and conveyed. Although DICOM Segmentation objects are appropriate to store nodule segmentations, and DICOM SR objects are appropriate to store measurement results. These , thesestandards are recommended but it is recognized that these tools are not yet widely enough deployed to make support for them mandatory in this Profile. Similarly, conveying the segmentations and measurements from baseline (and other time points prior to the current exam) is not done consistently enough to mandate that it happen and to require their import into the Image Analysis Tool. Managing and forwarding the data files may exceed the practical capabilities of the participating sites.
Image analysis can be performed on any equipment that complies with the specifications set out in this Profile. However, we strongly encourage performing all analysis for an individual subject on the same platform (manufacturer, model and version) which we expect will further reduce variation.
Medical Devices such as the Image Analysis Tool are typically made up of multiple components (the hardware, the operating system, the application software, and various function libraries within those). Changes in any of the components can affect the behavior of the device. In this specification, the device version should reflect the total set of components and any changes to components should result in a change in the recorded device version. This device version may thus be different than the product release version that appears in vendor documentation. Is it true that the computer hardware and operating system affect the results of the application software or can this paragraph be deleted?
Image analysis should be performed using software programs that have received appropriate scientific validation. Because different programs use different segmentation algorithms that result in different volumetric measurements even for ideal nodules, and different versions of the same program or its components may change its performance, a nodule being evaluated for change should be analyzed at both time points on the same platform (manufacturer, model and version).
These Image Analysis specifications are intended to apply to a For analysis methods that involve an operator (e.g. to draw or edit boundaries, set seed points or adjust parameters), the operator is effectively a component of the system, with an impact on the reproducibility of the measurements, and it is important to record the operators identify as well. Fully automated analysis software removes that source of variation; although even then, since a human is generally responsible for the final results, they retain the power to approve or reject measurements so their identity should be recorded.
The Nodule Volume Change performance specification below includes the operator performance and is intended to be evaluated based on a typical operator user (i.e. without extraordinary training or ability). This should be kept in mind by vendors measuring the performance of their tools and sites validating the performance of their installation. Although the performance of some methods may depend on the judgment and skill of the operatoruser, it is beyond this Profile to specify the qualifications or experience of the operator.
Determination of which nodules should be measured is out of scope for this Profile. Such determination may be specified within a protocol or specified by formal response criteria standards, or may be determined by clinical requirements. Nodules to be measured may be designated by the clinicianoncologist or clinical investigator, by a radiologist at a clinical site, by a reader at a central reading facility, or they may be designated automatically by a software analysis tool.
3.4.2 Specification
ParameterSpecificationImage Analysis PlatformThe Image Analysis Platform shall have appropriate scientific validation.
The same Image Analysis Platform (manufacturer, model, version) shall be used for measurements at all time points.Image Display SettingsImage display setting (window and level) during the segmentation initiation and review process shall be the same at all time points.Nodule AnalysisThe user shall visually inspect segmentation results in three dimensions and exclude nodules with inadequate segmentations or nodules with noncomparable segmentations at both time points from quantitative volumetric assessment.RecordingThe Image Analysis Tool shall record the percentage volume change relative to baseline for each nodule, the device version and the actual model-specific Analysis Software set-up and configuration parameters utilized.
The Image Analysis Tool shall be capable of recording the nodule segmentation as a DICOM Segmentation.
The Image Analysis Tool shall record the identity of each individual making and/or approving a nodule measurement using the software.
4. Compliance
To comply with this Profile, participating staff and equipment (Actors) shall support each of the activities assigned to them in Table 1.
For each activity, the compliance requirements (sometimes referred to as the shall language) for each Actor are documented in Section 3.
Although most of the requirements described in Section 3 are feature-oriented and compliance can be assessed by direct observation, some of the requirements are performance-oriented. The following sub-sections elaborate on the meaning of performance-oriented requirements and how they are intended to be correctly assessed.
Formal claims of compliance by the organization responsible for an Actor shall be in the form of a published QIBA Conformance Statement. Vendors publishing a QIBA Conformance Statement shall provide a set of Model-specific Parameters (as shown in Appendix D) describing how their product was configured to achieve compliance. Vendors shall also provide access or describe the characteristics of the test set used for compliance testing.
4.1. Performance Assessment: Nodule Volume Change Variability
Note: The procedure in this section is currently only a proposal.
A more detailed procedure and pointers to valid test datasets will be provided in the future.
Until then, there is no approved way to claim conformance to this performance requirement.
Nodule Volume Change Variability performance can be assessed with the following procedure:
Obtain a designated test image set (see 4.1.1).
Determine the volume change for designated nodules (see 4.1.2).
Calculate descriptive statistics (see 4.1.3).
Compare against the Nodule Volume Change Variability performance level specified in 3.4.2.
This procedure can be used by a vendor or an imaging site to evaluate the performance of an Image Analysis Tool (in automatic mode, or with a typical operator), or the combined performance of an Image Analysis Tool together with a particular Radiologist to determine if they are in compliance with the Nodule Volume Change Variability performance requirement in Section 3.4.2.
4.1.1 Test Image Set
Discussion:
We have many test image cases where the true change is known to be 0% (Coffee break).
We have many test image cases where the true change is unknown (although change is clearly present).
Are we missing data to show both sensitivity and specificity?
What exactly is our goal with this performance assessment?
Consider a multi- step assessment?
1) Assess (change?) sensitivity (in terms of inherent measurement variation) using No change data
2) Assess (volume?) bias using data with a known volume (phantom?)
3) Assess change performance against consensus values (rather than measured/known truth?)
Nodule segmentation performance can be affected by the accuracy or variations in the seed point or axis provided. Consider preparing the test set with test inputs (either with a click here dot on the image, or some method for feeding coordinates to the application).
Ideally we want fully realistic images (not phantom) but with known truth for nodule volume change. Would it be possible to digitally insert nodules into real acquired human images?
What is the best way to go about assembling and hosting these datasets? Such a public dataset is not currently known to exist.
4.1.2 Determine Volume Change
Determine the measured proportional percentage volume change for each designated nodule in each image multiple times by multiple readers.
Discussion:
Should the (minimum) number of readers and the (minimum) number of repeats for each reader (for each nodule?) be prescribed in the procedure?
Will those numbers be different for fully automated measurements (which are presumably more consistent among repeats on the same data but are generally cheap to run more repeats.)?
Consider whether the procedure should allow a small number of segmentation or volume change results to be set aside prior to calculation of the descriptive statistics to avoid a couple unusual cases from distorting the summary statistics. Such failures could still be reported individually in the results.
Would such blow ups be easily distinguished by the algorithm or operator? Dan Barboriak has done work on related issues.
4.1.3 Calculate Descriptive Statistics
Calculate descriptive statistics that represent the joint-distribution of true proportional percentage volume change and measured proportional percentage volume change.
Discussion:
The performance score statistics should not be a simple total of all the nodule change vales, but rather we should quote performance on individual nodules over a specified number of repeats for a specified number of nodules.
Given the volume measure at Time1 and Time2, consider both the variance and the correlation between the two measurements (i.e. the variance of the individual measurements and also
(sigma of the delta)**2 = 2 (1-rho) sigma**2
It is expected that correlation across visits will be dominated by using a different device?
Consider calculating and expressing in terms of the confidence that a change of size X is really more than Y. ie. in the P(A|B)>C can we fix or vectorize any of the three variables? Note that the target zones for change confidence might be different for clinical trials vs patient management. Does this point us toward two claims? Or maybe a claim in the form of a vector of values or a curve?
Alternatively, consider (as suggested by TSB in comment #164) evaluating performance relative to a specified (e.g. expert consensus derived) truth value.
Keep in mind that we need to maintain consistency between our claim and our performance measures (e.g. focus on repeatability vs. accuracy).
It is important to characterize individual volume measurement performance since that value is an input to a variety of models (and would be useful for patient enrichment in trials). So, for example:
For each nodule(t)
Average the (r) measurements of t
Enumerate the number of measurements N(t) that are within 30% of the average
N=Sum N(t)
If N >= 95% of t*r then the 95% confidence performance specification has been met.
It might be useful to explore the Visual Analog Scale (VAS Score) as a categorization tool for the target nodules and set different variance or performance targets for each category, or consider weighting the errors based on the VAS Score.
4.2. Performance Assessment: Image Acquisition Site
Note: The procedure in this section is currently only a proposal.
A more detailed procedure and pointers to valid test datasets will be provided in the future.
Until then, there is no approved way to claim conformance to this performance requirement.
Site performance can be assessed with the following procedure:
Validate image acquisition (see 4.2.1).
Generate a test image set (see 4.2.2).
Assess Nodule Volume Change Variability (see 4.1.2, 4.1.3 above).
Compare against the Nodule Volume Change Variability performance level specified in 3.4.2.
This procedure can be used by an imaging site to evaluate the performance of each of the Actors and Activities in use. In principle, the final result represents an assessment of the combined performance of all the Actors and Activities at the site.
The procedure presumes that the Actors being used by the site are capable of meeting the requirements described in Section 3 of this document; however it is not a pre-requisite that those Actors have published QIBA Conformance Statements (although that would be both useful and encouraging).
Discussion:
Duke is working on a platform that includes a phantom and an analysis tool that may inform the future contents of this section.
Sites that carry out this procedure should really record the parameters they used and document them in something similar to a Conformance Statement. This would be a useful QA record and could be submitted to clinical trials looking for QIBA compliant test sites.
Are there other criteria that should be worked into this procedure?
Typically clinical sites are selected due to their competence in oncology and access to a sufficiently large patient population under consideration. For imaging it is important to consider the availability of:
- appropriate imaging equipment and quality control processes,
- appropriate injector equipment and contrast media,
- experienced CT Technologists for the imaging procedure, and
- processes that assure imaging Profile compliant image generation at the correct point in time.
A clinical trial might specify A calibration and QA program shall be designed consistent with the goals of the clinical trial. This program shall include (a) elements to verify that sites are performing correctly, and (b) elements to verify that sites CT scanner(s) is (are) performing within specified calibration values. These may involve additional phantom testing that address issues relating to both radiation dose and image quality (which may include issues relating to water calibration, uniformity, noise, spatial resolution -in the axial plane-, reconstructed slice thickness z-axis resolution, contrast scale, CT number calibration and others). This phantom testing may be done in additional to the QA program defined by the device manufacturer as it evaluates performance that is specific to the goals of the clinical trial.
4.2.1 Acquisition Validation
Review patient handling procedures for compliance with Section 3.1
Establish acquisition protocols and reconstruction settings on the Acquisition Device compliant with Section 3.2 and Section 3.3. If a QIBA Conformance Statement is available from the Acquisition Device vendor, it may provide parameters useful for this step.
Acquire images of a 20cm water phantom, reconstruct and confirm performance requirements in Section 3.3.2 are met.
Discussion:
UCLA may have more detailed and more complete procedures to recommend for this section.
4.2.2 Test Image Set
Locally acquire a test image set using the protocols established and tested in Section 4.2.1.
The test image set should conform to the characteristics described in Section 4.1.1.
Discussion:
It is highly likely that due to practical constraints the test image set prepared at an individual site would be much less comprehensive than the test image sets prepared by QIBA. Further consideration of what a more limited but still useful test image set would look like.
References
ADDIN EN.REFLIST 1. Moertel, C.G. and J.A. Hanley, The effect of measuring error on the results of therapeutic trials in advanced cancer. Cancer, 1976. 38(1): p. 388-94.
2. Lavin, P.T. and G. Flowerdew, Studies in variation associated with the measurement of solid tumors. Cancer, 1980. 46(5): p. 1286-90.
3. Eisenhauer, E.A., et al., New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer, 2009. 45(2): p. 228-47.
4. Zhao, B., et al., Lung cancer: computerized quantification of tumor response--initial results. Radiology, 2006. 241(3): p. 892-8.
5. Zhao, B., et al., A pilot study of volume measurement as a method of tumor response evaluation to aid biomarker development. Clin Cancer Res, 2010. 16(18): p. 4647-53.
6. Schwartz, L.H., et al., Volumetric 3D CT analysis - an early predictor of response to therapy. J Clin Oncol, 2007. 25(18s): p. abstr 4576.
7. Altorki, N., et al., Phase II proof-of-concept study of pazopanib monotherapy in treatment-naive patients with stage I/II resectable non-small-cell lung cancer. J Clin Oncol, 2010. 28(19): p. 3131-7.
8. Gietema, H.A., et al., Pulmonary nodules: Interscan variability of semiautomated volume measurements with multisection CT-- influence of inspiration level, nodule size, and segmentation performance. Radiology, 2007. 245(3): p. 888-94.
9. Zhao, B., L.H. Schwartz, and S.M. Larson, Imaging surrogates of tumor response to therapy: anatomic and functional biomarkers. J Nucl Med, 2009. 50(2): p. 239-49.
10. Wormanns, D., et al., Volumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibility. Eur Radiol, 2004. 14(1): p. 86-92.
11. Boll, D.T., et al., Volumetric assessment of pulmonary nodules with ECG-gated MDCT. AJR Am J Roentgenol, 2004. 183(5): p. 1217-23.
12. Hein, P.A., et al., Linear and volume measurements of pulmonary nodules at different CT dose levels - intrascan and interscan analysis. Rofo, 2009. 181(1): p. 24-31.
13. Meyer, C.R., et al., Evaluation of lung MDCT nodule annotation across radiologists and methods. Acad Radiol, 2006. 13(10): p. 1254-65.
14. Marten, K., et al., Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria. Eur Radiol, 2006. 16(4): p. 781-90.
15. Wang, Y., et al., Effect of nodule characteristics on variability of semiautomated volume measurements in pulmonary nodules detected in a lung cancer screening program. Radiology, 2008. 248(2): p. 625-31.
16. Revel, M.P., et al., Pulmonary nodules: preliminary experience with three-dimensional evaluation. Radiology, 2004. 231(2): p. 459-66.
17. Sohns, C., et al., Measurement of response of pulmonal tumors in 64-slice MDCT. Acta Radiol, 2010. 51(5): p. 512-21.
18. Werner-Wasik, M., et al., Assessment of lung cancer response after nonoperative therapy: tumor diameter, bidimensional product, and volume. A serial CT scan-based study. Int J Radiat Oncol Biol Phys, 2001. 51(1): p. 56-61.
19. Tran, L.N., et al., Comparison of treatment response classifications between unidimensional, bidimensional, and volumetric measurements of metastatic lung nodules on chest computed tomography. Acad Radiol, 2004. 11(12): p. 1355-60.
20. Jennings, P., et al., Inter-laboratory comparison of human renal proximal tubule (HK-2) transcriptome alterations due to Cyclosporine A exposure and medium exhaustion. Toxicol In Vitro, 2009. 23(3): p. 486-99.
21. Prasad, S.R., et al., CT tumor measurement for therapeutic response assessment: comparison of unidimensional, bidimensional, and volumetric techniques initial observations. Radiology, 2002. 225(2): p. 416-9.
22. Rohde, S., et al., Reliability of CT-based tumor volumetry after intraarterial chemotherapy in patients with small carcinoma of the oral cavity and the oropharynx. Neuroradiology, 2006. 48(6): p. 415-21.
23. Lee, S.M., et al., Usefulness of CT volumetry for primary gastric lesions in predicting pathologic response to neoadjuvant chemotherapy in advanced gastric cancer. Abdom Imaging, 2009. 34(4): p. 430-40.
24. Beer, A.J., et al., Adenocarcinomas of esophagogastric junction: multi-detector row CT to evaluate early response to neoadjuvant chemotherapy. Radiology, 2006. 239(2): p. 472-80.
25. Griffith, J.F., et al., Assessing chemotherapy response of squamous cell oesophageal carcinoma with spiral CT. Br J Radiol, 1999. 72(859): p. 678-84.
26. Benz, M.R., et al., Combined assessment of metabolic and volumetric changes for assessment of tumor response in patients with soft-tissue sarcomas. J Nucl Med, 2008. 49(10): p. 1579-84.
27. Willett, C.G., et al., Three-dimensional volumetric assessment of response to treatment: stage I and II diffuse large cell lymphoma of the mediastinum. Radiother Oncol, 1988. 12(3): p. 193-8.
28. Willett, C.G., et al., Stage IA to IIB mediastinal Hodgkin's disease: three-dimensional volumetric assessment of response to treatment. J Clin Oncol, 1988. 6(5): p. 819-24.
29. Wakelee, H.A., et al., Changes in the natural history of nonsmall cell lung cancer (NSCLC)--comparison of outcomes and characteristics in patients with advanced NSCLC entered in Eastern Cooperative Oncology Group trials before and after 1990. Cancer, 2006. 106(10): p. 2208-17.
30. Gandara, D.R., et al., Radiographic imaging of bronchioloalveolar carcinoma: screening, patterns of presentation and response assessment. J Thorac Oncol, 2006. 1(9 Suppl): p. S20-6.
31. Goldstraw, P., et al., The IASLC Lung Cancer Staging Project: proposals for the revision of the TNM stage groupings in the forthcoming (seventh) edition of the TNM Classification of malignant tumours. J Thorac Oncol, 2007. 2(8): p. 706-14.
32. Jemal, A., et al., Cancer statistics, 2008. CA Cancer J Clin, 2008. 58(2): p. 71-96.
33. Jaffe, C.C., Measures of response: RECIST, WHO, and new alternatives. J Clin Oncol, 2006. 24(20): p. 3245-51.
34. Zhao, B., et al., A Pilot Study of Volume Measurement as a Method of Tumor Response Evaluation to Aid Biomarker Development. Clin Cancer Res, 2010. 16: p. 4647-4653.
35. Fraioli, F., et al., Volumetric evaluation of therapy response in patients with lung metastases. Preliminary results with a computer system (CAD) and comparison with unidimensional measurements. Radiol Med, 2006. 111(3): p. 365-75.
36. Parkin, D.M., et al., Global cancer statistics, 2002. CA Cancer J Clin, 2005. 55(2): p. 74-108.
37. Llovet, J.M. and J. Bruix, Systematic review of randomized trials for unresectable hepatocellular carcinoma: Chemoembolization improves survival. Hepatology, 2003. 37(2): p. 429-42.
38. Lopez, P.M., A. Villanueva, and J.M. Llovet, Systematic review: evidence-based management of hepatocellular carcinoma--an updated analysis of randomized controlled trials. Aliment Pharmacol Ther, 2006. 23(11): p. 1535-47.
39. Koshariya, M., et al., An update and our experience with metastatic liver disease. Hepatogastroenterology, 2007. 54(80): p. 2232-9.
40. Keil, S., et al., Semi-automated measurement of hyperdense, hypodense and heterogeneous hepatic metastasis on standard MDCT slices. Comparison of semi-automated and manual measurement of RECIST and WHO criteria. Eur Radiol, 2008. 18(11): p. 2456-65.
41. Therasse, P., et al., New guidelines to evaluate the response to treatment in solid tumors. European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada. J Natl Cancer Inst, 2000. 92(3): p. 205-16.
42. Llovet, J.M., et al., Sorafenib in advanced hepatocellular carcinoma. N Engl J Med, 2008. 359(4): p. 378-90.
43. Cheng, A.L., et al., Efficacy and safety of sorafenib in patients in the Asia-Pacific region with advanced hepatocellular carcinoma: a phase III randomised, double-blind, placebo-controlled trial. Lancet Oncol, 2009. 10(1): p. 25-34.
44. Garant, M., et al., Liver metastasis: comparison of 2 methods for reporting of disease in patients receiving chemotherapy. Can Assoc Radiol J, 1999. 50(1): p. 13-6.
45. Stillwagon, G.B., et al., 194 hepatocellular cancers treated by radiation and chemotherapy combinations: toxicity and response: a Radiation Therapy Oncology Group Study. Int J Radiat Oncol Biol Phys, 1989. 17(6): p. 1223-9.
46. Sohaib, S.A., et al., CT assessment of tumour response to treatment: comparison of linear, cross-sectional and volumetric measures of tumour size. Br J Radiol, 2000. 73(875): p. 1178-84.
47. Luccichenti, G., et al., Radiologic assessment of rectosigmoid cancer before and after neoadjuvant radiation therapy: comparison between quantitation techniques. AJR Am J Roentgenol, 2005. 184(2): p. 526-30.
48. Rohde, S., et al., CT-based evaluation of tumor volume after intra-arterial chemotherapy of locally advanced carcinoma of the oral cavity: comparison with clinical remission rates. Cardiovasc Intervent Radiol, 2007. 30(1): p. 85-91.
49. Therasse, P., Measuring the clinical response. What does it mean? Eur J Cancer, 2002. 38(14): p. 1817-23.
50. McHugh, K. and S. Kao, Response evaluation criteria in solid tumours (RECIST): problems and need for modifications in paediatric oncology? Br J Radiol, 2003. 76(907): p. 433-6.
Appendices
Appendix A: Acknowledgements and Attributions
This document is proffered by the Radiological Society of North America (RSNA) Quantitative Imaging Biomarker Alliance (QIBA) Volumetric Computed Tomography (v-CT) Technical Committee. The v-CT technical committee is composed of scientists representing the imaging device manufacturers, image analysis software developers, image analysis laboratories, biopharmaceutical industry, academia, government research organizations, professional societies, and regulatory agencies, among others. All work is classified as pre-competitive.
A more detailed description of the v-CT group and its work can be found at the following web link: http://qibawiki.rsna.org/index.php?title=Volumetric_CT.
The Volumetric CT Technical Committee (in alphabetical order):
Athelogou, M. Definiens AG
Avila, R. Kitware, Inc.
Beaumont, H. Median Technologies
Borradaile, K. Core Lab Partners
Buckler, A. BBMSC
Clunie, D. Core Lab Partners
Cole, P. Imagepace
Conklin, J. ICON Medical Imaging
Dorfman, GS. Weill Cornell Medical College
Fenimore, C. Nat Inst Standards & Technology
Ford, R. Princeton Radiology Associates.
Garg, K. University of Colorado
Garrett, P. Smith Consulting, LLC
Goldmacher, G. ICON Medical Imaging
Gottlieb, R. University of Arizona
Gustafson, D. Intio, Inc.
Hayes, W. Bristol Myers Squibb
Hillman, B. Metrix, Inc.
Judy, P. Brigham and Womens Hospital
Kim, HJ. University of California Los Angeles
Kohl, G. Siemens AG
Lehner, O. Definiens AG
Lu, J. Nat Inst Standards & Technology
McNitt-Gray, M. University California Los Angeles
Mozley, PD. Merck & Co Inc.
Mulshine, JL. Rush University
Nicholson, D. Definiens AG
O'Donnell, K. Toshiba Medical Research Institute - USA
O'Neal, M. Core Lab Partners
Petrick, N. US Food and Drug Administration
Reeves, A. Cornell University
Richard, S. Duke University
Rong, Y. Perceptive Informatics, Inc.
Schwartz, LH. Columbia University
Saiprasad, G. University of Maryland
Samei, E. Duke University
Siegel, E. University of Maryland
Silver, M. Toshiba Medical Research Institute USA
Steinmetz, N. Translational Sciences Corporation
Sullivan, DC. RSNA Science Advisor and Duke University
Tang, Y. CCS Associates
Thorn, M. Siemens AG
Vining, DJ. MD Anderson Cancer Center
Yankelovitz, D. Mt. Sinai School of Medicine
Yoshida, H. Harvard MGH
Zhao, B. Columbia University
The Volumetric CT Technical Committee is deeply grateful for the support and technical assistance provided by the staff of the Radiological Society of North America.
Appendix B: Background Information Does this belong here?
B.1 QIBA
The Quantitative Imaging Biomarker Alliance (QIBA) is an initiative to promote the use of standards to reduce variability and improve performance of quantitative imaging in medicine. QIBA provides a forum for volunteer committees of care providers, medical physicists, imaging innovators in the device and software industry, pharmaceutical companies, and other stakeholders in several clinical and operational domains to reach consensus on standards-based solutions to critical quantification issues. QIBA publishes the specifications they produce (called QIBA Profiles), first to gather public comment and then for field test by vendors and users.
QIBA envisions providing a process for developers to test their implementations of QIBA Profiles through a compliance mechanism. Purchasers can specify conformance with appropriate QIBA Profiles as a requirement in Requests For Proposals (RFPs). Vendors who have successfully implemented QIBA Profiles in their products can publish QIBA Conformance Statements. The Conformance Statements are accompanied by Model-specific Parameters (as shown in Appendix D) describing how to configure their product for alignment with the Profile.
General information about QIBA, including its governance structure, sponsorship, member organizations and work process, is available at HYPERLINK "http://qibawiki.rsna.org/index.php?title=Main_Page"http://qibawiki.rsna.org/index.php?title=Main_Page.
QIBA has constructed a systematic approach for standardizing and qualifying volumetry as a biomarker of response to treatments for a variety of medical conditions, including cancers in the lung (either primary cancers or cancers that metastasize to the lung [18]).
B.2 CT Volumetry: Overview and Summary
Table B.1 Summarizing the precision/reproducibility of volumetric measurements from clinical studies reported in the literature
ScanReader# of Readers# of Patients# of NodulesNodule Size,
Mean (range)Organ SystemVolumetry,
95% CI of Measurement DifferenceVolumetry, Measurement Difference %1D Measurement, 95% CI of Measurement Difference1D, Mean Measurement Difference %Slice Thickness /Recon Interval, mmAuthor, Yearrepeat scans intra-reader1202189.85 mmlung, mets -21.2 to 23.8% 1.30%1.0/0.7Gietama et al. 2007 ADDIN EN.CITE Gietema200759[8]595917Gietema, H. A.Schaefer-Prokop, C. M.Mali, W. P.Groenewegen, G.Prokop, M.Department of Radiology, University Medical Center, Heidelberglaan 100, 3584 CX Utrecht, the Netherlands. h.gietema@umcutrecht.nlPulmonary nodules: Interscan variability of semiautomated volume measurements with multisection CT-- influence of inspiration level, nodule size, and segmentation performanceRadiologyRadiology888-942453AdultAgedAged, 80 and overAlgorithmsFemaleHumansInhalationMaleMiddle AgedProspective StudiesSolitary Pulmonary Nodule/ pathology/ radiographyTomography, X-Ray Computed/ methods20071527-1315 (Electronic)
0033-8419 (Linking)1792350820072452061054 [pii]
10.1148/radiol.2452061054 [doi]Nlmeng[ HYPERLINK \l "_ENREF_8" \o "Gietema, 2007 #59" 8]repeat scans intra-reader3323238 mm (1193 mm)lung, NSCLC -12 to 13.4%0.70% -7.3% to 6.2%-0.60%1.25/1.25Zhao et al. 2009 ADDIN EN.CITE Zhao200931[9]313117Zhao, B.Schwartz, L. H.Larson, S. M.Department of Medical Physics, Memorial Sloan-Kettering Cancer Center, New York, New York 10021, USA. zhaob@mskcc.orgImaging surrogates of tumor response to therapy: anatomic and functional biomarkersJ Nucl MedJ Nucl Med239-49502AnimalsCarcinoma, Non-Small-Cell Lung/radiography/radionuclide imaging/therapyFemaleFluorine Radioisotopes/diagnostic useFluorodeoxyglucose F18/diagnostic useHumansLung Neoplasms/radiography/radionuclide imaging/therapyMaleNeoplasms/ radiography/ radionuclide imaging/therapyPositron-Emission TomographyRadiopharmaceuticals/diagnostic useTomography, X-Ray ComputedTumor Markers, Biological/metabolism20090161-5505 (Print)
0161-5505 (Linking)191642182009jnumed.108.056655 [pii]
10.2967/jnumed.108.056655 [doi]Nlmeng[ HYPERLINK \l "_ENREF_9" \o "Zhao, 2009 #31" 9]same scanintra-reader110506.9 mm (2.220.5 mm)lung, mets -3.9 to 5.7%0.90%not reportednot reported1.25/0.8Wormanns et al. 2004 ADDIN EN.CITE Wormanns200474[10]747417Wormanns, D.Kohl, G.Klotz, E.Marheine, A.Beyer, F.Heindel, W.Diederich, S.Department of Clinical Radiology, University of Muenster, Albert-Schweitzer-Strasse 33, 48149, Muenster, Germany. dag.wormanns@uni-muenster.deVolumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibilityEur RadiolEur Radiol86-92141AdultAgedFemaleHumansImage Processing, Computer-AssistedLung Neoplasms/ radiography/ secondaryMaleMiddle AgedObserver VariationProbabilityRadiation DosageRadiographic Image Interpretation, Computer-AssistedRisk FactorsSampling StudiesSensitivity and SpecificitySoftwareSolitary Pulmonary Nodule/pathology/ radiographyTomography, X-Ray Computed/ methods20040938-7994 (Print)
0938-7994 (Linking)14615902200410.1007/s00330-003-2132-0 [doi]Nlmeng[ HYPERLINK \l "_ENREF_10" \o "Wormanns, 2004 #74" 10]same scaninter-reader210506.9 mm (2.220.5 mm)lung, mets -5.5 to 6.6%0.50%not reportednot reported1.25/0.8Wormanns et al. 2004 ADDIN EN.CITE Wormanns200474[10]747417Wormanns, D.Kohl, G.Klotz, E.Marheine, A.Beyer, F.Heindel, W.Diederich, S.Department of Clinical Radiology, University of Muenster, Albert-Schweitzer-Strasse 33, 48149, Muenster, Germany. dag.wormanns@uni-muenster.deVolumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibilityEur RadiolEur Radiol86-92141AdultAgedFemaleHumansImage Processing, Computer-AssistedLung Neoplasms/ radiography/ secondaryMaleMiddle AgedObserver VariationProbabilityRadiation DosageRadiographic Image Interpretation, Computer-AssistedRisk FactorsSampling StudiesSensitivity and SpecificitySoftwareSolitary Pulmonary Nodule/pathology/ radiographyTomography, X-Ray Computed/ methods20040938-7994 (Print)
0938-7994 (Linking)14615902200410.1007/s00330-003-2132-0 [doi]Nlmeng[ HYPERLINK \l "_ENREF_10" \o "Wormanns, 2004 #74" 10]repeat scans not specifiednot specified101517.4 (2.220.5 mm)lung, mets -20.4 to 21.9%1.50%not reportednot reported1.25/0.8Wormanns et al. 2004 ADDIN EN.CITE Wormanns200474[10]747417Wormanns, D.Kohl, G.Klotz, E.Marheine, A.Beyer, F.Heindel, W.Diederich, S.Department of Clinical Radiology, University of Muenster, Albert-Schweitzer-Strasse 33, 48149, Muenster, Germany. dag.wormanns@uni-muenster.deVolumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibilityEur RadiolEur Radiol86-92141AdultAgedFemaleHumansImage Processing, Computer-AssistedLung Neoplasms/ radiography/ secondaryMaleMiddle AgedObserver VariationProbabilityRadiation DosageRadiographic Image Interpretation, Computer-AssistedRisk FactorsSampling StudiesSensitivity and SpecificitySoftwareSolitary Pulmonary Nodule/pathology/ radiographyTomography, X-Ray Computed/ methods20040938-7994 (Print)
0938-7994 (Linking)14615902200410.1007/s00330-003-2132-0 [doi]Nlmeng[ HYPERLINK \l "_ENREF_10" \o "Wormanns, 2004 #74" 10]repeat scans not specifiednot specified10105 <10 mmlung, mets -19.3 to 20.4%1.70%not reportednot reported1.25/0.8Wormanns et al. 2004 ADDIN EN.CITE Wormanns200474[10]747417Wormanns, D.Kohl, G.Klotz, E.Marheine, A.Beyer, F.Heindel, W.Diederich, S.Department of Clinical Radiology, University of Muenster, Albert-Schweitzer-Strasse 33, 48149, Muenster, Germany. dag.wormanns@uni-muenster.deVolumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibilityEur RadiolEur Radiol86-92141AdultAgedFemaleHumansImage Processing, Computer-AssistedLung Neoplasms/ radiography/ secondaryMaleMiddle AgedObserver VariationProbabilityRadiation DosageRadiographic Image Interpretation, Computer-AssistedRisk FactorsSampling StudiesSensitivity and SpecificitySoftwareSolitary Pulmonary Nodule/pathology/ radiographyTomography, X-Ray Computed/ methods20040938-7994 (Print)
0938-7994 (Linking)14615902200410.1007/s00330-003-2132-0 [doi]Nlmeng[ HYPERLINK \l "_ENREF_10" \o "Wormanns, 2004 #74" 10]same scan (5 sets, 1 set/phase) intra-reader ? (consensus by 2 readers), 3 x reading23073~19 mm [25.3 (0.2399 mm3)]lung, noncalcified nodulescoefficient of variance as large as 34.5% (95% CI not reported)not reportednot reportednot reported0.75/0.6Boll et al. 2004 ADDIN EN.CITE Boll200453[11]535317Boll, D. T.Gilkeson, R. C.Fleiter, T. R.Blackham, K. A.Duerk, J. L.Lewin, J. S.Department of Radiology, University Hospitals of Cleveland, Case Western Reserve University, 11100 Euclid Avenue, Cleveland, OH 44106-5056, USA. boll@uhrad.comVolumetric assessment of pulmonary nodules with ECG-gated MDCTAJR Am J RoentgenolAJR Am J Roentgenol1217-231835AdultAgedAged, 80 and overElectrocardiographyFemaleHumansLung/radiographyMaleMiddle AgedPhantoms, ImagingSolitary Pulmonary Nodule/ radiographyTomography, X-Ray Computed20040361-803X (Print)
0361-803X (Linking)155052802004183/5/1217 [pii]Nlmeng[ HYPERLINK \l "_ENREF_11" \o "Boll, 2004 #53" 11]same scan inter-reader23322910.8 mm (2.843.6 mm), median 8.2 mmlung, primary or mets -9.4 to 8.0%0.70% -31.0 to 27%-2.00%1.0/0.8Hein et al. 2009 ADDIN EN.CITE ADDIN EN.CITE.DATA [ HYPERLINK \l "_ENREF_12" \o "Hein, 2009 #320" 12]same scaninter-reader, inter-algorithms (6 readers x 3 algorithms)61623not reportedlung, nodules 55% (upper limit)not reportednot reportednot reported1.25/0.625Meyer et al. 2006 ADDIN EN.CITE ADDIN EN.CITE.DATA [ HYPERLINK \l "_ENREF_13" \o "Meyer, 2006 #492" 13]same scanintra-reader2502023.165195 mm3, median 182.22 mm3lung, mets% not reported0.15 to 0.22%% not reported2.343.73% (p<0.05 1D vs 3D) 0.75/0.70Marten et al. 2006 ADDIN EN.CITE Marten200636[14]363617Marten, K.Auer, F.Schmidt, S.Kohl, G.Rummeny, E. J.Engelke, C.Department of Radiology, Klinikum rechts der Isar, Technical University Munich, Germany. Katharina.Marten@roe.med.tum.deInadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteriaEur RadiolEur Radiol781-90164AdultAgedAnalysis of VarianceFemaleHumansImage Processing, Computer-AssistedLinear ModelsLongitudinal StudiesLung Neoplasms/ classification/ radiography/secondary/therapyMaleMiddle AgedNeoplasm MetastasisReproducibility of ResultsSoftwareTomography, X-Ray Computed/ methods20060938-7994 (Print)
0938-7994 (Linking)16331462200610.1007/s00330-005-0036-x [doi]Nlmeng[ HYPERLINK \l "_ENREF_14" \o "Marten, 2006 #36" 14]same scaninter-reader2502023.165195 mm3, median 182.22 mm3lung, mets% not reported0.22 to 0.29%% not reported3.533.76% (p<0.05 1D vs 3D)0.75/0.70Marten et al. 2006 ADDIN EN.CITE Marten200636[14]363617Marten, K.Auer, F.Schmidt, S.Kohl, G.Rummeny, E. J.Engelke, C.Department of Radiology, Klinikum rechts der Isar, Technical University Munich, Germany. Katharina.Marten@roe.med.tum.deInadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteriaEur RadiolEur Radiol781-90164AdultAgedAnalysis of VarianceFemaleHumansImage Processing, Computer-AssistedLinear ModelsLongitudinal StudiesLung Neoplasms/ classification/ radiography/secondary/therapyMaleMiddle AgedNeoplasm MetastasisReproducibility of ResultsSoftwareTomography, X-Ray Computed/ methods20060938-7994 (Print)
0938-7994 (Linking)16331462200610.1007/s00330-005-0036-x [doi]Nlmeng[ HYPERLINK \l "_ENREF_14" \o "Marten, 2006 #36" 14]same scaninter-reader22239422515500 mm3 (effective diameter 3.19.8 mm)lung, nodules -13.4 to 14.5%0.50%not reportednot reported1.0/0.7Wang et al. 2008 ADDIN EN.CITE ADDIN EN.CITE.DATA [ HYPERLINK \l "_ENREF_15" \o "Wang, 2008 #7906" 15]same scanintra-reader224528.5 mm (<5 to 18 mm)lung, noncalcified nodules8.9 % (upper limit)not reportednot reportednot reported1.25 or 2.5/not specifiedRevel et al. ADDIN EN.CITE Revel20047908[16]7908790817Revel, M. P.Lefort, C.Bissery, A.Bienvenu, M.Aycard, L.Chatellier, G.Frija, G.Department of Radiology, Assistance Publique des Hopitaux de Paris/INSERM, Georges Pompidou European University Hospital, 20 Rue Leblanc, 75015 Paris, France. marie-pierre.revel@hop.egp.ap-hop-paris.frPulmonary nodules: preliminary experience with three-dimensional evaluationRadiologyRadiologyRadiologyRadiologyRadiologyRadiology459-6623122004/05/07AdultAgedAged, 80 and overFemaleHumansImaging, Three-DimensionalLung Diseases/*radiographyMaleMiddle AgedObserver Variation*Tomography, X-Ray Computed/methods/statistics & numerical data2004May0033-8419 (Print)
0033-8419 (Linking)15128991http://www.ncbi.nlm.nih.gov/pubmed/1512899110.1148/radiol.2312030241eng[ HYPERLINK \l "_ENREF_16" \o "Revel, 2004 #7908" 16]same scaninter-reader (3 readers x 3 measurements)324528.5 mm (< 18 mm)lung, noncalcified nodules6.38 % (upper limit)not reportednot reportednot reported1.25 or 2.5/not specifiedRevel et al. ADDIN EN.CITE Revel20047908[16]7908790817Revel, M. P.Lefort, C.Bissery, A.Bienvenu, M.Aycard, L.Chatellier, G.Frija, G.Department of Radiology, Assistance Publique des Hopitaux de Paris/INSERM, Georges Pompidou European University Hospital, 20 Rue Leblanc, 75015 Paris, France. marie-pierre.revel@hop.egp.ap-hop-paris.frPulmonary nodules: preliminary experience with three-dimensional evaluationRadiologyRadiologyRadiologyRadiologyRadiologyRadiology459-6623122004/05/07AdultAgedAged, 80 and overFemaleHumansImaging, Three-DimensionalLung Diseases/*radiographyMaleMiddle AgedObserver Variation*Tomography, X-Ray Computed/methods/statistics & numerical data2004May0033-8419 (Print)
0033-8419 (Linking)15128991http://www.ncbi.nlm.nih.gov/pubmed/1512899110.1148/radiol.2312030241eng[ HYPERLINK \l "_ENREF_16" \o "Revel, 2004 #7908" 16]
Abbreviations: 1D = unidimensional; mets = metastasis; CI = confidence interval
The above table provides a basis for the 30% value in the Profile Claim. The range between the minimum and maximum values in the 95% CI of the measurement difference column is mostly within +/- 15%. Considering a large study from Wang et al using 2239 patients ADDIN EN.CITE ADDIN EN.CITE.DATA [ HYPERLINK \l "_ENREF_15" \o "Wang, 2008 #7906" 15], the 95% confidence interval ranged [-13.4%, 14.5%]. Thus, 30% is a conservative threshold of measurement variation. For example, the 30% change in the claim is the outside of 95% confidence interval of 15% of measurement variability when sample size is 40 or more.
Appendix C: Conventions and Definitions
Acquisition vs. Analysis vs. Interpretation: This document organizes acquisition, reconstruction, post-processing, analysis and interpretation as steps in a pipeline that transforms data to information to knowledge. Acquisition, reconstruction and post-processing are considered to address the collection and structuring of new data from the subject. Analysis is primarily considered to be computational steps that transform the data into information, extracting important values. Interpretation is primarily considered to be judgment that transforms the information into knowledge. (The transformation of knowledge into wisdom is beyond the scope of this document.)
Image Analysis, Image Review, and/or Read: Procedures and processes that culminate in the generation of imaging outcome measures, such tumor response criteria. Reviews can be performed for eligibility, safety or efficacy. The review paradigm may be context specific and dependent on the specific aims of a trial, the imaging technologies in play, and the stage of drug development, among other parameters.
Image Header: that part of the image file (or dataset containing the image) other than the pixel data itself.
Imaging Phantoms: devices used for periodic testing and standardization of image acquisition. This testing must be site specific and equipment specific and conducted prior to the beginning of a trial (baseline), periodically during the trial and at the end of the trial.
Time Point: a discrete period during the course of a clinical trial when groups of imaging exams or clinical exams are scheduled.
Tumor Definition Variability: the clarity of the tumor boundary in the images. It originates from the biological characteristics of the tumor, technical characteristics of the imaging process, and perhaps on the perception, expertise and education of the operator.
Technical Variability - originates only from the ability to drawing unequivocal objects. In other words, the perception of tumor definition is supposed absolutely clear and similar for any given operator when attempting to assess Technical variability.
Global Variability - partitioned as the variability in the tumor definition plus the Technical variability.
Intra-Rater Variability - is the variability in the interpretation of a set of images by the same reader after an adequate period of time inserted to reduce recall bias.
Inter-Rater Variability - is the variability in the interpretation of a set of images by the different readers.
Repeatability considers multiple measurements taken under the same conditions (same equipment, parameters, reader, algorithm, etc) but different subjects.
Reproducibility considers multiple measurements taken where one or more conditions have changed.
Appendix D: Model-specific Instructions and Parameters
For acquisition modalities, reconstruction software and software analysis tools, Profile compliance requires meeting the Activity specifications above; e.g. in Sections 3.2, 3.3 and 3.4.
This Appendix provides, as an informative annex to the Profile, some specific acquisition parameters, reconstruction parameters and analysis software parameters that are expected to be compatible with meeting the Profile requirements. Just using these parameters without meeting the requirements specified in the Profile is not sufficient to achieve compliance. Conversely, it is possible to use different compatible parameters and still achieve compliance.
Additional parameter sets may be found in QIBA Conformance Statements published by vendors and sites. Vendors claiming product compliance with this QIBA Profile are required to provide such instructions and parameters describing the conditions under which their product achieved compliance.
Sites using models listed here are encouraged to consider these parameters for both simplicity and consistency. Sites using models not listed here may be able to devise their own settings that result in data meeting the requirements. Tables like the following may be used by sites that wish to publish their successful/best practices.
In any case, sites are responsible for adjusting the parameters as appropriate for individual subjects.
Discussion:
It would likely be useful to include a description of the imaging subject in the following tables.
In terms of standardization, it may make sense to ask vendors to publish parameters for a known reference phantom as a stable benchmark for sites to adjust for individual patient variations.
Table D.1 Model-specific Parameters for Acquisition Devices
Acquisition DeviceSettings Compatible with Compliance
Submitted by:kVp
Number of Data Channels (N)
Width of Each Data Channel (T, in mm)
Gantry Rotation Time in seconds
mA
Pitch
Scan FoV
Table D.2 Model-specific Parameters for Reconstruction Software
Reconstruction SoftwareSettings Compatible with Compliance
Submitted by:Reconstructed Slice Width, mm
Reconstruction Interval
Display FOV, mm
Recon kernel
Table D.3 Model-specific Parameters for Image Analysis Software
Image Analysis SoftwareSettings Compatible with Compliance
Submitted by:a
b
c
d
QIBA Profile Format 2.1
QIBA Profile Format 2.1
Document generated by .\Profile Editor\ProfileTemplate.spsPage:
Document generated by .\Profile Editor\ProfileTemplate.spsPage:
QIBA Profile Format 2.0
Document generated by .\Profile Editor\1. sharable sections\ProfileTemplate.spsPage: Document generated by .\Profile Editor\1. sharable sections\ProfileTemplate.spsPage:
QIBA Profile Format 2.1
QIBA Profile Format 2.1
Document generated by .\Profile Editor\ProfileTemplate.spsPage:
Document generated by .\Profile Editor\ProfileTemplate.spsPage:
QIBA Profile Format 2.0
Document generated by .\Profile Editor\1. sharable sections\ProfileTemplate.spsPage: Document generated by .\Profile Editor\1. sharable sections\ProfileTemplate.spsPage:
Concept of screening has been lost. Reconsider??
does QIBA want to get into this aspect of things.
It is really about growthchange in volume over time (since growth can be confused with measurement artifact)
QIBA did not seem interested in this aspect of things
Placeholder for future discussion about Claim, after technical parameters are decided.
Need to perform similar review in the various screening cohorts
Imaging Agent should be removed from diagram
What is the (2) for?
Why are there multiple background boxes?
Not my experience
IS this within our scope?
Reconsider this and allow 4 or higher if scan can be completed in a single breath hold with pitch