Profile Development

QIBA fMRI Subcommittee Draft Profile for Brain Mapping

June 24, 2010 Dr Elsinger

Draft QIBA profile for fMRI brain mapping: Initial Draft Version last submitted by EAD, edits in blue are JP, suggestions in red CE.

1. Intended use: fMRI is used as a tool for pre-treatment planning in individual patients with brain lesions, including tumors, vascular malformation and epileptogenic foci. The presenting symptoms and location of the affected brain tissue determine the particular region or regions of the brain to be mapped and the behavioral paradigm(s) selected (e.g. motor task, language task). The change in BOLD signal (relative to a control condition) provides information about the brain region(s) involved in task performance and about the proximity of this eloquent cortex to brain site(s) to be treated. Endpoints that will influence treatment planning include risk assessment (impact of treatment on functioning cortex, e.g. surgical or radiation induced damage) and predictive value estimation (will damage to eloquent tissue result in a deficit). The goal of this profile is to specify the procedures and quantitative parameters under which BOLD fMRI is an accurate and reliable predictor of brain function, that is, as a valid imaging biomarker for medically meaningful changes in brain activity elicited by a particular task.

2. Claims

A. On a test-retest basis, fMRI can be performed reproducibly to a level such that the center of mass of activation of a focus of interest is within 5mm of itself, with at least 90% overlap of the activation clusters.

B. On a test-retest basis, fMRI can be performed reproducibly to a level such that the relative magnitude of activation in homologous regions across hemispheres should be within 10%.

C. Quantitative measures of “risk” to eloquent brain structures… distance metrics… etc. 3. Sources of Variance:

4. Protocol

A. Patient evaluation and fMRI paradigm selection B. Paradigm design C. Patient training on behavioral task and quantitative assessment of performance D. Equipment Q/A checks and calibrations E. Patient preparation, instructions, behavioral check, adjustment of peripheral equipment F. Pre-scan setup, shimming, selection of slices and other imaging parameters G. Performance of fMRI scans including patient task instructions, performance monitoring H. Post-scan evaluations of alertness, performance I. Post processing, artifact detection etc. J. Report generation and content, including technical Q/A and evaluation K. Clinical interpretation L. Archiving and export to treatment systems M. Follow-up

5. Compliance checks

fMRI signals reflect changes in local blood oxygenation and volume triggered by net changes in local neuronal activity. Such signals are therefore an indirect measure of brain activity. Moreover, the functional specificity of the fMRI activity depends on the behavioral task paradigm and the patient’s ability to perform the task. Thus, fMRI signals and the ability to identify functionally-specific eloquent cortex can be compromised by:

2. Behavioral compliance – Since the fMRI signal depends on the patient’s ability to perform the required behavioral task and thereby activate neurons in task-related brain areas, the occurrence, magnitude and extent of fMRI activation can vary in proportion to the accuracy, reliability, speed, etc of the patient’s performance.

3. Consistency over time of activation cluster 4. Detectability of signal changes in phantom 5. Temporal stability of signal changes in phantom (1/week)

QIBA Profiles consist of: 1. Intended use (clinical context), e.g. risk assessment or clinical trials 2. Claims (level of performance, both state-of-the-art and areas for improvement) 3. Protocol (structure provided by UPICT template) 4. Compliance check It tells a user what can be accomplished by following the Profile. ("Profile Claims") E.g. you will be able to detect volume changes of greater than <20%> in Stage IV Lung Nodules which are <5mm> in diameter or greater. It tells a vendor what they must implement in their product to state compliance with the Profile. ("Profile Details") E.g. to comply, the scanner must be able to:

	scan a <Mark-324> Chest Phantom, identify the smallest resolvable target, display the diameter of that target 
	demonstrate resolving targets at least as small as <2mm> diameter on the Mark-324 phantom 
	scan patients according to the ACRIN NLST acquisition protocol 

E.g. to comply, the quantification application must be able to:

	segment a nodule (automatically or manually), derive the volume, store it in a DICOM object 
	run a user through a set of test data with known volumes and at the end display an accuracy score 

It may also tell the user staff what they must do for the Profile Claims to be realized. ("Profile Details") E.g. to comply, the site CT techs must be able to:

	scan the patient within 10 minutes of contrast injection 

E.g. to comply, the radiologist must be able to:

	achieve a score of <95%> or better using their segmentation application on the <Lung Invaders> test set. 

From the UPICT Template Acquisition vs. Analysis vs. Interpretation This document organizes acquisition, reconstruction, post-processing, analysis and interpretation as steps in a pipeline that transforms data to information to knowledge. Acquisition, reconstruction and post-processing are considered to address the collection and structuring of new data from the subject. Analysis is primarily considered to be computational steps that transform the data into information, extracting important values. Interpretation is primarily considered to be judgment that transforms the information into knowledge. (The transformation of knowledge into wisdom is beyond the scope of this document.)

Bulls-eye Compliance Levels Acquisition parameter values and some other requirements in this protocol are specified using a “bullseye” approach. Three rings are considered from widest to narrowest with the following semantics: ACCEPTABLE: failing to meet this specification will result in data that is likely unacceptable for the intended use of this protocol. TARGET: meeting this specification is considered to be achievable with reasonable effort and equipment and is expected to provide better results than meeting the ACCEPTABLE specification. IDEAL: meeting this specification may require unusual effort or equipment, but is expected to provide better results than meeting the TARGET. An ACCEPTABLE value will always be provided for each parameter. When there is no reason to expect better results (e.g. in terms of higher image quality, greater consistency, lower dose, etc.), TARGET and IDEAL values are not provided. Some protocols may need sites that perform at higher compliance levels do so consistently, so sites may be requested to declare their “level of compliance”. If a site declares they will operate at the TARGET level, they must achieve the TARGET specification whenever it is provided and the ACCEPTABLE specification when a TARGET specification is not provided. Similarly, if they declare IDEAL, they must achieve the IDEAL specification whenever it is provided, the TARGET specification where no IDEAL level is specified, and the ACCEPTABLE level for the rest.

QIBA fMRI Subcommittee Profile Draft

March 22, 2010

Cathy Elsinger (co-chair)

QIBA Profile Draft:

Intended Use: fMRI is used as a tool for diagnosis and pre-treatment planning in patients with brain disorders. The presenting symptoms and location of affected brain tissue determine the particular region or regions of the brain to be investigated and the stimulus paradigm(s) selected (e.g. motor task, language task). The change in BOLD signal (relative to a rest condition) provides information about the brain region(s) involved in task performance and the proximity of this functioning cortex to brain tissue to be treated. Pre-treatment planning endpoints involve risk assessment (impact of treatment course on functioning cortex, e.g. surgical pathway, tissue resection, or off-target irradiation) and predictive value assessment (will loss of a particular region of functioning cortex result in a deficit) that influence patient treatment. Our goal is to determine the quantitative value of BOLD fMRI as predictor of cortical function and/or essential function and thus a medically meaningful surrogate for change in brain activity - a valid imaging biomarker.

Assumption: It is appropriate for the QIBA fMRI subcommittee to work towards development of technical and practical guidelines which cover data acquisition, statistical analysis and paradigm design procedures in order to reduce variability, increase repeatability and reliability in BOLD fMRI exam administration and the quantitative measures derived. An ultimate goal is to arrive at a set of quantitative measures that can be used to validate/verify appropriateness of stimulus paradigms and associated protocols for future clinical applications and clinical research trial applications.

Short Term Goal: QIBA fMRI subcommittee to establish guidelines and recommendations to optimize acquisition parameters, paradigm design, and statistical analysis for single-subject BOLD fMRI exams to increase quality control, clinical utility and characterize clinical uses of fMRI which are appropriate. Initial test case will be mapping of motor cortex for presurgical planning purposes.

Long Term Goal: Individual QIBA Profiles can address different use-cases, which could be expanded and addressed through different subgroups (e.g.,Presurgical planning endpoint involves risk assessment and predictive value assessment for patient undergoing surgery while a pharma/clinical trial involves therapeutic effects/response)

QIBA Profiles consist of:

1. Intended use (clinical context), e.g. risk assessment or clinical trials

2. Claims (level of performance, both state-of-the-art and areas for improvement)

3. Protocol (structure provided by UPICT template)

4. Compliance check

QIBA Profile Draft:

Intended Use: fMRI is used as a tool for presurgical planning in patients undergoing brain surgery. The location of the brain tissue to be resected determines the particular region or regions of the brain to be investigated and the stimulus paradigm(s) selected (e.g. motor task, language task). The change in BOLD signal (relative to a rest condition) provides information about the brain region(s) involved in task performance and the proximity of this eloquent cortex to brain tissue to be resected. Presurgical planning endpoints involve risk assessment (how close is tissue to be resected to areas of eloquent cortex) and predictive value assessment (will removal of eloquent cortex result in a deficit) that influence patient surgery. Our goal is to determine the quantitative value of BOLD fMRI as predictor of cortical function and/or essential function and thus a medically meaningful surrogate for change in brain activity - a valid imaging biomarker.

Claim 1 – with the appropriate MR compatible peripheral equipment and software for stimulus delivery, patients can perform cognitive or sensorimotor tasks while they are in the MR scanner, and this behavior can be measured.

Claim 2 – the behavior elicited from the patient can be synchronized with MR image acquisition using the appropriate pulse sequence and the statistical result (change in MR signal) can be measured.

Claim 3 – The statistical result (change in MR signal) is quantifiable in terms of (1) magnitude of task-related activity relative to baseline with a statistical parameter such as F or T (and its p value); (2) temporal correlation of the signal with the task timing; (3) the spatial pattern of activation (density, clustering); and (4) proximity to a site of pathology or proposed resection margin.

Subgoals for development activities:

1. Guidelines for acquisition of fMRI data for presurgical mapping (claims 1 & 2)

2. Guidelines for processing and statistical analysis of data (quality control analysis and determination of degree of tolerable variability, i.e., what is clinically acceptable variance?) (claim 3)

3. Guidelines for design of clinically appropriate task paradigms based on clinical application, e.g. motor mapping (what are feasible motor task for motor cortex?) or language mapping (what are feasible language paradigms for laterality?)

4. Explore methodology (optimal vs. workable); QIBA reflects a bull’s eye approach of acceptable, target and ideal performance levels

How do we develop a roadmap for the groundwork to be done in terms of the characterizing the technical standards or methodology to be detailed, determining the accuracy and reproducibility of the measures acquired or the level of performance that would be considered acceptable, thus improving the clinical efficacy of fMRI exams?

Retrospective - What current state of the art is as reflected in the literature?

Collaborative analytical work with data derived from the ASFNR study?

Prospective studies?

Discussion surrounding initial Claim sentence:

To create a method for validating/optimizing design parameters of stimulus paradigms used in single-subject BOLD studies.

Objective:

Characterize the robustness of signal change measurements in brain tissue that will lead to an understanding of the minimal threshold required to classify MR signal change in eloquent tissue as medically meaningful surrogate(s) for changes in brain activity.

Dr. DeYoe's Claim Comments

Initial Proposed claim:

Initial Claim Sentence: Goal is to investigate the technical feasibility of creating a method for validating/optimizing design parameters of stimulus paradigms used in single-subject BOLD studies.

Discussion surrounding initial Claim sentence: To create a method for validating/optimizing design parameters of stimulus paradigms used in single-subject BOLD studies.

Objective: Characterize the robustness of signal change measurements in brain tissue that will lead to an understanding of the minimal threshold required to classify MR signal change in eloquent brain tissue as medically meaningful surrogate(s) for changes in brain activity.

Discussion notes from E. DeYoe:

To establish a context for the claim(s) that QIBA will address, we could propose an overall claim that would seem to be clinically relevant such as:

Claim 1: fMRI brain activation is a valid indicator (biomarker) of brain function in individual patients.

For specific paradigms this might be cast as:

Claim 2: fMRI activation within brain area X produced by paradigm Y is a valid indicator of the function of area X. (which for presurgical mapping could be further restricted to mean that excision or damage of the area would produce a neurological deficit.)

However, the focus of the QIBA group might be more restricted: Specifying specific measures (eg readouts) that could be used to address the preceding claims and establishing the characteristics of those measures.

Claim 3: For specific fMRI paradigms, the following measures of the BOLD signal can be used to determine if Claim 2 is true (How the measures are used to do this must also be specified for each paradigm).

• (1) The intensity of task relative to baseline with a statistical parameter such as F or T (and its p value)
• (2) The temporal correlation of the signal with the task timing (eg block design, event related, phase mapped).
• (3) The spatial pattern of activation (density, clustering) and proximity to a site of pathology or proposed resection margin).
• (4) Paradigm-specific indices derived from the BOLD signals.

For Claims 2 and 3 to be true, certain conditions or criteria must be met:

• a. The scanner, and all associated peripherals are operating properly and this can be verified

• b. The nature of the behavioral task is appropriate to activate the brain area(s) of interest. (This includes appropriate selection of a “baseline” condition against which the activation is detected.)

• c. The parametric design of the fMRI task and the selection of scanner pulse sequence and parameters are appropriate and, preferably optimal.

• d. The patient performs the behavioral task adequately and this can be verified. (This includes monitoring behavioral “state” variables such as attention and alertness.)

• e. A valid computational method exists to obtain the various measures of the BOLD response from voxel signals and determine their statistical significance. (This means understanding the statistical sampling distributions of each BOLD measure and thus their reproducibility.)

• f. Statistical acceptance criteria can be established that ensure that Claim 2 is valid at a specified level of probability.

• g. The BOLD hemodynamic mechanism within the brain region of interest is not compromised (eg. there is no neurovascular uncoupling.)

Additional comments: Ultimately for surgical planning/guidance Claim 2 can be interpreted farther still to mean: If brain area X is to be surgically removed or compromised, does the fMRI activation predict whether a corresponding neurological deficit will occur? This is probably beyond the scope of what the QIBA group wants to address? Rather, we might focus primarily on conditions b-e listed above and assume that conditions a,f,g are met. Condition f is a sticky one because from a clinical perspective it not simply sufficient to say that a BOLD signal is statistically different from the baseline since this does not necessarily indicate that the associate brain region is functioning normally. However, our goal may be just to ensure that a statistical measure is mathematically accurate given the statistics of the task and baseline conditions.

As discussed in the recent phone conference there are several “levels” of technological and neurological issues that should be clearly differentiated. QIBA may only want to address some of these:

• Level A. fMRI measurement and paradigm technology
• Level B. Paradigm design and appropriateness for activating particular brain regions
• Level C. The contribution of particular brain regions to specific behavioral or mental functions. This includes recognition that a given area may contribute to more than one behavior or mental function.
• Level D. Clinical interpretation of A-C relative to a particular patient’s pathology, treatment choices, and potential outcomes (eg amelioration of pathology and quality of life).

It may only be appropriate for the QIBA group to address A and B?

An alternate way to cast the QIBA specific claims would be to focus on establishing the criteria for evaluating a particular paradigm and the BOLD measures derived from it.

Dr. Tucker's Claim Comments

CLAIMS: 'Used appropriately' fMRI can provide useful information within the clinical contexts of pre-surgical mapping and disease diagnoses.

OBJECTIVE: The objective of the QIBA group is to characterize clinical uses of fMRI which are appropriate and to develop a standard of best practices (guidelines) pertaining to - paradigm development, representation, and management, - data acquisition, analysis and interpretation, and - facilities outfitting, staffing and management that will guide those clinical users (not scientists) to use fMRI in the most appropriate manner for their clinical context.

This largely boils down to answering questions generated by the usual suspects: who, what, when, where and how

• Who should be involved in these studies? This should address both the patient and the personnel conducting the study
  • For the patient, things like screening for suitability (can they follow directions? Can they actually move their fingers? Etc).
  • For personnel conducting the exam, what are the roles? For each role, what are the qualifications? Etc.
• What should be done? This should be stated in the context of goals for the study (for example, adult studies vs peds). This should include things like 'paradigm design should include measureable patient responses' so as to assess whether the patient is actually performing the task, and 'patient responses should be recorded' for obvious reasons. The fMRI exam should not be a fishing experiment -- that will lead to no good. There was lots of this discussion today cast in light of 'do I have enough data?' (When is it appropriate to trade clinical efficiency for …?)
• When should an exam be performed? There are competing technologies out there; when is fMRI the right choice? Is it a diagnostic tool? Or therapeutic management? Or something else?
• Where should it be conducted? Inpatient? Outpatient? Is 0.5T okay? Do I need a dedicated stimulation system? Is flashing the room lights okay?
• How should it be conducted? What are the steps involved? In setting up a program? Do I need dedicated personnel? Equipment? How should I conduct an exam? Is there a 'workflow' I can follow (perhaps checklists I can use to make sure I've done everything I need to do'

RATIONALE: QIBA needs to focus on use of the technology, and to standardize that practice.

• At the two extremes:
  • I think it is not appropriate for the QIBA group to validate clinical utility (indications for use, etc). That is for the professional societies (like ASFNR) to develop. QIBA should support that activity (i.e. for the proposed multi-site fMRI study, paradigms and data collection should be regimented in a manner consistent with QIBA, etc).
  • fMRI is not a quantitative procedure. It’s a big research project to try to establish fMRI as a quantitative test – nothing this group will ever solve. Trying to associate absolute measures (A Z score >= N, or activated volume >= V cc's) with a paradigm is inappropriate; it depends too much on the subject, the environment, etc.

IMO -The 'sweet spot' for QIBA lies between these ends...