Process Coordinating Committee

Chairs: Kevin O'Donnell (Chair); Dan Sullivan, MD (Vice-chair)

Secretary: Susan Weinmann (RSNA Staff)

Roster: <link to roster of active attendees and who has voting privileges>

Scope

The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.

• Develop common Templates & Processes
• Support & mentor adoption of above
• Drive consistent content & format of Profiles
• Support infrastructure for Committees

Meetings

• Upcoming: Biweekly Tcons, Wednesdays @ 3-4pm Central Time (Contact Susan for details)
• Summaries of Past Calls/Meetings

Working Documents

• Claim Guidance
• Assessment Procedure Guidance

• QIBA Profile Stages

• Wiki Use Cases - https://docs.google.com/spreadsheets/d/1bVP9pSjxtCVinlASCAeGDf5vjc2lCfj16qEg7hkft0Q/edit?usp=sharing

Current Work

• Conformance Process

• What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
• Need a common view of what conformance entails, how it is achieved, what it looks like to a product, how it is documented, etc.
• Then we need to ensure that the QIBA Templates/Process supports those things clearly.
• Kevin will put some initial thoughts on a Wiki page and we can review/add
• How often should conformance assessment be done, who will assess, who will keep records?
• Should QIBA get into hosting the Conformance Statements on our site (and then we could link to them from the Profiles page which would be very handy)
• Consider the RSNA Image Share Conformance Assessment Program and whether we should do something similar for QIBA

• Process committee can think through the details of a process and confer with RSNA whether they would be interested

• Profile Template

• Adjust Profile Template (change tracked) to regroup time-muddled Activities
• Conformance Section - connect the dots
• Embed the checklist as an appendix

• Template 2017.04.19.b Draft

• Support adoption of the updated Profile Template.
• Harmonize assessment procedures across profiles

• Statistical Assumption Testing Guidance

• Conformance with the statistical assumptions is required at the various QIBA profile stages.
• Need guidance on the assumptions in the profile, validating the assumptions and testing actors for meeting those assumptions.
• Symmetrical vs Assymetrical change intervals?
• Questions about the numbers in "Academic Radiology: ?? Paper"

• Clinical Confirmation Process

• Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"

• Committee Operations

• Coordinating Cmtes need to remind/confirm BC committees keep their page up to date (especially links to current draft documents)
• Add as standing agenda item on the Coordinating Cmte call to check with their Biomarker Cmte chairs
• Sketch a Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
• Propose policy/process to consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.

• timing overlap of prior cycle completion period with new proposal period needs to be one considered.
• possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal

• QIBA Tooling

• How to use Google Docs?

• Issues with references, access restricted by federal agencies and some companies

• Round trip fidelity between Word-Google-Word

• QIBA Wiki

• QIBA Wiki Improvements
• Consider how to handle handoffs between RSNA website
• Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).

• Maybe a new TF to support Adoption (can include some Process people)
• Ed, Brad, Joe, Kevin, Dan, RSNA Webperson

• Start by looking at what is on the RSNA QIBA pages,

• Literature Search

• Ask for a status update?
• Process/best practices for initial literature search around a new biomarker/profile
• Does someone have a draft set of guidance to kick this off? (Nancy, Anna Wu, Hendrik Lau, Andy)
• Where do you store references, what kind of search works best, what exactly are you looking for?
• Also meta-analysis guidance to determine sources and levels of variance
• PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible

• Profile Selection Process

• Consider additional criteria/evaluation based on experiences

• Profile Maintenance Process

• Propose change tracking process
• Propose conventions for Profile editions/versions

• Profile Retirement Process

• When do we "withdraw" or at least stop maintaining/promoting a Profile
• Review/borrow from corresponding IHE Process - http://wiki.ihe.net/index.php/Evaluation_of_Published_Profiles
• Review DICOM retirement concept (stays available in archives, not maintained/promoted)

• Publication Process

• Sketch up publication process for Official Docs (CC approval vote, send to RSNA Staff)
• Write up Standard Approval Practices for committee documents

• Quality check mechanism, voting mechanism

• Start simple with this and add complexity only as needed.
• Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)

• Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.

• On-boarding process

• Help new committees, new members, new authors, new reviewers, new adopters
• Add some "readers guide" text for reviewers, e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
• "How to develop a Profile" Guidance
• Joe's intro email, current template and stage docs, QIBA Fact Sheet
• Could use a "fresh eyes"/newcomer commentary

• Dan will send the current Fact Sheet to UCSF for feedback on contents
• Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.

• Guidelines for Reference Implementations and commercial products in Profiles (underlying algorithm available, list all comers, non-commercial?, etc).
• Guidelines for assessment procedures to vet SW in Profiles
• Clarify QIBA Profile vs UPICT Protocol (Venn Diagram)

• Claim Guidance

• Next revision cycle (2017Q3?)

• Try to include a greater variety of modalities/targets.
• See if we have come across a good example for TDI based CI (e.g. Scenarios BDFMNO)
• PDFF will bring in a new style with a bias and slope that isn't 1.
• MRE also different. Getting absolute measure not relative? Longitudinal claim. Assuming linearity based on some data, but not assessing yet.

• Profile Disclaimer

• Revist if Discriminatory Claims are addressed.
• Consult RSNA about what disclaimer should address
• Draft standardized disclaimer text for profiles
• If QIBA did Discriminatory Claims, then there would be a stronger need for disclaimer, but lacking clinical outcome statments, maybe we don't have as strong a need? We stick to technical performance.

See Also

This page is based on the Committee Template. Add any good ideas you have to the template.