Process Coordinating Committee
- Chairs: Kevin O'Donnell (Chair); Dan Sullivan, MD (Vice-chair)
- RSNA Staff Support: Susan Stanfa
The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.
- Develop common Templates & Processes
- Support & mentor adoption of above
- Drive consistent content & format of Profiles
- Support infrastructure for Committees
- Upcoming: Biweekly Tcons, 1st and 3rd Wednesdays of each month @ 3-4pm Central Time (Contact Susan for details)
Approved call summaries:
- Wiki Use Cases - https://docs.google.com/spreadsheets/d/1bVP9pSjxtCVinlASCAeGDf5vjc2lCfj16qEg7hkft0Q/edit?usp=sharing
- Draft standing agenda item for Coordinating Cmtes to remind/confirm BC committees keeping their BC Wiki page up to date (especially links to current draft documents)
- Process Cmte will do a survey about every 6 mo and feed results to the Steering and Coordinating Cmtes. (Could add row to dashboard)
- Public Comment resolution
- Set up Wiki page for holding all the comment resolution records from past public comment. - DONE Profiles
- Could build it into the automatic confirmation message that the confirmation form sends out. (Susan)
- Sketch Committee Sunset Process for Task Forces and Biomarker Committees - DONE
- On-boarding process
- Goal: Help new committees, new members, new authors, new reviewers, new adopters
- Reviewed "How to develop a Profile" Guidance
- Email How-To guide to current authors and ask them to read/feedback
- Then email to the current community since it is helpful for all contributors
- TODO update Profiles and Protocols page to focus less on UPICT and more on QIBA Profiles today.
- TODO Add some "How to Read a Profile" guidance (also useful for reviewers), e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
- Explain about the Claims, the Actors, the Requirements and the Assessment Procedures structure of a Profile
- Advice may vary depending on viewpoint of the reader.
- Joe's intro email, current template and stage docs, QIBA Fact Sheet
- Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
- Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.
- Include link to How-To guide
- Pictures and diagrams would help a lot for people reading what is "dry" text.
- Types of newcomers/stakeholders - link from an "intro to QIBA - mother page"
- Practicing Radiologists (Dan), physicists (Ed), technologists (Mike), imaging vendors (Kevin), analysis SW vendors/developers (Cathy?), researchers, clinical trialists, pharma (Gudrun?), core lab (Pat?), regulatory staff/reviewers (both device and drug?) (Nick), patients/advocates/lay public, government policy, payers, statisticians (Nancy)
- TODO targeted intro to each?groups? of those viewpoints
- address key value of QIBA to their job, how they would make use of QIBA (some of them are "actors", some are not)
- easy language, a few paragraphs, and links to specific wiki pages/documents/slides that would be a good next step for them
- Review Executive Cmte charter to look for any process changes/additions
- Contact page on wiki? Point to firstname.lastname@example.org for questions/info on any committee, QIDW, etc.
- Profile Selection Process
- Review/update review criteria/evaluations for new BC proposals
- Harvest from past experiences
- Given limited resources, how to approve not just good proposals but the best proposals
- Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
- timing overlap of prior cycle completion period with new proposal period needs to be one considered.
- Communicating Uncertainty to Radiologists
- "No measurement is complete without a statement of it's uncertainty" - NIST
- Measurement uncertainty is at the heart of what QIBA does
- If people don't recognize/appreciate measurement uncertainty they can't grasp the value of QIBA fully (or at all?)
- How do we get radiologists to appreciate the uncertainty of each measurement and incorporate it into their thinking and decision making?
- What should best practice be for communicating the uncertainty of each measurement to radiologists?
- Always give a confidence interval rather than a value?
- Adoption of Profiles
- Maybe a new TF to support Adoption (can include some Process people)
- Ed, Brad, Joe, Kevin, Dan, RSNA Webperson
- Start by looking at what is on the RSNA QIBA pages,
- Conformance Process
- What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
- Then we need to ensure that the QIBA Templates/Process supports those things clearly.
- Kevin will put some initial thoughts on a Wiki page and we can review/add
- How often should conformance assessment be done, who will assess, who will keep records?
- Clinical Confirmation Process
- Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"
- QIBA Tooling
- How to use Google Docs?
- Issues with references, access restricted by federal agencies and some companies
- Round trip fidelity between Word-Google-Word
- QIBA Wiki
- QIBA Wiki Improvements
- Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).
- Wiki vs RSNA.org
- Consider how to handle handoffs between RSNA website
- Literature Search
- Assemble draft guidance to kick this off (Hendrik Laue, Ona Wu, Caroline? Chung)
- Where do you store references, what kind of search works best, what exactly are you looking for?
- Also meta-analysis guidance to determine sources and levels of variance
- Profile Maintenance Process
- Propose change tracking process
- Propose conventions for Profile editions/versions
- Guidelines for Reference Implementations and commercial products in Profiles
- (underlying algorithm available, list all comers, non-commercial?, etc)
- TODO add detail
- Public Comment attribution
Todo items that are on hold pending some trigger:
- Claim Guidance
- Hold pending people with questions/clarification needs on current text
- Profile Template
- Hold pending new suggestions/clarification needs
- Harmonize assessment procedures across profiles
- Profile Retirement Process
- Hold pending a profile we want to retire
- When do we "withdraw" or at least stop maintaining/promoting a Profile
- Review DICOM retirement concept (stays available in archives, not maintained/promoted)
- Profile Disclaimer
- Hold pending revist of Discriminatory Claims
- Statistical Assumption Testing Guidance
- Conformance with the statistical assumptions is required at the various QIBA profile stages.
- Symmetrical vs Assymetrical change intervals?
- Questions about the numbers in "Academic Radiology: ?? Paper"
This page is based on the Committee Template. Add any good ideas you have to the template.