Process Coordinating Committee
- Chairs: Kevin O'Donnell (Chair); Michael Boss, PhD (Vice Chair)
- RSNA Staff Support: Susan Stanfa
The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.
- Develop common Templates & Processes
- Support & mentor adoption of above
- Drive consistent content & format of Profiles
- Support infrastructure for Committees
- Upcoming: Biweekly Tcons, 1st and 3rd Wednesdays of each month @ 3-4pm Central Time (Contact Susan for details)
Approved call summaries:
- Wiki Use Cases - https://docs.google.com/spreadsheets/d/1bVP9pSjxtCVinlASCAeGDf5vjc2lCfj16qEg7hkft0Q/edit?usp=sharing
- Biomarker Adoption Steps: https://docs.google.com/document/d/1YM0mreBRLNNQZicDatx1IE3Zi1NZeaojQJczrEIbvsk/edit?usp=sharing
- Basis for reviewing organization/content/structure of current QIBA documents/deliverables
- Update/Iterate Profile Writing Guidance
- Ideas on what to add/review
- Conformance Process
- What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
- Then we need to ensure that the QIBA Templates/Process supports those things clearly.
- Kevin will put some initial thoughts on a Wiki page and we can review/add
- How often should conformance assessment be done, who will assess, who will keep records?
- Draft standing agenda item for Coordinating Cmtes to remind/confirm BC committees keeping their BC Wiki page up to date (especially links to current draft documents)
- Maybe 5-10 minute slot
- 1st - highlight guidelines page and items that keep profile "slim"/concise
- Claim statistics - particularly new BC's
- How to make/use/ a checklist with a Profile - avoid heterogeneity, its a re-sort, not different content, not a superset, not a subset
- Ask BC Chairs to take it back to their BCs. A BC can ask for additional participation from Process Cmte as needed.
- Some problems with lack of BC representation on CC.
- Dan & RSNA staff can work out getting a slot on upcoming calls.
- Close by noting they can ask for future topics to be covered
- Thoughts about process improvements
- Dashboard item to confirm representative(s) for major stakeholder groups: Radiologists (incl. academic, private practice (might challenging in terms of time) but if we get ACR traction, and QIBs are billable, interest might grow), Physicists, Technologists, Scanner vendors, SW vendors, CRO (implications for recruitment, burden for conducting), Pharma (biological use of the metric, claim details), Statisticians, Regulatory (device regulators, biologic regulators), ...
- When you have a "gap", get them during public comment, or periodically on a focused call, invite to face-to-face meetings,
- Q. If there are multiple, who do you list?
- Public Comment resolution
- (DONE) Could build it into the automatic confirmation message that the confirmation form sends out.
- TODO get additional profiles to post their resolutions
- On-boarding process
- Goal: Help new committees, new members, new authors, new reviewers, new adopters
- Reviewed "How to develop a Profile" Guidance
- Email How-To guide to current authors and ask them to read/feedback
- Then email to the current community since it is helpful for all contributors
- TODO update Profiles and Protocols page to focus less on UPICT and more on QIBA Profiles today.
- Linear layout
- Explain the mechanics of who does what with a profile (vendors, rads, techs, regulators, etc.)
- Advice may vary depending on viewpoint of the reader.
- Joe's intro email, current template and stage docs, QIBA Fact Sheet
- Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
- Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.
- Include link to How-To guide
- Pictures and diagrams would help a lot for people reading what is "dry" text.
- DONE targeted intro to each?groups? of those viewpoints
- TODO First Steps for each page (see QIBA for Statisticians for example)
- Contact page on wiki? Point to firstname.lastname@example.org for questions/info on any committee, QIDW, etc.
- Profile Selection Process
- Review/update review criteria/evaluations for new BC proposals
- Harvest from past experiences
- Given limited resources, how to approve not just good proposals but the best proposals
- Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
- timing overlap of prior cycle completion period with new proposal period needs to be one considered.
- Committee Leadership Process
- Address BC and CC leadership
- Stakeholder diversity (silent, recommend, or require)
- Variability in the size of some committees so tough to enforce
- For comparison - DICOM:
- For the DSC (Steering Cmte) it's more constrained
- The Committee has the right to recall a co-chair.
- For comparison - IHE:
- For comparison - RSNA:
- A nomination announcement contains a link to an online nomination form and deadline
- Communicating Uncertainty to Radiologists
- "No measurement is complete without a statement of it's uncertainty" - NIST
- Measurement uncertainty is at the heart of what QIBA does
- What should best practice be for communicating the uncertainty of each measurement to radiologists?
- Always give a confidence interval rather than a value?
- Adoption of Profiles
- Maybe a new TF to support Adoption (can include some Process people)
- Ed, Brad, Joe, Kevin, Dan, RSNA Webperson
- Start by looking at what is on the RSNA QIBA pages,
- Clinical Confirmation Process
- Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"
- QIBA Tooling
- How to use Google Docs?
- Issues with references, access restricted by federal agencies and some companies
- Round trip fidelity between Word-Google-Word
- QIBA Wiki
- QIBA Wiki Improvements
- Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).
- Wiki vs RSNA.org
- Consider how to handle handoffs between RSNA website
- Literature Search
- Assemble draft guidance to kick this off (Hendrik Laue, Ona Wu, Caroline? Chung)
- Where do you store references, what kind of search works best, what exactly are you looking for?
- Also meta-analysis guidance to determine sources and levels of variance
- Profile Maintenance Process
- Propose change tracking process
- Propose conventions for Profile editions/versions
- Guidelines for Reference Implementations and commercial products in Profiles
- (underlying algorithm available, list all comers, non-commercial?, etc)
- TODO add detail
- Public Comment attribution
Todo items that are on hold pending some trigger:
- Claim Guidance
- Hold pending people with questions/clarification needs on current text
- Profile Template
- Hold pending new suggestions/clarification needs
- Harmonize assessment procedures across profiles
- Profile Retirement Process
- Hold pending a profile we want to retire
- When do we "withdraw" or at least stop maintaining/promoting a Profile
- Review DICOM retirement concept (stays available in archives, not maintained/promoted)
- Profile Disclaimer
- Hold pending revist of Discriminatory Claims
- Statistical Assumption Testing Guidance
- Conformance with the statistical assumptions is required at the various QIBA profile stages.
- Symmetrical vs Assymetrical change intervals?
- Questions about the numbers in "Academic Radiology: ?? Paper"
This page is based on the Committee Template. Add any good ideas you have to the template.