Difference between revisions of "Process Coordinating Committee"

From QIBA Wiki
Jump to navigation Jump to search
Line 18: Line 18:
'''Approved call summaries:'''
'''Approved call summaries:'''
[[Media:2019_02-05_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|February 5, 2019]]
*[[Media:2019_01-15_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|January 15, 2019]]
*[[Media:2019_01-15_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|January 15, 2019]]
*[[Media:2018_12-18_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|December 18, 2018]]
*[[Media:2018_12-18_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|December 18, 2018]]
*[[Media:2018_12-04_QIBA_Process_Cmte_Call_Summary-FINAL.pdf|December 4, 2018]]
'''''[[Process Committee Call Summaries Archive]]'''''
'''''[[Process Committee Call Summaries Archive]]'''''

Revision as of 14:58, 20 February 2019

Chairs: Kevin O'Donnell (Chair); Dan Sullivan, MD (Vice-chair)
Secretary: Susan Stanfa (RSNA Staff)


The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.

  • Develop common Templates & Processes
  • Support & mentor adoption of above
  • Drive consistent content & format of Profiles
  • Support infrastructure for Committees

Meetings/Call Summaries

  • Upcoming: Biweekly Tcons, 1st and 3rd Wednesdays of each month @ 3-4pm Central Time (Contact Susan for details)

Approved call summaries: February 5, 2019

Process Committee Call Summaries Archive

Working Documents

Current Work

  • Draft standing agenda item for Coordinating Cmtes to remind/confirm BC committees keeping their BC Wiki page up to date (especially links to current draft documents)
  • Advise - put "recent" working draft of profiles at least quarterly on the BC Cmte Page. (i.e. don't leave them on TF pages)
  • Public Comment resolution
  • Set up Wiki page for holding all the comment resolution records from past public comment.
  • Work with RSNA on a boilerplate email to send to commenters (who gave their email on the comment form) pointing to the resolutions document for their submitted comments.
  • Sketch Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
  • Outline who proposes, who considers, who decides
  • TF should be inherently ephemeral - BC should wrap them up quickly when the work is done. - TODO Brief Guidance
  • BC is more persistent - Dormant - BCs use resources and need to free those up at some point.
  • Maybe remind CC that all BCs should be reviewed -
  • (One BC might be waiting for groundwork and could go dormant and free up time for others in the same CC - but there is a risk that the new one is still working when groundwork comes back.)
  • Limited meeting slots - BC frequency is an issue/question too. - directly linear load on RSNA - staff and minutes
  • CC might be the one to coordinate allocation of the resources approved by Steering?
  • Allows some flexibility -
  • Note - there is an overhead factor (pre-call prep, followup, about 2x overhead per hour of call)
  • Also limited slots for calls when Europe is involved (late morning East Coast)
  • Define Dormant vs Dissolve - opens space for new Cmte (RSNA Resources), migrate active members to new work
  • BC might go dormant because no things in the near future they expect to add - give time for adoption, free up resources for others
  • Keep Dissolve on the process books but likely won't use it often? - Dissolve can give a restart scenario a clean sheet
  • Mostly would go Dormant - profiles might need review/update cycle with advances/new data after a couple of years
  • Might have Dissolve only happen after Dormant. - Dissolve - no intention to revisit (but never say never, could recreate, but more work - fresh start)
  • Use new proposal to Coord Cmte as trigger to review current BC (conservation of resources)
  • Biomarker committees collect SMEs specific to the Biomarker in question
  • Don't need to set criteria, just triggers and process


  • Profile Selection Process
  • Review/update review criteria/evaluations for new BC proposals
  • Harvest from past experiences
  • Given limited resources, how to approve not just good proposals but the best proposals
  • Evidence that the biomarker already has vendor and user adoption and just needs the "last mile" improvements that QIBA brings.
  • Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
  • timing overlap of prior cycle completion period with new proposal period needs to be one considered.
  • possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal
  • On-boarding process
  • Help new committees, new members, new authors, new reviewers, new adopters
  • "How to develop a Profile" Guidance
  • Add some "How to Read a Profile" guidance (also useful for reviewers), e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
  • Joe's intro email, current template and stage docs, QIBA Fact Sheet
  • Find a way to link directly into the "current" version of documents without having to update the links. Now we link to the Profile template page then you have to click a second time put pull up the template.
  • Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
  • Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.


  • Conformance Process
  • What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
  • Need a common view of what conformance entails, how it is achieved, what it looks like to a product, how it is documented, etc.
  • Then we need to ensure that the QIBA Templates/Process supports those things clearly.
  • Kevin will put some initial thoughts on a Wiki page and we can review/add
  • How often should conformance assessment be done, who will assess, who will keep records?
  • Should QIBA get into hosting the Conformance Statements on our site (and then we could link to them from the Profiles page which would be very handy)
  • Consider the RSNA Image Share Conformance Assessment Program and whether we should do something similar for QIBA
  • Process committee can think through the details of a process and confer with RSNA whether they would be interested
  • Clinical Confirmation Process
  • Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"
  • QIBA Tooling
  • How to use Google Docs?
  • Issues with references, access restricted by federal agencies and some companies
  • Round trip fidelity between Word-Google-Word
  • QIBA Wiki
  • QIBA Wiki Improvements
  • Create page for comment resolutions of past Public Comment Profiles
  • Consider how to handle handoffs between RSNA website
  • RSNA website is controlled by the RSNA Marketing Department, must meet certain style requirements, and require resources beyond QIBA staff
  • QIBA would still like to find a way for the more formal publications (e.g. later stage profiles) to be accessible from the RSNA website.
  • Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).
  • Maybe a new TF to support Adoption (can include some Process people)
  • Ed, Brad, Joe, Kevin, Dan, RSNA Webperson
  • Start by looking at what is on the RSNA QIBA pages,
  • Literature Search
  • Process/best practices for review of literature around a new biomarker/profile (search is relatively straight forward)
  • Assemble draft guidance to kick this off (Hendrik Laue, Ona Wu, Caroline? Chung)
  • Where do you store references, what kind of search works best, what exactly are you looking for?
  • Also meta-analysis guidance to determine sources and levels of variance
  • PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible
  • Profile Maintenance Process
  • Propose change tracking process
  • Propose conventions for Profile editions/versions
  • Wiki vs RSNA.org
  • Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)
  • Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.
  • Guidelines for Reference Implementations and commercial products in Profiles
  • (underlying algorithm available, list all comers, non-commercial?, etc)
  • TODO add detail
  • Public Comment attribution
  • don't push for anonymity. Encourage people to identify themselves (deserve credit and we want to be able to followup)
  • form can accept anonymous, but encourage constructive criticism that will be taken in the spirit given.


Todo items that are on hold pending some trigger:

  • Claim Guidance
  • Hold pending people with questions/clarification needs on current text
  • Profile Template
  • Hold pending new suggestions/clarification needs
  • Harmonize assessment procedures across profiles
  • Profile Retirement Process
  • Profile Disclaimer
  • Hold pending revist of Discriminatory Claims
  • For now QIBA addresses technical performance. Getting into the clinical performance associated with Discriminatory Claims would open a lot of issues.
  • Statistical Assumption Testing Guidance
  • Conformance with the statistical assumptions is required at the various QIBA profile stages.
  • Need guidance on the assumptions in the profile, validating the assumptions and testing actors for meeting those assumptions.
  • Symmetrical vs Assymetrical change intervals?
  • Questions about the numbers in "Academic Radiology: ?? Paper"

See Also

This page is based on the Committee Template. Add any good ideas you have to the template.