Process Coordinating Committee
- Chairs: Kevin O'Donnell (Chair); Dan Sullivan, MD (Vice-chair)
- Secretary: Susan Stanfa (RSNA Staff)
Scope
The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.
- Develop common Templates & Processes
- Support & mentor adoption of above
- Drive consistent content & format of Profiles
- Support infrastructure for Committees
Meetings/Call Summaries
- Upcoming: Biweekly Tcons, 1st and 3rd Wednesdays of each month @ 3-4pm Central Time (Contact Susan for details)
Approved call summaries:
Process Committee Call Summaries Archive
Working Documents
- Wiki Use Cases - https://docs.google.com/spreadsheets/d/1bVP9pSjxtCVinlASCAeGDf5vjc2lCfj16qEg7hkft0Q/edit?usp=sharing
Current Work
- Draft standing agenda item for Coordinating Cmtes to remind/confirm BC committees keeping their BC Wiki page up to date (especially links to current draft documents)
- Advise - put "recent" working draft of profiles at least quarterly on the BC Cmte Page. (i.e. don't leave them on TF pages)
- Public Comment resolution
- Set up Wiki page for holding all the comment resolution records from past public comment.
- Work with RSNA on a boilerplate email to send to commenters (who gave their email on the comment form) pointing to the resolutions document for their submitted comments.
- Sketch Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
- Outline who proposes, who considers, who decides
- TF should be inherently ephemeral - BC should wrap them up quickly when the work is done. - TODO Brief Guidance
- BC is more persistent - Dormant - BCs use resources and need to free those up at some point.
- Maybe remind CC that all BCs should be reviewed -
- (One BC might be waiting for groundwork and could go dormant and free up time for others in the same CC - but there is a risk that the new one is still working when groundwork comes back.)
- Limited meeting slots - BC frequency is an issue/question too. - directly linear load on RSNA - staff and minutes
- CC might be the one to coordinate allocation of the resources approved by Steering?
- Allows some flexibility -
- Note - there is an overhead factor (pre-call prep, followup, about 2x overhead per hour of call)
- Also limited slots for calls when Europe is involved (late morning East Coast)
- Define Dormant vs Dissolve - opens space for new Cmte (RSNA Resources), migrate active members to new work
- BC might go dormant because no things in the near future they expect to add - give time for adoption, free up resources for others
- Keep Dissolve on the process books but likely won't use it often? - Dissolve can give a restart scenario a clean sheet
- Mostly would go Dormant - profiles might need review/update cycle with advances/new data after a couple of years
- Might have Dissolve only happen after Dormant. - Dissolve - no intention to revisit (but never say never, could recreate, but more work - fresh start)
- Use new proposal to Coord Cmte as trigger to review current BC (conservation of resources)
- Biomarker committees collect SMEs specific to the Biomarker in question
- Don't need to set criteria, just triggers and process
Priority
- Profile Selection Process
- Review/update review criteria/evaluations for new BC proposals
- Harvest from past experiences
- Given limited resources, how to approve not just good proposals but the best proposals
- Evidence that the biomarker already has vendor and user adoption and just needs the "last mile" improvements that QIBA brings.
- Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
- timing overlap of prior cycle completion period with new proposal period needs to be one considered.
- possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal
- On-boarding process
- Help new committees, new members, new authors, new reviewers, new adopters
- "How to develop a Profile" Guidance
- Add some "How to Read a Profile" guidance (also useful for reviewers), e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
- Joe's intro email, current template and stage docs, QIBA Fact Sheet
- Find a way to link directly into the "current" version of documents without having to update the links. Now we link to the Profile template page then you have to click a second time put pull up the template.
- Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
- Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.
ToDo
- Conformance Process
- What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
- Need a common view of what conformance entails, how it is achieved, what it looks like to a product, how it is documented, etc.
- Then we need to ensure that the QIBA Templates/Process supports those things clearly.
- Kevin will put some initial thoughts on a Wiki page and we can review/add
- How often should conformance assessment be done, who will assess, who will keep records?
- Should QIBA get into hosting the Conformance Statements on our site (and then we could link to them from the Profiles page which would be very handy)
- Consider the RSNA Image Share Conformance Assessment Program and whether we should do something similar for QIBA
- Process committee can think through the details of a process and confer with RSNA whether they would be interested
- Clinical Confirmation Process
- Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"
- QIBA Tooling
- How to use Google Docs?
- Issues with references, access restricted by federal agencies and some companies
- Round trip fidelity between Word-Google-Word
- QIBA Wiki
- QIBA Wiki Improvements
- Create page for comment resolutions of past Public Comment Profiles
- Consider how to handle handoffs between RSNA website
- RSNA website is controlled by the RSNA Marketing Department, must meet certain style requirements, and require resources beyond QIBA staff
- QIBA would still like to find a way for the more formal publications (e.g. later stage profiles) to be accessible from the RSNA website.
- Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).
- Maybe a new TF to support Adoption (can include some Process people)
- Ed, Brad, Joe, Kevin, Dan, RSNA Webperson
- Start by looking at what is on the RSNA QIBA pages,
- Literature Search
- Process/best practices for review of literature around a new biomarker/profile (search is relatively straight forward)
- Assemble draft guidance to kick this off (Hendrik Laue, Ona Wu, Caroline? Chung)
- See existing material - http://qibawiki.rsna.org/index.php/Tools_and_recommendation_for_structured_literature_review
- Where do you store references, what kind of search works best, what exactly are you looking for?
- Also meta-analysis guidance to determine sources and levels of variance
- PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible
- Profile Maintenance Process
- Propose change tracking process
- Propose conventions for Profile editions/versions
- Wiki vs RSNA.org
- Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)
- Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.
- Guidelines for Reference Implementations and commercial products in Profiles
- (underlying algorithm available, list all comers, non-commercial?, etc)
- TODO add detail
- Public Comment attribution
- don't push for anonymity. Encourage people to identify themselves (deserve credit and we want to be able to followup)
- form can accept anonymous, but encourage constructive criticism that will be taken in the spirit given.
Hold
Todo items that are on hold pending some trigger:
- Claim Guidance
- Hold pending people with questions/clarification needs on current text
- Profile Template
- Hold pending new suggestions/clarification needs
- Harmonize assessment procedures across profiles
- Profile Retirement Process
- Hold pending a profile we want to retire
- When do we "withdraw" or at least stop maintaining/promoting a Profile
- Review/borrow from corresponding IHE Process - http://wiki.ihe.net/index.php/Evaluation_of_Published_Profiles
- Review DICOM retirement concept (stays available in archives, not maintained/promoted)
- Profile Disclaimer
- Hold pending revist of Discriminatory Claims
- For now QIBA addresses technical performance. Getting into the clinical performance associated with Discriminatory Claims would open a lot of issues.
- Statistical Assumption Testing Guidance
- Conformance with the statistical assumptions is required at the various QIBA profile stages.
- Need guidance on the assumptions in the profile, validating the assumptions and testing actors for meeting those assumptions.
- Symmetrical vs Assymetrical change intervals?
- Questions about the numbers in "Academic Radiology: ?? Paper"
See Also
This page is based on the Committee Template. Add any good ideas you have to the template.