Difference between revisions of "Process Coordinating Committee"

From QIBA Wiki
Jump to navigation Jump to search
Line 41: Line 41:
  
 
==Priority==
 
==Priority==
 +
:* Public Comment resolution
 +
::* Set up Wiki page for holding all the comment resolution records from past public comment.
 +
::* Work with RSNA on a boilerplate email to send to commenters (who gave their email on the comment form) pointing to the resolutions document for their submitted comments.
 +
 
:* Sketch Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
 
:* Sketch Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
 
::* Outline who proposes, who considers, who decides
 
::* Outline who proposes, who considers, who decides
Line 48: Line 52:
 
::* Don't need to set criteria, just triggers and process
 
::* Don't need to set criteria, just triggers and process
  
 +
:* Profile Selection Process
 +
::* Consider additional criteria/evaluation based on experiences
 
:* Review/update new BC proposal review criteria
 
:* Review/update new BC proposal review criteria
 
::* Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
 
::* Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
 
:::* timing overlap of prior cycle completion period with new proposal period needs to be one considered.
 
:::* timing overlap of prior cycle completion period with new proposal period needs to be one considered.
 
:::* possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal
 
:::* possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal
 +
 +
:* On-boarding process
 +
::* Help new committees, new members, new authors, new reviewers, new adopters
 +
::* "How to develop a Profile" Guidance
 +
::* Add some "readers guide" text for reviewers, e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
 +
::* Joe's intro email, current template and stage docs, QIBA Fact Sheet
 +
::* Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
 +
:::* Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.
  
 
==ToDo==
 
==ToDo==
−
 
  
 
:* Conformance Process
 
:* Conformance Process
Line 105: Line 118:
 
::* PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible
 
::* PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible
  
−
:* Profile Selection Process
 
−
::* Consider additional criteria/evaluation based on experiences
 
  
 
:* Profile Maintenance Process
 
:* Profile Maintenance Process
Line 112: Line 123:
 
::* Propose conventions for Profile editions/versions
 
::* Propose conventions for Profile editions/versions
  
−
:* Profile Retirement Process
 
−
::* When do we "withdraw" or at least stop maintaining/promoting a Profile
 
−
::* Review/borrow from corresponding IHE Process - http://wiki.ihe.net/index.php/Evaluation_of_Published_Profiles
 
−
::* Review DICOM retirement concept (stays available in archives, not maintained/promoted)
 
  
−
:* Publication Process
+
:* Wiki vs RSNA.org
−
::* Sketch up publication process for Official Docs (CC approval vote, send to RSNA Staff)
 
−
::* Write up Standard Approval Practices for committee documents
 
−
:::* Quality check mechanism, voting mechanism
 
−
::* Start simple with this and add complexity only as needed.
 
 
::* Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)
 
::* Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)
 
:::* Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.
 
:::* Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.
  
−
:* On-boarding process
+
:* Guidelines for Reference Implementations and commercial products in Profiles
−
::* Help new committees, new members, new authors, new reviewers, new adopters
+
::* (underlying algorithm available, list all comers, non-commercial?, etc)
−
::* Add some "readers guide" text for reviewers, e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
+
::* TODO add detail
−
::* "How to develop a Profile" Guidance
+
 
−
::* Joe's intro email, current template and stage docs, QIBA Fact Sheet
+
:* Public Comment attribution
−
::* Could use a "fresh eyes"/newcomer commentary
+
::* don't push for anonymity. Encourage people to identify themselves (deserve credit and we want to be able to followup)
−
:::* Dan will send the current Fact Sheet to UCSF for feedback on contents
+
::* form can accept anonymous, but encourage constructive criticism that will be taken in the spirit given.
−
:::* Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.
 
  
−
:* Guidelines for Reference Implementations and commercial products in Profiles (underlying algorithm available, list all comers, non-commercial?, etc).
+
==Hold==
−
:* Guidelines for assessment procedures to vet SW in Profiles
+
Todo items that are on hold pending some trigger:
−
:* Clarify QIBA Profile vs UPICT Protocol (Venn Diagram)
 
  
 
:* Claim Guidance
 
:* Claim Guidance
−
::* Next revision cycle (2017Q3?)
+
::* Hold pending people with questions/clarification needs on current text
−
:::* Try to include a greater variety of modalities/targets.
+
 
−
:::* See if we have come across a good example for TDI based CI (e.g. Scenarios BDFMNO)
+
:* Profile Retirement Process
−
:::* PDFF will bring in a new style with a bias and slope that isn't 1.
+
::* Hold pending a profile we want to retire
−
:::* MRE also different. Getting absolute measure not relative? Longitudinal claim. Assuming linearity based on some data, but not assessing yet.
+
::* When do we "withdraw" or at least stop maintaining/promoting a Profile
 +
::* Review/borrow from corresponding IHE Process - http://wiki.ihe.net/index.php/Evaluation_of_Published_Profiles
 +
::* Review DICOM retirement concept (stays available in archives, not maintained/promoted)
  
 
:* Profile Disclaimer
 
:* Profile Disclaimer
−
::* Revist if Discriminatory Claims are addressed.
+
::* Hold pending revist of Discriminatory Claims
−
::* Consult RSNA about what disclaimer should address
+
::* For now QIBA addresses technical performance. Getting into the clinical performance associated with Discriminatory Claims would open a lot of issues.
−
::* Draft standardized disclaimer text for profiles
 
−
::* If QIBA did Discriminatory Claims, then there would be a stronger need for disclaimer, but lacking clinical outcome statments, maybe we don't have as strong a need?  We stick to technical performance.
 
  
−
:* Public Comment attribution - don't push for anonymity. Encourage people to identify themselves (deserve credit and we want to be able to followup)
 
−
::* form can accept anonymous, but encourage constructive criticism that will be taken in the spirit given.
 
−
::* TODO - set up Wiki page for holding all the comment resolution records from past public comment.
 
−
::* TODO - work with RSNA on a boilerplate email to send to commenters (who gave their email on the comment form) pointing to the resolutions document for their submitted comments.
 
  
 
==See Also==
 
==See Also==

Revision as of 00:11, 8 February 2019

Chairs: Kevin O'Donnell (Chair); Dan Sullivan, MD (Vice-chair)
Secretary: Susan Stanfa (RSNA Staff)


Scope

The Process Committee facilitates the QIBA Coordinating Committees and their profiling activities by defining processes and tools to promote consistent quality work product.

  • Develop common Templates & Processes
  • Support & mentor adoption of above
  • Drive consistent content & format of Profiles
  • Support infrastructure for Committees

Meetings/Call Summaries

  • Upcoming: Biweekly Tcons, 1st and 3rd Wednesdays of each month @ 3-4pm Central Time (Contact Susan for details)


Approved call summaries:

Process Committee Call Summaries Archive

Working Documents

Current Work

  • Draft standing agenda item for Coordinating Cmtes to remind/confirm BC committees keeping their BC Wiki page up to date (especially links to current draft documents)
  • Advise - put "recent" working draft of profiles at least quarterly on the BC Cmte Page. (i.e. don't leave them on TF pages)

Priority

  • Public Comment resolution
  • Set up Wiki page for holding all the comment resolution records from past public comment.
  • Work with RSNA on a boilerplate email to send to commenters (who gave their email on the comment form) pointing to the resolutions document for their submitted comments.
  • Sketch Sunset procedure for retiring Task Forces and Biomarker Committees that have completed their work.
  • Outline who proposes, who considers, who decides
  • Define Dormant vs Dissolve - opens space for new Cmte (RSNA Resources), migrate active members to new work
  • Use new proposal to Coord Cmte as trigger to review current BC (conservation of resources)
  • Biomarker committees collect SMEs specific to the Biomarker in question
  • Don't need to set criteria, just triggers and process
  • Profile Selection Process
  • Consider additional criteria/evaluation based on experiences
  • Review/update new BC proposal review criteria
  • Consider Prior Project completion when Steering Cmte reviews new Proposals from the same PI.
  • timing overlap of prior cycle completion period with new proposal period needs to be one considered.
  • possible solution would require final reports for prior COMPLETED funding cycles before considering a new project proposal
  • On-boarding process
  • Help new committees, new members, new authors, new reviewers, new adopters
  • "How to develop a Profile" Guidance
  • Add some "readers guide" text for reviewers, e.g. focus on Spec tables. Look at your relevant Activities/Procedures. Read the Discussion sections for more background.
  • Joe's intro email, current template and stage docs, QIBA Fact Sheet
  • Could use a "fresh eyes"/newcomer commentary - find a newcomer guinea pig
  • Kevin will work with Joe to get it into a Google Doc for revision and linking from the Wiki.

ToDo

  • Conformance Process
  • What should go into a Conformance Record? - (e.g. findings from Technical Confirmation tests)
  • Need a common view of what conformance entails, how it is achieved, what it looks like to a product, how it is documented, etc.
  • Then we need to ensure that the QIBA Templates/Process supports those things clearly.
  • Kevin will put some initial thoughts on a Wiki page and we can review/add
  • How often should conformance assessment be done, who will assess, who will keep records?
  • Should QIBA get into hosting the Conformance Statements on our site (and then we could link to them from the Profiles page which would be very handy)
  • Consider the RSNA Image Share Conformance Assessment Program and whether we should do something similar for QIBA
  • Process committee can think through the details of a process and confer with RSNA whether they would be interested
  • Profile Template
  • Adjust Profile Template (change tracked) to regroup time-muddled Activities
  • Conformance Section - connect the dots
  • Embed the checklist as an appendix
  • Support adoption of the updated Profile Template.
  • Harmonize assessment procedures across profiles
  • Statistical Assumption Testing Guidance
  • Conformance with the statistical assumptions is required at the various QIBA profile stages.
  • Need guidance on the assumptions in the profile, validating the assumptions and testing actors for meeting those assumptions.
  • Symmetrical vs Assymetrical change intervals?
  • Questions about the numbers in "Academic Radiology: ?? Paper"
  • Clinical Confirmation Process
  • Methodology/Study Design guidance/requirements for achieving "Clinically Confirmed"
  • QIBA Tooling
  • How to use Google Docs?
  • Issues with references, access restricted by federal agencies and some companies
  • Round trip fidelity between Word-Google-Word
  • QIBA Wiki
  • QIBA Wiki Improvements
  • Create page for comment resolutions of past Public Comment Profiles
  • Consider how to handle handoffs between RSNA website
  • Wiki serves QIBA Committee work. Consider a new user oriented site (clinicians, administrators).
  • Maybe a new TF to support Adoption (can include some Process people)
  • Ed, Brad, Joe, Kevin, Dan, RSNA Webperson
  • Start by looking at what is on the RSNA QIBA pages,
  • Literature Search
  • Ask for a status update?
  • Process/best practices for initial literature search around a new biomarker/profile
  • Does someone have a draft set of guidance to kick this off? (Nancy Obuchowski, Ona Wu, Hendrik Laue, Andy Milkowski)
  • Where do you store references, what kind of search works best, what exactly are you looking for?
  • Also meta-analysis guidance to determine sources and levels of variance
  • PDF/DWI - using Google Sheet to track/tabulate/organize - inherently input from many sources so nice to have centralized, commonly edited, accessible


  • Profile Maintenance Process
  • Propose change tracking process
  • Propose conventions for Profile editions/versions


  • Wiki vs RSNA.org
  • Address relationship between wiki content (current, easy to update) and website content (authoritative, stable)
  • Likely RSNA staff will periodically mirror wiki content up onto the website; address this in publication process.
  • Guidelines for Reference Implementations and commercial products in Profiles
  • (underlying algorithm available, list all comers, non-commercial?, etc)
  • TODO add detail
  • Public Comment attribution
  • don't push for anonymity. Encourage people to identify themselves (deserve credit and we want to be able to followup)
  • form can accept anonymous, but encourage constructive criticism that will be taken in the spirit given.

Hold

Todo items that are on hold pending some trigger:

  • Claim Guidance
  • Hold pending people with questions/clarification needs on current text
  • Profile Retirement Process
  • Profile Disclaimer
  • Hold pending revist of Discriminatory Claims
  • For now QIBA addresses technical performance. Getting into the clinical performance associated with Discriminatory Claims would open a lot of issues.


See Also

This page is based on the Committee Template. Add any good ideas you have to the template.

Retrieved from "https://qibawiki.rsna.org/index.php?title=Process_Coordinating_Committee&oldid=13627"