Difference between revisions of "Public Comment Process"

From QIBA Wiki
Jump to: navigation, search
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
Public Comment is intended to allow both regular participants in QIBA Activities and also those who don't have the time to dedicate to regular participation a chance to contribute their thoughts on proposed specifications before they are formally released.  It's also an excellent chance to engage relevant stakeholder individuals and organizations. Comments are accepted from all.
 
Public Comment is intended to allow both regular participants in QIBA Activities and also those who don't have the time to dedicate to regular participation a chance to contribute their thoughts on proposed specifications before they are formally released.  It's also an excellent chance to engage relevant stakeholder individuals and organizations. Comments are accepted from all.
  
NOTE: ''If you receive few or no public comments, that's a'' '''bad sign.'''  Presuming we didn't hit perfection on the first try, it means we didn't convince people it was important enough to read in any great detail.  That bodes poorly for our ability to convince people it is important enough to read and conform to.  Good outreach is a significant part of Public Comment.  It's our first stage at marketing the Profile.
+
NOTE: ''If you receive few or no public comments, that's a'' '''bad sign.'''  Presuming we didn't hit perfection on the first try, it means we didn't convince people it was important enough to read in any great detail.  That bodes poorly for our ability to convince people it is important enough to read and conform to.  Good [[Stakeholder Contacts|outreach]] is a significant part of Public Comment.  It's our first stage at marketing the Profile.
  
  
Period: 30 days (may be longer at the discretion of the committee)
+
'''Preparation''':
  
* [[Review Process|Approve for public comment]] ('''Biomarker & Modality Committees''')
+
* ''Biomarker Committee'' follows the '''[[Review Process]]''' to review and approve the draft for Public Comment publication
* Tidy up PC draft (e.g. "accept all changes" in Word)
+
* ''Editor'' tidies up PC draft (e.g. "accept all changes" in Word) and submits to RSNA Staff
* [[Publication Process|Post PC draft of Document on the Wiki]] ('''editor/secretariat''')
+
* ''RSNA Staff'' '''[[Publication Process|publishes PC draft]]'''
* The Profile has now reached the [[QIBA Profile Stages|Public Comment Stage]]. Start working on [[QIBA Profile Stages|Consensus]].
+
* The Profile has now reached the [[QIBA Profile Stages|Public Comment Stage]]. Work toward [[QIBA Profile Stages|Consensus]] begins.
  
  
* Send announcement to mailing lists ('''secretariat''')
+
Period: 60 - 90 days (may be extended at the discretion of the committee)
** detail location of document to be reviewed, deadline for comment submission, method of submission
+
 
** provide a link to the [http://tinyurl.com/QIBA-Public-Comment-Form QIBA Public Comment Form]
+
* ''RSNA Staff'' sends Public Comment announcement to mailing lists
** reach out to any organizations and individuals that might have and interest or expertise to contribute
+
** explain the location of document to be reviewed, deadline for comment submission, method of submission
* Email comments to secretariat ('''Commenters''')
+
** provide a link to the '''[http://tinyurl.com/QIBA-Public-Comment-Form QIBA Public Comment Form]'''
 +
** reach out to any '''[[Stakeholder Contacts|organizations and individuals]]''' that might have an interest or expertise to contribute
 +
* ''Commenters'' submit comments to RSNA Staff (via web form or emailed comment form)
 
** commenters include committee members, other QIBA people, external bodies
 
** commenters include committee members, other QIBA people, external bodies
 
** this is the key point for asynchronous engagement
 
** this is the key point for asynchronous engagement
** please use the provided comment form, it saves the editor having to transcribe your comments
+
** please use the comment forms; it saves the Editor having to transcribe your comments
 +
 
 +
* When the public comment period expires
 +
** ''RSNA Staff''
 +
*** updates the bullet for this profile on the [[Profiles]] page to note when comments closed
 +
*** adds a row on the [[Comment Resolutions]] page for the profile (link to the PC doc in the first cell)
 +
 
  
 +
'''Resolution''':
  
* Collate all comments into a resolution spreadsheet (see [https://docs.google.com/spreadsheets/d/1o3_ssHHIABGKNesUJ_-jL5RYqSx9Eq4o6C2dlIsHQo4/edit?usp=sharing Google template] or '''[[:Media:QIBA_Public_Comment_Template-2011.01.doc | Word table]]''' ('''secretariat/editor''')
+
* ''RSNA Staff/editor'' collates all comments into a '''resolution spreadsheet''' (see '''[https://docs.google.com/spreadsheets/d/1o3_ssHHIABGKNesUJ_-jL5RYqSx9Eq4o6C2dlIsHQo4/edit?usp=sharing Google Sheet template]''' or '''[[:Media:QIBA_Public_Comment_Template-2011.01.doc | Word table]]'''
 
** sort by line number and priority (Low/Med/High)
 
** sort by line number and priority (Low/Med/High)
 
** re-title the leftmost column from "Leave Blank" to "#"
 
** re-title the leftmost column from "Leave Blank" to "#"
 
** fill the leftmost column with incrementing numbers.  Now you can refer to "Comment #43", or re-sort the table if you like
 
** fill the leftmost column with incrementing numbers.  Now you can refer to "Comment #43", or re-sort the table if you like
 
** re-title the rightmost column from "Leave Blank" to "Resolution"
 
** re-title the rightmost column from "Leave Blank" to "Resolution"
* Resolve comments with priority Low ('''editor''')
+
* ''Editor'' resolves comments with priority Low
 
** turn on Change Tracking in Word
 
** turn on Change Tracking in Word
 
** for most Low comments (typos), fix it in the Profile document and enter "Done" in the Resolution column of the resolution table
 
** for most Low comments (typos), fix it in the Profile document and enter "Done" in the Resolution column of the resolution table
 
** if any Low comments prove problematic, elevate to Med
 
** if any Low comments prove problematic, elevate to Med
* Review Med & High comments ('''Biomarker Committee''')
+
* ''Biomarker Committee'' reviews Med & High comments
 
** step through the resolution table
 
** step through the resolution table
 
** each comment may be:
 
** each comment may be:
Line 37: Line 46:
 
*** rejected = committee does not agree with issue (document reason, e.g. out of scope, )
 
*** rejected = committee does not agree with issue (document reason, e.g. out of scope, )
 
*** resolved = issue accepted but resolved differently than proposed
 
*** resolved = issue accepted but resolved differently than proposed
** you may want a 3-person sub-committee to take a first pass at this and focus the full committee on items needing discussion
+
** consider getting a 3-person sub-committee to take a '''first pass''' at this and focus the full committee on items needing discussion
* Record decision and explanation (as needed) in the Resolution column of the resolution table/spreadsheet ('''editor''')
+
*** discussion attempts to achieve [[Committee_Procedures#Consensus_Process| consensus]] on the resolutions whenever possible
* The Google template includes a status column with codewords to track completion progress.
+
*** eventually, if consensus is not forthcoming, the minority opinion may accede or put the issue to a vote.
 +
** The Google template includes a status column with codewords (Done, TODO, etc) to track progress resolving comments.
 +
* ''Editor'' records decision and explanation (as needed) in the Resolution column of the resolution table/spreadsheet
 +
** When a comment is not accepted, it is particularly important to include a good description of the rationale in the Resolution column as part of meeting our obligation to transparency and avoiding conflicts of interest.
  
  
* Post resolution spreadsheet and change-tracked Profile Document on the Wiki ('''editor/secretariat''')
+
* ''RSNA Staff'' posts resolution spreadsheet and change-tracked Profile Document on the Wiki
 
** This helps people review the resolution of their comments and the changes that resulted from the Public Comment process
 
** This helps people review the resolution of their comments and the changes that resulted from the Public Comment process
* [[Review Process|Approve as Consensus]] ('''Biomarker & Modality Committees''')
+
* ''Biomarker Committee'' [[Review Process|reviews and approves as Consensus]]
* Tidy up Consensus draft (e.g. "accept all changes" in Word)
+
* ''Modality Coordination Committee'' [[Review Process|reviews and approves as Consensus]]
* [[Publication Process|Post Consensus Document on the Wiki]] ('''editor/secretariat''')
+
* ''Editor'' tidies up Consensus draft (e.g. "accept all changes" in Word) and submits to RSNA Staff
 +
* ''RSNA Staff'' '''[[Publication Process|publishes Consensus draft]]'''
 
* The Profile has now reached the [[QIBA Profile Stages|Consensus Stage]].
 
* The Profile has now reached the [[QIBA Profile Stages|Consensus Stage]].

Latest revision as of 18:59, 27 August 2020

Public Comment is intended to allow both regular participants in QIBA Activities and also those who don't have the time to dedicate to regular participation a chance to contribute their thoughts on proposed specifications before they are formally released. It's also an excellent chance to engage relevant stakeholder individuals and organizations. Comments are accepted from all.

NOTE: If you receive few or no public comments, that's a bad sign. Presuming we didn't hit perfection on the first try, it means we didn't convince people it was important enough to read in any great detail. That bodes poorly for our ability to convince people it is important enough to read and conform to. Good outreach is a significant part of Public Comment. It's our first stage at marketing the Profile.


Preparation:

  • Biomarker Committee follows the Review Process to review and approve the draft for Public Comment publication
  • Editor tidies up PC draft (e.g. "accept all changes" in Word) and submits to RSNA Staff
  • RSNA Staff publishes PC draft
  • The Profile has now reached the Public Comment Stage. Work toward Consensus begins.


Period: 60 - 90 days (may be extended at the discretion of the committee)

  • RSNA Staff sends Public Comment announcement to mailing lists
  • Commenters submit comments to RSNA Staff (via web form or emailed comment form)
    • commenters include committee members, other QIBA people, external bodies
    • this is the key point for asynchronous engagement
    • please use the comment forms; it saves the Editor having to transcribe your comments
  • When the public comment period expires
    • RSNA Staff
      • updates the bullet for this profile on the Profiles page to note when comments closed
      • adds a row on the Comment Resolutions page for the profile (link to the PC doc in the first cell)


Resolution:

  • RSNA Staff/editor collates all comments into a resolution spreadsheet (see Google Sheet template or Word table
    • sort by line number and priority (Low/Med/High)
    • re-title the leftmost column from "Leave Blank" to "#"
    • fill the leftmost column with incrementing numbers. Now you can refer to "Comment #43", or re-sort the table if you like
    • re-title the rightmost column from "Leave Blank" to "Resolution"
  • Editor resolves comments with priority Low
    • turn on Change Tracking in Word
    • for most Low comments (typos), fix it in the Profile document and enter "Done" in the Resolution column of the resolution table
    • if any Low comments prove problematic, elevate to Med
  • Biomarker Committee reviews Med & High comments
    • step through the resolution table
    • each comment may be:
      • accepted = proposed text accepted as is
      • rejected = committee does not agree with issue (document reason, e.g. out of scope, )
      • resolved = issue accepted but resolved differently than proposed
    • consider getting a 3-person sub-committee to take a first pass at this and focus the full committee on items needing discussion
      • discussion attempts to achieve consensus on the resolutions whenever possible
      • eventually, if consensus is not forthcoming, the minority opinion may accede or put the issue to a vote.
    • The Google template includes a status column with codewords (Done, TODO, etc) to track progress resolving comments.
  • Editor records decision and explanation (as needed) in the Resolution column of the resolution table/spreadsheet
    • When a comment is not accepted, it is particularly important to include a good description of the rationale in the Resolution column as part of meeting our obligation to transparency and avoiding conflicts of interest.