Public Comment Process
Public Comment is intended to allow both regular participants in QIBA Activities and also those who don't have the time to dedicate to regular participation a chance to contribute their thoughts on proposed specifications before they are formally released. It's also an excellent chance to engage relevant stakeholder individuals and organizations. Comments are accepted from all.
NOTE: If you receive few or no public comments, that's a bad sign. Presuming we didn't hit perfection on the first try, it means we didn't convince people it was important enough to read in any great detail. That bodes poorly for our ability to convince people it is important enough to read and conform to. Good outreach is a significant part of Public Comment. It's our first stage at marketing the Profile.
- Biomarker Committee follows the Review Process to review and approve the draft for Public Comment publication
- Editor tidies up PC draft (e.g. "accept all changes" in Word) and submits to RSNA Staff
- RSNA Staff publishes PC draft
- The Profile has now reached the Public Comment Stage. Work toward Consensus begins.
Period: 60 - 90 days (may be extended at the discretion of the committee)
- RSNA Staff sends Public Comment announcement to mailing lists
- Commenters submit comments to RSNA Staff (via web form or emailed comment form)
- commenters include committee members, other QIBA people, external bodies
- this is the key point for asynchronous engagement
- please use the comment forms; it saves the Editor having to transcribe your comments
- When the public comment period expires
- RSNA Staff/editor collates all comments into a resolution spreadsheet (see Google Sheet template or Word table
- sort by line number and priority (Low/Med/High)
- re-title the leftmost column from "Leave Blank" to "#"
- fill the leftmost column with incrementing numbers. Now you can refer to "Comment #43", or re-sort the table if you like
- re-title the rightmost column from "Leave Blank" to "Resolution"
- Editor resolves comments with priority Low
- turn on Change Tracking in Word
- for most Low comments (typos), fix it in the Profile document and enter "Done" in the Resolution column of the resolution table
- if any Low comments prove problematic, elevate to Med
- Biomarker Committee reviews Med & High comments
- step through the resolution table
- each comment may be:
- accepted = proposed text accepted as is
- rejected = committee does not agree with issue (document reason, e.g. out of scope, )
- resolved = issue accepted but resolved differently than proposed
- consider getting a 3-person sub-committee to take a first pass at this and focus the full committee on items needing discussion
- discussion attempts to achieve consensus on the resolutions whenever possible
- eventually, if consensus is not forthcoming, the minority opinion may accede or put the issue to a vote.
- The Google template includes a status column with codewords (Done, TODO, etc) to track progress resolving comments.
- Editor records decision and explanation (as needed) in the Resolution column of the resolution table/spreadsheet
- When a comment is not accepted, it is particularly important to include a good description of the rationale in the Resolution column as part of meeting our obligation to transparency and avoiding conflicts of interest.
- Consider having assigned reviewers that cross-check the resolution table against the updated Profile document to confirm everything go in correctly
- Likely split this task up, giving a chunk of resolutions to each reviewer.
- RSNA Staff posts resolution spreadsheet and change-tracked Profile Document on the Wiki
- This helps people review the resolution of their comments and the changes that resulted from the Public Comment process
- Biomarker Committee reviews and approves as Consensus
- Modality Coordination Committee reviews and approves as Consensus
- Editor tidies up Consensus draft (e.g. "accept all changes" in Word) and submits to RSNA Staff
- RSNA Staff publishes Consensus draft
- The Profile has now reached the Consensus Stage.