Segmentation and Markup Formats
Non-standard proprietary formats (from research or commercial software) may be tolerable for current Groundwork activities. The applications may already be installed and the researchers familiar with their use, allowing groundwork to proceed immediately. Such formats often contain no "context" or "header" information about which subject or exam date to which they apply or any information about when they were created, by whom and for what purpose, and are limited only to the "payload" of coordinates or contours or segmentation. They may or may not contain robust image references, or be dependent about assumptions about the image data they are derived from or intended to be applied to. They may or may not contain meta-data (categorical or quantitative) derived from the images. Managing the files manually may be tractable at small the scale of groundwork studies (as long as the files are grouped with the image data that they relate to, contain in a hierarchical folder organization or organized with some naming convention). Preferably the proprietary file format will be documented rather then dependent on a particular software program and version to be read.
Standard formats that are in widespread commercial use may be preferable for larger scale experiments and in the Profiles themselves. Standards formats like DICOM address a number of issues that will be faced when we try to roll this out in a clinical environment: where do the segmentations get stored, how do we move them over the network, how do we exchange them on portable media, how do we associate them with the patient record, how do we find/retrieve them later for comparison, how do we identify the images from which they were derived, how do we identify the space in which they exist for later registration, how do we de-identify them for research purposes, etc.
Research Segmentation Formats
<<Include details of 3D Doctor and PLY formats here>>
DICOM Segmentation, Region of Interest and Other Annotation Formats
DICOM provides several ways to encode segmentations, regions of interest and other types of annotation.
Two groups of DICOM formats are defined, those that are preferred for QIBA use, and those that may be encountered and are not ideal but are likely better than any proprietary non-standard format.
All of the DICOM objects share with ordinary DICOM images a "header" containing the information about the patient (subject) and other management information that provides the "context" for use of the content, including globally unique identifiers. I.e., they are self-contained and not dependent on any file format, file organization or file naming convention.
NOTE: The DICOM Standard is referenced below. Individual supplement documents that may originally have defined a specific feature are not maintained after they have been incorporated into the DICOM Standard, and hence are unreliable references. The DICOM Standard may contain fixes and updates not in the original supplement.
Recommended DICOM Formats for QIBA Experiments
Structured Reports
The DICOM Structured Report (SR) mechanism allows encoding of a tree (or directed acyclic graph) of structured, coded and numerical information, which may include references to images, frames within a multi-frame image, coordinates in those images, and patient-relative 3D coordinates independent of images.
The structured content and codes used that are specific to a particular application, experiment or trial may be defined in a "template"; the re-use of such templates, or common sub-patterns and code sets within templates is encouraged. A library of templates is defined in DICOM PS 3.16.
Contours encoded in Structured Reports
Structured reports may contain content items (nodes in the tree) that are graphic outlines defined as closed polylines; these may be image (or frame) relative (2D) or patient relative (3D). They are required to be co-planar in either case. The tree structure of the SR may be used to group a set of such closed polylines to represent a single object (such as an entire lesion segmentation).
Other nodes in the SR tree may be used to describe properties of the contours or objects, including identifiers (human readable or unique), technique (such as how the segmentation was performed), and derived information (including quantitative information like volume, mean density, etc.).
Standard templates and codes are defined in DICOM PS 3.16 for many of these measurements, and have been specified for clinical use in such applications as obstetric ultrasound, vascular and cardiac ultrasound (echocardiography), quantitative cardiac CT and MRI, and various Computed Assisted Detection (CAD) applications (including mammography, chest and virtual colonoscopy CAD).
For further information see DICOM PS 3.3 C.18.6 Spatial Coordinates Macro and C.18.9 3D Spatial Coordinates Macro.
The coordinate and image references in DICOM SR can also include a reference to a DICOM Presentation State, which allows the window center and width and the pan/zoom state used when the segmentation was performed to be recorded and replayed.
Segmentations referenced from Structured Reports
Rather than encoding segmentations in-line as graphic contours, SRs may reference external objects that define the segmentation, yet still convey the context and the meta-data about such segmentations. For example, an SR object may define a set of lesions, and the volume and mean density of each, but reference separate objects that define the graphical information either as rasterized or surface mesh objects, as defined below.
Rasterized Segmentations
Those voxels (on a 3D image set) or pixels (on a 2D image set) representing a particular class can be encoded as a separate rasterized image, referred to as a "segmentation object".
A degenerate case is a bit-mask corresponding to the original image voxels or pixels, in which there is a 1:1 correspondence between image voxels and segmentation object voxels and each segmentation object voxel is zero or one. This is functionally equivalent to an "overlay", but has specifically defined semantics.
Each segmentation object is a multi-frame image representing a classification of pixels in one or more referenced images. Segmentations are either binary or fractional. The segmentation is not required to have the same spatial sampling or extent as the referenced images, if a common 3D frame of reference is defined.
Pixel/voxel masks in the plane of an image define images pixels that are part of a segmentation. Masks may be drawn on multiple images in a set to segment a volume. The mask values may be binary or fractional. Fraction may be used to represent either the fraction of the voxel which is a part of the segmentation (partial volumes), or the fractional probability that the voxel belongs to the segmentation.
The masks are stored in objects that look very much like DICOM images, except instead of image pixels they contain mask pixels.
Document: Supplement 111 (See Pg 4 for a cursory overview; See Pgs 15-19, 28-31, 34 for the basic details)
Specification: DICOM 2008 (Supp 111 is Final Text)
Surface Segmentations
A Surface Mesh (i.e. polygons/triangles) connecting points in space, which may or may not be in the plane of an image, define the volume which is part of a segmentation.
The surfaces are stored in a DICOM object which can be managed similarly to other DICOM files, but contains lists of points and vectors instead of an array of pixels.
Document: Supplement 132 (See Pgs 4, 38-40 for a cursory overview; See Pgs 9, 15-25 for the basic details)
Specification: (Supp 132 was approved for Final Text in late 2008 - will be incorporated shortly)
Other DICOM Formats
Burned into Pixel Data
An application may burned in an annotation, outline or segmentation to the pixel data of an underlying image and re-save it as a secondary capture image or an encapsulated PDF object; this is suitable for human review only and the information is essentially lost for the purpose of further processing.
Bitmap Overlays
An annotation or contour or segmentation may be rasterized into a 2D bitmap and either encoded in unused high bits of the pixel data in a re-saved image, stored separately in an attribute of a re-saved image (Overlay Data (0x60xx,0x3000)), or stored in a separate Presentation State object.
Curves
A means of describing curves for graphics (and waveforms) was originally defined in the standard but has been removed and should not be used for QIBA purposes. Their function has been replaced by Graphic Annotations in Presentation State objects and time-based Waveform objects.
Graphic Annotations in Presentation States
Presentation States contain a means of encoding simple 2D image-relative vector graphics, and may be be used to encode, for example, isocontours of a region as a set of closed polylines. There is no standard or coded or structured means of describing their semantics, however. This format is not recommended for QIBA purposes in view of the lack of such semantics, but since they are very widely implemented in commercial systems in which the purpose is to capture consistent presentation (rather than meaning), it is possible to use these by defining conventions for the users (creators) to follow (such as using the same text label for isocontours on successive slices to indicate the same lesion).