Segmentation and Markup Formats

Non-standard proprietary formats (from research or commercial software) may be tolerable for current Groundwork activities. The applications may already be installed and the researchers familiar with their use, allowing groundwork to proceed immediately. Such formats often contain no "context" or "header" information about which subject or exam date to which they apply or any information about when they were created, by whom and for what purpose, and are limited only to the "payload" of coordinates or contours or segmentation. They may or may not contain robust image references, or be dependent about assumptions about the image data they are derived from or intended to be applied to. They may or may not contain meta-data (categorical or quantitative) derived from the images. Managing the files manually may be tractable at small the scale of groundwork studies (as long as the files are grouped with the image data that they relate to, contain in a hierarchical folder organization or organized with some naming convention). Preferably the proprietary file format will be documented rather then dependent on a particular software program and version to be read.

Standard formats that are in widespread commercial use may be preferable for larger scale experiments and in the Profiles themselves. Standards formats like DICOM address a number of issues that will be faced when we try to roll this out in a clinical environment: where do the segmentations get stored, how do we move them over the network, how do we exchange them on portable media, how do we associate them with the patient record, how do we find/retrieve them later for comparison, how do we identify the images from which they were derived, how do we identify the space in which they exist for later registration, how do we de-identify them for research purposes, etc.

Research Segmentation Formats

<<Include details of 3D Doctor and PLY formats here>>

DICOM Segmentation, Region of Interest and Other Annotation Formats

DICOM provides several ways to encode segmentations, regions of interest and other types of annotation.

NOTE: The DICOM Standard is referenced below. Individual supplement documents that may originally have defined a specific feature are not maintained after they have been incorporated into the DICOM Standard, and hence are unreliable references. The DICOM Standard may contain fixes and updates not in the original supplement.

Two groups of DICOM formats are defined, those that are preferred for QIBA use, and those that may be encountered and are not ideal but are likely better than any proprietary non-standard format.

All of the DICOM objects share with ordinary DICOM images a "header" containing the information about the patient (subject) and other management information that provides the "context" for use of the content, including globally unique identifiers. I.e., they are self-contained and not dependent on any file format, file organization or file naming convention.

Recommended DICOM Formats for QIBA Experiements

Structured Reports

The DICOM Structured Report (SR) mechanism allows encoding of a tree (or directed acyclic graph) of structured, coded and numerical information, which may include references to images, frames within a multi-frame image, coordinates in those images, and patient-relative 3D coordinates independent of images.

The structured content and codes used that are specific to a particular application, experiment or trial can be defined a priori in a "template"; the re-use of such templates, or common sub-patterns and code sets within templates is encouraged. A library of templates is defined in DICOM PS 3.16.

Contours encoded in Structured Reports

Polyline contours in the plane of an image define a boundary enclosing a segmentation. Contours may be drawn on multiple images in a set to segment a volume.

The contours are embedded into a DICOM Structured Report object as a list of polyline vertices. This supplement was written to address the needs of storing the results of Chest CAD based on X-ray or CT images. The approach could be adapted to other applications and appropriate extensions made to DICOM.

Document: Supplement 65 (See Pg 4-5, 15-18, 22-24 for a cursory overview; See Pgs 30-42 for the basic details and various code tables and code definitions on Pgs 55-105 e.g. RECIST Repsonse Criteria codes on Pg 77)

Specification: DICOM 2008 (Supp 65 is Final Text)

Rasterized Segmentations

Pixel/voxel masks in the plane of an image define images pixels that are part of a segmentation. Masks may be drawn on multiple images in a set to segment a volume. The mask values may be binary or fractional. Fraction may be used to represent either the fraction of the voxel which is a part of the segmentation (partial volumes), or the fractional probability that the voxel belongs to the segmentation.

The masks are stored in objects that look very much like DICOM images, except instead of image pixels they contain mask pixels.

Document: Supplement 111 (See Pg 4 for a cursory overview; See Pgs 15-19, 28-31, 34 for the basic details)

Specification: DICOM 2008 (Supp 111 is Final Text)

Surface Segmentations

A Surface Mesh (i.e. polygons/triangles) connecting points in space, which may or may not be in the plane of an image, define the volume which is part of a segmentation.

The surfaces are stored in a DICOM object which can be managed similarly to other DICOM files, but contains lists of points and vectors instead of an array of pixels.

Document: Supplement 132 (See Pgs 4, 38-40 for a cursory overview; See Pgs 9, 15-25 for the basic details)

Specification: (Supp 132 was approved for Final Text in late 2008 - will be incorporated shortly)

Other DICOM Formats

Burned into Pixel Data

An application may burned in an annotation, outline or segmentation to the pixel data of an underlying image and re-save it as a secondary capture image or an encapsulated PDF object; this is suitable for human review only and the information is essentially lost for the purpose of further processing.

Bitmap Overlays

An annotation or contour or segmentation may be rasterized into a 2D bitmap and either encoded in unused high bits of the pixel data in a re-saved image, stored separately in an attribute of a re-saved image (Overlay Data (0x60xx,0x3000)), or stored in a separate Presentation State object.

Curves

A means of describing curves for graphics (and waveforms) was originally defined in the standard but has been removed and should not be used for QIBA purposes. Their function has been replaced by Graphic Annotations in Presentation State objects and time-based Waveform objects.

Graphic Annotations in Presentation States

Presentation States contain a means of encoding simple 2D image-relative vector graphics, and may be be used to encode, for example, isocontours of a region as a set of closed polylines. There is no standard or coded or structured means of describing their semantics, however. This format is not recommended for QIBA purposes in view of the lack of such semantics, but since they are very widely implemented in commercial systems in which the purpose is to capture consistent presentation (rather than meaning), it is possible to use these by defining conventions for the users (creators) to follow (such as using the same text label for isocontours on successive slices to indicate the same lesion).