Annotations: kolena.workflow.annotation

Annotations are visualized in Kolena as overlays on top of TestSample objects.

Annotation	Valid TestSample Types
BoundingBox	Image, Video
BoundingBox3D	PointCloud
Polygon	Image, Video
Polyline	Image, Video
Keypoints	Image, Video
SegmentationMask	Image, Video
BitmapMask	Image, Video
Label	Text, Document, Image, PointCloud, Audio, Video
TimeSegment	Audio, Video

For example, when viewing images in the Studio, any annotations (such as lists of BoundingBox objects) present in the TestSample, GroundTruth, Inference, or MetricsTestSample objects are rendered on top of the image.

ClassificationLabel = Label module-attribute

Alias for Label.

ScoredClassificationLabel = ScoredLabel module-attribute

Alias for ScoredLabel.

Annotation

Bases: TypedDataObject[_AnnotationType]

The base class for all annotation types.

BitmapMask

Bases: Annotation

Arbitrary bitmap mask. The locator is the URL to the image file representing the mask.

locator: str instance-attribute

URL of the bitmap data.

BoundingBox

Bases: Annotation

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices.

The reserved fields width, height, area, and aspect_ratio are automatically populated with values derived from the provided coordinates.

top_left: Tuple[float, float] instance-attribute

The top left vertex (in (x, y) pixel coordinates) of this bounding box.

bottom_right: Tuple[float, float] instance-attribute

The bottom right vertex (in (x, y) pixel coordinates) of this bounding box.

BoundingBox3D

Bases: Annotation

Three-dimensional cuboid bounding box in a right-handed coordinate system.

Specified by (x, y, z) coordinates for the center of the cuboid, (x, y, z) dimensions, and a rotation parameter specifying the degrees of rotation about each axis (x, y, z) ranging [-π, π].

The reserved field volume is automatically derived from the provided dimensions.

center: Tuple[float, float, float] instance-attribute

(x, y, z) coordinates specifying the center of the bounding box.

dimensions: Tuple[float, float, float] instance-attribute

(x, y, z) measurements specifying the dimensions of the bounding box.

rotations: Tuple[float, float, float] instance-attribute

Rotations in degrees about each (x, y, z) axis.

Keypoints

Bases: Annotation

Array of any number of keypoints specified in pixel coordinates.

Optionally include a skeleton field to indicating connections between points indices.

Keypoints(points=[(0,0),(0,1),(1,1)], skeleton=[(0,1),(0,2)]) will represent three points with a connection between the (0,0) and (0,1) point, and between the (0,0) and (1,1) point.

points: List[Tuple[float, float]] instance-attribute

The sequence of discrete (x, y) pixel coordinates comprising this keypoints annotation.

Label

Bases: Annotation

Label, e.g. for classification.

label: str instance-attribute

String label for this classification.

LabeledBoundingBox

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices and a string label.

label: str instance-attribute

The label (e.g. model classification) associated with this bounding box.

LabeledBoundingBox3D

Bases: BoundingBox3D

BoundingBox3D with an additional string label.

label: str instance-attribute

The label associated with this 3D bounding box.

LabeledPolygon

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates and a string label.

label: str instance-attribute

The label (e.g. model classification) associated with this polygon.

LabeledTimeSegment

Bases: TimeSegment

Time segment with accompanying label, e.g. audio transcription.

label: str instance-attribute

The label associated with this time segment.

Polygon

Bases: Annotation

Arbitrary polygon specified by three or more pixel coordinates.

points: List[Tuple[float, float]] instance-attribute

The sequence of (x, y) pixel coordinates comprising the boundary of this polygon.

Polyline

Bases: Annotation

Polyline with any number of vertices specified in pixel coordinates.

points: List[Tuple[float, float]] instance-attribute

The sequence of connected (x, y) pixel coordinates comprising this polyline.

ScoredBoundingBox

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices and a float score.

score: float instance-attribute

The score (e.g. model confidence) associated with this bounding box.

ScoredBoundingBox3D

Bases: BoundingBox3D

BoundingBox3D with an additional float score.

score: float instance-attribute

The score associated with this 3D bounding box.

ScoredLabel

Bases: Label

Label with accompanying score.

score: float instance-attribute

Score associated with this label.

ScoredLabeledBoundingBox

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices, a string label, and a float score.

label: str instance-attribute

The label (e.g. model classification) associated with this bounding box.

score: float instance-attribute

The score (e.g. model confidence) associated with this bounding box.

ScoredLabeledBoundingBox3D

Bases: BoundingBox3D

BoundingBox3D with an additional string label and float score.

label: str instance-attribute

The label associated with this 3D bounding box.

score: float instance-attribute

The score associated with this 3D bounding box.

ScoredLabeledPolygon

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates with a string label and a float score.

label: str instance-attribute

The label (e.g. model classification) associated with this polygon.

score: float instance-attribute

The score (e.g. model confidence) associated with this polygon.

ScoredLabeledTimeSegment

Bases: TimeSegment

Time segment with accompanying label and score.

label: str instance-attribute

The label associated with this time segment.

score: float instance-attribute

The score associated with this time segment.

ScoredPolygon

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates and a float score.

score: float instance-attribute

The score (e.g. model confidence) associated with this polygon.

ScoredTimeSegment

Bases: TimeSegment

Time segment with additional float score, representing e.g. model prediction confidence.

score: float instance-attribute

The score associated with this time segment.

SegmentationMask

Bases: Annotation

Raster segmentation mask. The locator is the URL to the image file representing the segmentation mask.

The segmentation mask must be rendered as a single-channel, 8-bit-depth (grayscale) image. For the best results, use a lossless file format such as PNG. Each pixel's value is the numerical ID of its class label, as specified in the labels map. Any pixel value not present in the labels map is rendered as part of the background.

For example, labels = {255: "object"} will highlight all pixels with the value of 255 as "object". Every other pixel value will be transparent.

labels: Dict[int, str] instance-attribute

Mapping of unique label IDs (pixel values) to unique label values.

locator: str instance-attribute

URL of the segmentation mask image.

TimeSegment

Bases: Annotation

Segment of time in the associated audio or video file.

When a group is specified, segments are displayed on Kolena with different colors for each group present in a List[TimeSegment]. Example usage:

transcription: List[TimeSegment] = [
    LabeledTimeSegment(group="A", label="Knock, knock.", start=0, end=1),
    LabeledTimeSegment(group="B", label="Who's there?", start=2, end=3),
    LabeledTimeSegment(group="A", label="Example.", start=3.5, end=4),
    LabeledTimeSegment(group="B", label="Example who?", start=4.5, end=5.5),
    LabeledTimeSegment(group="A", label="Example illustrating two-person dialogue using `group`.", start=6, end=9),
]

start: float instance-attribute

Start time, in seconds, of this segment.

end: float instance-attribute

End time, in seconds, of this segment.