Skip to content

kolena.annotation

Annotations are visualized in Kolena as overlays on top of datapoints.

Annotation Conditions for visualization
BoundingBox Limited to Image or Video data
BoundingBox3D Limited to PointCloud data
Polygon Limited to Image or Video data
Polyline Limited to Image or Video data
Polyline3D Can be visualized on PointCloud data or in standalone 3D space
Keypoints Limited to Image or Video data
Keypoints3D Can be visualized on PointCloud data, or in standalone 3D space
SegmentationMask Limited to Image or Video data
BitmapMask Limited to Image or Video data
Label Valid for all data
TimeSegment Limited to Audio or Video data
TextSegment Limited to Text data
CustomAnnotation Experimental feature for custom annotations

For example, when viewing images in the Studio, any annotations (such as lists of BoundingBox objects) present in the datapoints are rendered on top of the image.

ClassificationLabel = Label module-attribute #

Alias for Label.

ScoredClassificationLabel = ScoredLabel module-attribute #

Alias for ScoredLabel.

Annotation #

Bases: TypedDataObject[_AnnotationType]

The base class for all annotation types.

BoundingBox #

Bases: Annotation

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices.

The reserved fields width, height, area, and aspect_ratio are automatically populated with values derived from the provided coordinates.

top_left: Tuple[float, float] instance-attribute #

The top left vertex (in (x, y) pixel coordinates) of this bounding box.

bottom_right: Tuple[float, float] instance-attribute #

The bottom right vertex (in (x, y) pixel coordinates) of this bounding box.

LabeledBoundingBox #

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices and a string label.

label: str instance-attribute #

The label (e.g. model classification) associated with this bounding box.

ScoredBoundingBox #

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices and a float score.

score: float instance-attribute #

The score (e.g. model confidence) associated with this bounding box.

ScoredLabeledBoundingBox #

Bases: BoundingBox

Rectangular bounding box specified with pixel coordinates of the top left and bottom right vertices, a string label, and a float score.

label: str instance-attribute #

The label (e.g. model classification) associated with this bounding box.

score: float instance-attribute #

The score (e.g. model confidence) associated with this bounding box.

Polygon #

Bases: Annotation

Arbitrary polygon specified by three or more pixel coordinates.

points: List[Tuple[float, float]] instance-attribute #

The sequence of (x, y) pixel coordinates comprising the boundary of this polygon.

LabeledPolygon #

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates and a string label.

label: str instance-attribute #

The label (e.g. model classification) associated with this polygon.

ScoredPolygon #

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates and a float score.

score: float instance-attribute #

The score (e.g. model confidence) associated with this polygon.

ScoredLabeledPolygon #

Bases: Polygon

Arbitrary polygon specified by three or more pixel coordinates with a string label and a float score.

label: str instance-attribute #

The label (e.g. model classification) associated with this polygon.

score: float instance-attribute #

The score (e.g. model confidence) associated with this polygon.

Keypoints #

Bases: Annotation

Array of any number of keypoints specified in pixel coordinates.

Optionally include a skeleton field to indicating connections between points indices.

Keypoints(points=[(0,0),(0,1),(1,1)], skeleton=[(0,1),(0,2)]) will represent three points with a connection between the (0,0) and (0,1) point, and between the (0,0) and (1,1) point.

points: List[Tuple[float, float]] instance-attribute #

The sequence of discrete (x, y) pixel coordinates comprising this keypoints annotation.

Keypoints3D #

Bases: Annotation

Array of any number of keypoints specified in a right-handed coordinate system.

Optionally include a skeleton field to indicating connections between points indices.

Keypoints(points=[(0,0,0),(1,0,0),(0,1,0),(0,0,1)], skeleton=[(1,2),(0,3)]) will represent four points with a connection between the (1,0,0) and (0,1,0) point, and between the (0,0,0) and (0,0,1) point.

points: List[Tuple[float, float, float]] instance-attribute #

The sequence of discrete (x, y, z) coordinates comprising this keypoints annotation.

Polyline #

Bases: Annotation

Polyline with any number of vertices specified in pixel coordinates.

points: List[Tuple[float, float]] instance-attribute #

The sequence of connected (x, y) pixel coordinates comprising this polyline.

Polyline3D #

Bases: Annotation

A three-dimensional Polyline with any number of vertices specified a right-handed coordinate system.

points: List[Tuple[float, float, float]] instance-attribute #

The sequence of connected (x, y, z) coordinates comprising this polyline.

BoundingBox3D #

Bases: Annotation

Three-dimensional cuboid bounding box in a right-handed coordinate system.

Specified by (x, y, z) coordinates for the center of the cuboid, (x, y, z) dimensions, and a rotation parameter specifying the degrees of rotation about each axis (x, y, z) ranging [-π, π].

The reserved field volume is automatically derived from the provided dimensions.

center: Tuple[float, float, float] instance-attribute #

(x, y, z) coordinates specifying the center of the bounding box.

dimensions: Tuple[float, float, float] instance-attribute #

(x, y, z) measurements specifying the dimensions of the bounding box.

rotations: Tuple[float, float, float] instance-attribute #

Rotations in degrees about each (x, y, z) axis.

LabeledBoundingBox3D #

Bases: BoundingBox3D

BoundingBox3D with an additional string label.

label: str instance-attribute #

The label associated with this 3D bounding box.

ScoredBoundingBox3D #

Bases: BoundingBox3D

BoundingBox3D with an additional float score.

score: float instance-attribute #

The score associated with this 3D bounding box.

ScoredLabeledBoundingBox3D #

Bases: BoundingBox3D

BoundingBox3D with an additional string label and float score.

label: str instance-attribute #

The label associated with this 3D bounding box.

score: float instance-attribute #

The score associated with this 3D bounding box.

SegmentationMask #

Bases: Annotation

Raster segmentation mask. The locator is the URL to the image file representing the segmentation mask.

The segmentation mask must be rendered as a single-channel, 8-bit-depth (grayscale) image. For the best results, use a lossless file format such as PNG. Each pixel's value is the numerical ID of its class label, as specified in the labels map. Any pixel value not present in the labels map is rendered as part of the background.

For example, labels = {255: "object"} will highlight all pixels with the value of 255 as "object". Every other pixel value will be transparent.

labels: Dict[int, str] instance-attribute #

Mapping of unique label IDs (pixel values) to unique label values.

locator: str instance-attribute #

URL of the segmentation mask image.

BitmapMask #

Bases: Annotation

Arbitrary bitmap mask. The locator is the URL to the image file representing the mask.

locator: str instance-attribute #

URL of the bitmap data.

Label #

Bases: Annotation

Label, e.g. for classification.

label: str instance-attribute #

String label for this classification.

ScoredLabel #

Bases: Label

Label with accompanying score.

score: float instance-attribute #

Score associated with this label.

TimeSegment #

Bases: Annotation

Segment of time in the associated audio or video file.

When a group is specified, segments are displayed on Kolena with different colors for each group present in a List[TimeSegment]. Example usage:

transcription: List[TimeSegment] = [
    LabeledTimeSegment(group="A", label="Knock, knock.", start=0, end=1),
    LabeledTimeSegment(group="B", label="Who's there?", start=2, end=3),
    LabeledTimeSegment(group="A", label="Example.", start=3.5, end=4),
    LabeledTimeSegment(group="B", label="Example who?", start=4.5, end=5.5),
    LabeledTimeSegment(group="A", label="Example illustrating two-person dialogue using `group`.", start=6, end=9),
]

start: float instance-attribute #

Start time, in seconds, of this segment.

end: float instance-attribute #

End time, in seconds, of this segment.

LabeledTimeSegment #

Bases: TimeSegment

Time segment with accompanying label, e.g. audio transcription.

label: str instance-attribute #

The label associated with this time segment.

ScoredTimeSegment #

Bases: TimeSegment

Time segment with additional float score, representing e.g. model prediction confidence.

score: float instance-attribute #

The score associated with this time segment.

ScoredLabeledTimeSegment #

Bases: TimeSegment

Time segment with accompanying label and score.

label: str instance-attribute #

The label associated with this time segment.

score: float instance-attribute #

The score associated with this time segment.

TextSegment #

Bases: Annotation

Represents a segment of text within a specified text field to highlight. The start index is inclusive and the end index is exclusive, following the convention of Python string slicing. The following example adds highlights to two text fields named text and summary.

text = "Hello, world"
summary = "A summary of current documentation."

text_segments: List[TextSegment] = [
    # Highlights "Hello" in "Hello, world"
    TextSegment(text_field="text", start=0, end=5),
    # Highlights "summary of" in "A summary of current documentation."
    TextSegment(text_field="summary", start=2, end=12),
]

text_field: str instance-attribute #

Text field column name containing the text segments.

start: int instance-attribute #

Zero-indexed start index (inclusive) of the text segment.

end: int instance-attribute #

Zero-indexed end index (exclusive) of the text segment.

LabeledTextSegment #

Bases: TextSegment

Text segment with accompanying label, e.g. Location.

label: str instance-attribute #

The label associated with this text segment.

ScoredTextSegment #

Bases: TextSegment

Text segment with additional float score, representing e.g. model prediction confidence.

score: float instance-attribute #

The score associated with this text segment.

ScoredLabeledTextSegment #

Bases: TextSegment

Text segment with accompanying label and score.

label: str instance-attribute #

The label associated with this text segment.

score: float instance-attribute #

The score associated with this text segment.

CustomAnnotation #

Bases: Annotation

Experimental

This class is considered experimental. Please reach out to the Kolena team for support if you need custom annotations.

Custom annotations.

kind: str instance-attribute #

The type of this custom annotation.