Skip to content

kolena.workflow.TestSample#

Test samples are the inputs to your models when testing.

For example, for a model that processes specific regions within a larger image, its test sample may be defined:

from dataclasses import dataclass

from kolena.workflow import Image
from kolena.workflow.annotation import BoundingBox

@dataclass(frozen=True)
class ImageWithRegion(Image):
    region: BoundingBox

example = ImageWithRegion(
    locator="s3://my-bucket/example-image.png",  # field from Image base class
    region=BoundingBox(top_left=(0, 0), bottom_right=(100, 100)),
)

Versioning for locator files

Kolena supports versioning for files stored in Amazon S3 or Google Cloud Storage. Simply enable versioning on your S3 or GCS bucket and pass the versionId or generation as a part of the locator:

  • S3 (using versionId): s3://my-bucket/example-image.png?versionId=Bv38GKqEKxwr_HYTEXYEx6TQG_4.LkAX
  • GCS (using generation): gs://my-bucket/example-image.png?generation=1701352005168905

Metadata = Dict[str, Union[None, StrictStr, StrictFloat, StrictInt, StrictBool, str, float, int, bool, List[Union[None, StrictStr, StrictFloat, StrictInt, StrictBool, str, float, int, bool]]]] module-attribute #

Type of the metadata field that can be included on TestSample definitions. String (str) keys and scalar values (int, float, str, bool, None) as well as scalar list values are permitted.

from dataclasses import dataclass, field
from kolena.workflow import Image, Metadata

@dataclass(frozen=True)
class ImageWithMetadata(Image):
    metadata: Metadata = field(default_factory=dict)

TestSample #

Bases: TypedDataObject[_TestSampleType]

The inputs to a model.

Test samples can be customized as necessary for a workflow by extending this class or one of the built-in test sample types.

Extensions to the TestSample class may define a metadata field of type Metadata containing a dictionary of scalar properties associated with the test sample, intended for use when sorting or filtering test samples.

Kolena handles the metadata field differently from other test sample fields. Updates to the metadata object for a given test sample are merged with previously uploaded metadata. As such, metadata for a given test sample within a test case is not immutable, and should not be relied on when an implementation of Model computes inferences, or when an implementation of Evaluator evaluates metrics.

Composite #

Bases: TestSample

A test sample composed of multiple basic TestSample elements.

An example application would be each test sample is a pair of face images, and the goal is to predict whether the two images are of the same person. For this use-case the test sample can be defined as:

class FacePairSample(Composite):
    source: Image
    target: Image

To facilitate visualization for this kind of use cases, see usage of GroundTruth and Inference.

Image #

Bases: TestSample

An image located in a cloud bucket or served at a URL.

locator: str instance-attribute #

The URL of this image, using e.g. s3, gs, or https scheme (s3://my-bucket/path/to/image.png).

ImagePair #

Bases: Composite

Two Images paired together.

a: Image instance-attribute #

The left Image in the image pair.

b: Image instance-attribute #

The right Image in the image pair.

Text #

Bases: TestSample

An inline text snippet.

text: str instance-attribute #

The text snippet.

ImageText #

Bases: Composite

An image paired with a text snippet.

image: Image instance-attribute #

The Image in this image-text pair.

text: Text instance-attribute #

The text snippet in this image-text pair.

BaseVideo #

Bases: TestSample

A video clip located in a cloud bucket or served at a URL.

locator: str instance-attribute #

URL (e.g. S3, HTTPS) of the video file.

Video #

Bases: BaseVideo

A video clip located in a cloud bucket or served at a URL.

locator: str instance-attribute #

URL (e.g. S3, HTTPS) of the video file.

thumbnail: Optional[ImageAsset] = None class-attribute instance-attribute #

Optionally provide asset locator for custom video thumbnail.

start: Optional[float] = None class-attribute instance-attribute #

Optionally specify start time of video snippet, in seconds.

end: Optional[float] = None class-attribute instance-attribute #

Optionally specify end time of video snippet, in seconds.

Document #

Bases: TestSample

A remotely linked document, e.g. PDF or TXT file.

locator: str instance-attribute #

URL (e.g. S3, HTTPS) of the document.

PointCloud #

Bases: TestSample

A pointcloud file located in a cloud bucket or served at a URL.

locator: str instance-attribute #

The URL of the pointcloud file, using e.g. s3, gs, or https scheme (s3://my-bucket/path/to/image.pcd).

Audio #

Bases: TestSample

An audio file located in a cloud bucket or served at a URL.

locator: str instance-attribute #

URL (e.g. S3, HTTPS) of the audio file.