kolena.workflow

• Developer Guide: Building a Workflow ↗
• Examples: kolena/examples ↗

kolena.workflow contains the definitions to build a workflow:

1. Design data types, including any annotations or assets:

   Defining a workflow

   TestSample, GroundTruth, and Inference can be thought of as the data model, or schema, for a workflow.

   Learn more ↗

   • TestSample: model inputs, e.g. images, videos, documents
   • GroundTruth: expected model outputs
   • Inference: real model outputs

2. Define metrics and how they are computed:

   • Evaluator: metrics computation engine

3. Create tests:

   Managing tests

   See the test case and test suite developer guide for an introduction to the test case and test suite concept.

   • TestCase: a test dataset, or a slice thereof
   • TestSuite: a collection of test cases

4. Test models:

   • Model: descriptor for a model
   • test: interface to run tests