Audio#
In this document we will review best practices when setting up Kolena datasets for Audio problems.
Basics#
Supported File Data Formats#
The Kolena SDK supports uploading of data in the Pandas
DataFrame format.
The Kolena web app supports the following file formats.
| Format | Description |
|---|---|
.csv |
Comma-separated values file, ideal for tabular data. |
.parquet |
Apache Parquet format, efficient for columnar storage. |
.jsonl |
JSON Lines format, suitable for handling nested data. |
Using the locator#
Kolena uses references to files stored in your cloud storage to render them. Refer to "Connecting Cloud Storage" for details on how to configure this.
Audio samples can be visualized on Kolena in one of two ways.
Gallery mode: visualizes each audio as a tile.
To enable the Gallery view store references to audio files in a column named locator. locator can be used as
the unique identifier of the datapoint which is also referenced by your model results.
Example
The Speaker Diarization ↗ example showcases how audio files can be uploaded in Gallery mode.
Tip
You can use TimeSegment's
or LabeledTimeSegment
to highlight segments of your audio file that are of interest to you.
Tabular: visualizes each datapoint as a row on a table with the audio file as an asset available for replay. To enable
this view, please use AudioAsset to link your audio files
in your dataset.
Example
The
Automatic Speech Recognition ↗
example showcases how AudioAssets can be attached to datapoints.
Kolena supports flac, mp3, wav, acc, ogg, ra and other web browser supported audio types.
Using fields#
You can add additional information about your audio files or other features of your data by
adding columns to the .CSV file or DataFrame with the meta-data name and values in each row.
Below is an example datapoint:
| Locator | Num Speakers | Average Amplitude | audio_length |
|---|---|---|---|
s3://kolena-public-examples/ICSI-corpus/data/audio/Bdb001/interval0.mp3 |
5 | 0.08951 | 448.454 |
Uploading Model Results#
Model results contain your model inferences as well as any custom metrics that you wish to monitor on Kolena. The data structure of model results is very similar to the structure of a dataset.
- make sure to link your inferences to the dataset using the same unique ID (for example the
locator) you used when uploading the dataset. - use
ScoredTimeSegmentorScoredLabeledTimeSegmentannotations to indicate the inference confidence score.