Skip to content

Audio#

In this document we will review best practices when setting up Kolena datasets for Audio problems.

Basics#

Supported File Data Formats#

The Kolena SDK supports upload of data in the Pandas DataFrame format.

The Kolena web app supports the following file formats.

Format Description
.csv Comma-separated values file, ideal for tabular data.
.parquet Apache Parquet format, efficient for columnar storage.
.jsonl JSON Lines format, suitable for handling nested data.

Using the locator#

Kolena uses references to files stored in your cloud storage to render them. Refer to "Connecting Cloud Storage" for details on how to configure this.

Audio samples can be visualized on Kolena one of two ways.

Gallery mode: visualizes each audio as a tile. To enable the Gallery view store references to audio files in a column named locator. locator can be used as the unique identifier of the datapoint which is also referenced by your model results.

Example

The Speaker Diarization ↗ example showcases how audio files can be uploaded in Gallery mode.

Audio Gallery View Audio Gallery View

Gallery View

Tip

You can use TimeSegment's or LabeledTimeSegment to highlight segments of your audio file that are of interest to you.

Tabular: visualizes each datapoint as a row on a table with the audio file as an asset available for replay. To enable this view, please use AudioAsset to link your audio files in your dataset.

Example

The Automatic Speech Recognition ↗ example showcases how AudioAssets can be attached to datapoints.

Audio Tabular View Audio Tabular View

Tabular View

Kolena supports flac, mp3, wav, acc, ogg, ra and other web browser supported audio types.

Using fields#

You can add additional information about your audio files or other features of your data by adding columns to the .CSV file or DataFrame with the meta-data name and values in each row. Below is an example datapoint:

Locator Num Speakers Average Amplitude audio_length
s3://kolena-public-examples/ICSI-corpus/data/audio/Bdb001/interval0.mp3 5 0.08951 448.454

Uploading Model Results#

Model results contain your model inferences as well as any custom metrics that you wish to monitor on Kolena. The data structure of model results is very similar to the structure of a dataset.

  • make sure to link your inferences to the dataset using the same unique ID (for example the locator) you used when uploading the dataset.
  • use ScoredTimeSegment or ScoredLabeledTimeSegment annotations to indicate the inference confidence score.