AssemblyAI

AssemblyAI is a speech-to-text plugin. It transcribes the caller’s audio into text for the LLM, using AssemblyAI’s universal streaming model with built-in end-of-turn detection.

Setup

Set your AssemblyAI API key in the worker environment. Generate a key from the AssemblyAI dashboard:

export ASSEMBLYAI_API_KEY=<key>

The Python SDK requires scipy for audio resampling. Install it with pip install scipy if it isn’t already present.

Usage

Import the plugin and pass it to the pipeline’s stt slot.

from zrt.plugins import AssemblyAISTT

stt = AssemblyAISTT()

# Pipeline(stt=stt, ...)

Parameters

Constructor parameters for the Python SDK (AssemblyAISTT). The JavaScript and Go SDKs expose equivalent options where supported, in their idiomatic form (camelCase / STTOptions struct).

Parameter	Type	Default	Description
`api_key`	`str`	`None`	AssemblyAI API key. Falls back to the `ASSEMBLYAI_API_KEY` environment variable when unset.
`input_sample_rate`	`int`	`48000`	Sample rate (Hz) of the incoming audio.
`output_sample_rate`	`int`	`16000`	Sample rate (Hz) the audio is resampled to before it is sent to AssemblyAI.
`format_turns`	`bool`	`True`	Apply capitalization and punctuation to completed turns.
`keyterms_prompt`	`list[str]`	`None`	Words or phrases to bias recognition toward.
`end_of_turn_confidence_threshold`	`float`	`0.4`	Confidence above which a turn is considered finished.
`min_end_of_turn_silence_when_confident`	`int`	`560`	Milliseconds of silence required to end a turn when end-of-turn confidence is high.
`max_turn_silence`	`int`	`2400`	Maximum silence (ms) allowed within a turn before it ends.
`speech_model`	`str`	`"universal-streaming-english"`	Recognition model: `"universal-streaming-english"` or `"universal-streaming-multilingual"`.
`language_detection`	`bool`	`False`	Automatically detect the spoken language.

Import paths

SDK	Import	Constructor
Python	`from zrt.plugins import AssemblyAISTT`	`AssemblyAISTT()`
JavaScript	`from '@zrt/js-sdk/plugins/assemblyai'`	`AssemblyAISTT()`
Go	`.../zrt-golang-sdk/plugins/assemblyai`	`assemblyai.NewSTT(assemblyai.STTOptions{})`

Transcribed text passes to the LLM once turn detection decides the caller has finished speaking.

Plugins

Inference Gateway

Speech-to-text

LLM

Text-to-speech

Realtime models

Avatars

Turn detection

Voice activity detection

Noise cancellation

Setup

Usage

Parameters

Import paths

​Setup

​Usage

​Parameters

​Import paths

Setup

Usage

Parameters

Import paths