> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Use the OpenAI speech-to-text plugin in a Zero Runtime pipeline, including Azure OpenAI. Setup, options, and usage.

OpenAI is a **speech-to-text** plugin. It transcribes the caller's audio into text for the
LLM using OpenAI's transcription models. By default it streams over OpenAI's realtime
transcription WebSocket; it can also run in non-streaming HTTP mode with built-in voice
activity detection.

## Setup

Set your OpenAI API key in the worker environment. Generate a key from the [OpenAI dashboard](https://platform.openai.com/api-keys):

```bash theme={null}
export OPENAI_API_KEY=<key>
```

## Usage

Import the plugin and pass it to the pipeline's `stt` slot.

<CodeGroup>
  ```python Python theme={null}
  from zrt.plugins import OpenAISTT

  stt = OpenAISTT(
      model="gpt-4o-transcribe",
      language="en",
  )

  # Pipeline(stt=stt, ...)
  ```
</CodeGroup>

## Azure OpenAI

To run OpenAI transcription through **Azure OpenAI**, use the separate `AzureOpenAISTT`
plugin. It reads its configuration from Azure environment variables when arguments are
omitted:

```bash theme={null}
export AZURE_OPENAI_API_KEY=<key>
export AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com
export AZURE_OPENAI_STT_DEPLOYMENT=<deployment-name>
```

```python Python theme={null}
from zrt.plugins import AzureOpenAISTT

stt = AzureOpenAISTT(
    azure_endpoint="https://<resource>.openai.azure.com",
    deployment="<deployment-name>",
    api_version="2025-03-01-preview",
)

# Pipeline(stt=stt, ...)
```

<Note>
  `api_key` falls back to `AZURE_API_KEY`, then `AZURE_OPENAI_API_KEY`. `deployment` falls
  back to `AZURE_OPENAI_STT_DEPLOYMENT`. `AzureOpenAISTT` runs non-streaming by default
  (`stream=False`).
</Note>

## Parameters

*Constructor parameters for the Python SDK (`OpenAISTT`).*

| Parameter                 | Type    | Default               | Description                                                                                                      |
| ------------------------- | ------- | --------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `api_key`                 | `str`   | `None`                | OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset.                              |
| `model`                   | `str`   | `"gpt-4o-transcribe"` | Transcription model. `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`, or `whisper-1`. |
| `language`                | `str`   | `"en"`                | Expected language of the caller's speech (ISO-639-1).                                                            |
| `stream`                  | `bool`  | `True`                | Stream partial transcripts as the caller speaks.                                                                 |
| `input_sample_rate`       | `int`   | `48000`               | Input audio sample rate in Hz.                                                                                   |
| `output_sample_rate`      | `int`   | `24000`               | Resampled rate sent to the model in Hz.                                                                          |
| `prompt`                  | `str`   | `None`                | Biasing prompt to guide transcription vocabulary/style.                                                          |
| `turn_detection`          | `str`   | `"server_vad"`        | VAD mode: `"server_vad"` or `"semantic_vad"`.                                                                    |
| `vad_threshold`           | `float` | `None`                | Speech-detection sensitivity, 0.0-1.0.                                                                           |
| `vad_prefix_padding_ms`   | `int`   | `None`                | Audio kept before detected speech, in ms.                                                                        |
| `vad_silence_duration_ms` | `int`   | `None`                | Silence before end-of-turn, in ms.                                                                               |
| `noise_reduction`         | `str`   | `"near_field"`        | Input denoising profile: `"near_field"`, `"far_field"`, or `None` to disable.                                    |
| `response_format`         | `str`   | `"json"`              | Transcript format (`"json"`, `"text"`, ...).                                                                     |
| `base_url`                | `str`   | `None`                | Override the OpenAI API base URL.                                                                                |

### Azure OpenAI (`AzureOpenAISTT`)

| Parameter            | Type    | Default                | Description                                                                                                         |
| -------------------- | ------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `api_key`            | `str`   | `None`                 | Azure OpenAI API key. Falls back to `AZURE_API_KEY`, then `AZURE_OPENAI_API_KEY`.                                   |
| `azure_endpoint`     | `str`   | `None`                 | Azure OpenAI endpoint. Falls back to `AZURE_OPENAI_ENDPOINT`.                                                       |
| `deployment`         | `str`   | `None`                 | Deployment name. Falls back to `AZURE_OPENAI_STT_DEPLOYMENT`.                                                       |
| `api_version`        | `str`   | `"2025-03-01-preview"` | Azure OpenAI REST API version.                                                                                      |
| `model`              | `str`   | `""`                   | Underlying model name passed in the request body (e.g. `"whisper-1"`). Usually left empty when `deployment` is set. |
| `language`           | `str`   | `"en"`                 | ISO-639-1 language code of the spoken audio.                                                                        |
| `stream`             | `bool`  | `False`                | When `True`, receive transcription results as a streaming response.                                                 |
| `input_sample_rate`  | `int`   | `48000`                | Input audio sample rate in Hz.                                                                                      |
| `output_sample_rate` | `int`   | `24000`                | Resampled rate sent to the Azure endpoint in Hz.                                                                    |
| `prompt`             | `str`   | `None`                 | Optional text to guide transcription style or continue prior context.                                               |
| `temperature`        | `float` | `None`                 | Sampling temperature in `[0.0, 1.0]`.                                                                               |
| `response_format`    | `str`   | `"json"`               | Response format: `"json"`, `"verbose_json"`, `"text"`, `"srt"`, or `"vtt"`.                                         |
| `turn_detection`     | `str`   | `"server_vad"`         | Turn-detection strategy for streaming: `"server_vad"` or `"none"`.                                                  |

## Import paths

| SDK        | Import                                              | Constructor                              |
| ---------- | --------------------------------------------------- | ---------------------------------------- |
| Python     | `from zrt.plugins import OpenAISTT, AzureOpenAISTT` | `OpenAISTT(...)` / `AzureOpenAISTT(...)` |
| JavaScript | -                                                   | Coming soon                              |
| Go         | -                                                   | Coming soon                              |

Transcribed text passes to the [LLM](/plugins/llm/google) once
[turn detection](/plugins/turn-detection/namo) decides the caller has finished speaking.