> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Use the OpenAI speech-to-text plugin in a Zero Runtime pipeline, including Azure OpenAI. Setup, options, and usage.

OpenAI is a **speech-to-text** plugin. It transcribes the caller's audio into text for the
LLM using OpenAI's transcription models. By default it streams over OpenAI's realtime
transcription WebSocket; it can also run in non-streaming HTTP mode with built-in voice
activity detection.

## Setup

Set your OpenAI API key in the worker environment. Generate a key from the [OpenAI dashboard](https://platform.openai.com/api-keys):

```bash theme={null}
export OPENAI_API_KEY=<key>
```

## Usage

Import the plugin and pass it to the pipeline's `stt` slot.

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime.plugins import OpenAISTT

  stt = OpenAISTT(
      model="gpt-4o-transcribe",
      language="en",
  )

  # Pipeline(stt=stt, ...)
  ```

  ```typescript Node JS theme={null}
  import { OpenAISTT } from '@zeroruntime/js-sdk/plugins';

  const stt = OpenAISTT({
    model: 'gpt-4o-transcribe',
    language: 'en',
  });

  // Pipeline({ stt, ... })
  ```
</CodeGroup>

## Azure OpenAI

To run OpenAI transcription through **Azure OpenAI**, use the separate `AzureOpenAISTT`
plugin. It reads its configuration from Azure environment variables when arguments are
omitted:

```bash theme={null}
export AZURE_OPENAI_API_KEY=<key>
export AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com
export AZURE_OPENAI_STT_DEPLOYMENT=<deployment-name>
```

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime.plugins import AzureOpenAISTT

  stt = AzureOpenAISTT(
      azure_endpoint="https://<resource>.openai.azure.com",
      deployment="<deployment-name>",
      api_version="2025-03-01-preview",
  )

  # Pipeline(stt=stt, ...)
  ```
</CodeGroup>

<Note>
  `AzureOpenAISTT` ships in the Python SDK only. The Node JS SDK does not ship an Azure OpenAI
  provider.
</Note>

<Note>
  `api_key` falls back to `AZURE_API_KEY`, then `AZURE_OPENAI_API_KEY`. `deployment` falls
  back to `AZURE_OPENAI_STT_DEPLOYMENT`. `AzureOpenAISTT` runs non-streaming by default
  (`stream=False`).
</Note>

## Parameters

*Constructor parameters for `OpenAISTT`. The Python and Node JS SDKs share these field names.*

| Parameter | Type | Default | Description |
| - | - | - | - |
| `api_key` | `str` | `None` | OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset. |
| `model` | `str` | `"gpt-4o-transcribe"` | Transcription model. `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`, or `whisper-1`. |
| `language` | `str` | `"en"` | Expected language of the caller's speech (ISO-639-1). |
| `stream` | `bool` | `True` | Stream partial transcripts as the caller speaks. |
| `input_sample_rate` | `int` | `48000` | Input audio sample rate in Hz. |
| `output_sample_rate` | `int` | `24000` | Resampled rate sent to the model in Hz. |
| `prompt` | `str` | `None` | Biasing prompt to guide transcription vocabulary/style. |
| `turn_detection` | `str` | `"server_vad"` | VAD mode: `"server_vad"` or `"semantic_vad"`. |
| `vad_threshold` | `float` | `None` | Speech-detection sensitivity, 0.0-1.0. |
| `vad_prefix_padding_ms` | `int` | `None` | Audio kept before detected speech, in ms. |
| `vad_silence_duration_ms` | `int` | `None` | Silence before end-of-turn, in ms. |
| `noise_reduction` | `str` | `"near_field"` | Input denoising profile: `"near_field"`, `"far_field"`, or `None` to disable. |
| `response_format` | `str` | `"json"` | Transcript format (`"json"`, `"text"`, ...). |
| `base_url` | `str` | `None` | Override the OpenAI API base URL. |

### Azure OpenAI (`AzureOpenAISTT`)

| Parameter | Type | Default | Description |
| - | - | - | - |
| `api_key` | `str` | `None` | Azure OpenAI API key. Falls back to `AZURE_API_KEY`, then `AZURE_OPENAI_API_KEY`. |
| `azure_endpoint` | `str` | `None` | Azure OpenAI endpoint. Falls back to `AZURE_OPENAI_ENDPOINT`. |
| `deployment` | `str` | `None` | Deployment name. Falls back to `AZURE_OPENAI_STT_DEPLOYMENT`. |
| `api_version` | `str` | `"2025-03-01-preview"` | Azure OpenAI REST API version. |
| `model` | `str` | `""` | Underlying model name passed in the request body (e.g. `"whisper-1"`). Usually left empty when `deployment` is set. |
| `language` | `str` | `"en"` | ISO-639-1 language code of the spoken audio. |
| `stream` | `bool` | `False` | When `True`, receive transcription results as a streaming response. |
| `input_sample_rate` | `int` | `48000` | Input audio sample rate in Hz. |
| `output_sample_rate` | `int` | `24000` | Resampled rate sent to the Azure endpoint in Hz. |
| `prompt` | `str` | `None` | Optional text to guide transcription style or continue prior context. |
| `temperature` | `float` | `None` | Sampling temperature in `[0.0, 1.0]`. |
| `response_format` | `str` | `"json"` | Response format: `"json"`, `"verbose_json"`, `"text"`, `"srt"`, or `"vtt"`. |
| `turn_detection` | `str` | `"server_vad"` | Turn-detection strategy for streaming: `"server_vad"` or `"none"`. |

## Import paths

| SDK | Import | Constructor |
| - | - | - |
| Python | `from zeroruntime.plugins import OpenAISTT, AzureOpenAISTT` | `OpenAISTT(...)` / `AzureOpenAISTT(...)` |

Transcribed text passes to the [LLM](/runtime-sdk/plugins/llm/google) once
[turn detection](/runtime-sdk/plugins/turn-detection/namo) decides the caller has finished speaking.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.