> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeroruntime.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Use the OpenAI LLM plugin in a Zero Runtime pipeline, including Azure OpenAI. Setup, options, and usage in Python and JavaScript.

OpenAI is an **LLM** plugin. It takes the transcribed conversation and generates the reply
using OpenAI's chat models, with tool calling and an optional low-latency WebSocket
streaming mode.

## Setup

Set your OpenAI API key in the worker environment. Generate a key from the [OpenAI dashboard](https://platform.openai.com/api-keys):

```bash theme={null}
export OPENAI_API_KEY=<key>
```

## Usage

Import the plugin and pass it to the pipeline's `llm` slot.

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime.plugins import OpenAILLM

  llm = OpenAILLM(
      model="gpt-5.4-nano",
      temperature=0.7,
  )

  # Pipeline(llm=llm, ...)
  ```

  ```typescript Node JS theme={null}
  import { OpenAILLM } from '@zeroruntime/js-sdk/plugins';

  const llm = OpenAILLM({
    model: 'gpt-5.4-nano',
    temperature: 0.7,
  });

  // Pipeline({ llm, ... })
  ```
</CodeGroup>

<Note>
  The default model is `gpt-5.4-nano`.
</Note>

## Low-latency streaming

Set `streaming` to use OpenAI's WebSocket Responses API instead of HTTP chat
completions. The connection is reused across turns and continues with
`previous_response_id` for lower per-turn latency.

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime.plugins import OpenAILLM

  llm = OpenAILLM(model="gpt-5.4-nano", streaming=True)
  ```

  ```typescript Node JS theme={null}
  import { OpenAILLM } from '@zeroruntime/js-sdk/plugins';

  const llm = OpenAILLM({ model: 'gpt-5.4-nano', streaming: true });
  ```
</CodeGroup>

## Azure OpenAI

To run OpenAI chat models through **Azure OpenAI**, use the separate `AzureOpenAILLM`
plugin. It reads its configuration from Azure environment variables when arguments are
omitted:

```bash theme={null}
export AZURE_OPENAI_API_KEY=<key>
export AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com
```

<CodeGroup>
  ```python Python theme={null}
  from zeroruntime.plugins import AzureOpenAILLM

  llm = AzureOpenAILLM(
      azure_endpoint="https://<resource>.openai.azure.com",
      deployment="<deployment-name>",
      api_version="2024-10-21",
  )

  # Pipeline(llm=llm, ...)
  ```
</CodeGroup>

<Note>
  `AzureOpenAILLM` ships in the Python SDK only. The Node JS SDK does not ship an Azure OpenAI
  provider.
</Note>

<Note>
  `deployment` is the custom name you chose when deploying a model in Azure (not the
  underlying model name). Authenticate with `api_key`, or fall back to the
  `AZURE_OPENAI_API_KEY` environment variable.
</Note>

## Configuration Options

*Constructor parameters for `OpenAILLM`. The Python and Node JS SDKs share these field names.*

### Core

| Parameter           | Type    | Default          | Description                                                                                           |
| ------------------- | ------- | ---------------- | ----------------------------------------------------------------------------------------------------- |
| `api_key`           | `str`   | `None`           | OpenAI API key. Falls back to the `OPENAI_API_KEY` environment variable when unset.                   |
| `model`             | `str`   | `"gpt-5.4-nano"` | Chat model used to generate replies.                                                                  |
| `temperature`       | `float` | `0.7`            | Sampling randomness; lower is more deterministic.                                                     |
| `tool_choice`       | `str`   | `None`           | How the model decides when to call tools (`"none"`, `"auto"`, `"required"`, or a specific tool spec). |
| `max_output_tokens` | `int`   | `1024`           | Caps the length of each reply. The legacy `max_completion_tokens` kwarg is accepted as an alias.      |
| `response_format`   | `dict`  | `None`           | Output format spec, e.g. `{"type": "json_object"}` or a `json_schema` for Structured Outputs.         |
| `stop`              | `str`   | `None`           | Up to 4 stop sequences at which generation halts.                                                     |
| `user`              | `str`   | `None`           | Stable end-user identifier for abuse monitoring.                                                      |

### Sampling

| Parameter             | Type    | Default | Description                                               |
| --------------------- | ------- | ------- | --------------------------------------------------------- |
| `top_p`               | `float` | `None`  | Nucleus-sampling probability cutoff.                      |
| `frequency_penalty`   | `float` | `None`  | Penalizes frequent tokens, reducing repetition.           |
| `presence_penalty`    | `float` | `None`  | Penalizes tokens already present, encouraging new topics. |
| `seed`                | `int`   | `None`  | Seed for deterministic sampling.                          |
| `parallel_tool_calls` | `bool`  | `None`  | Allow the model to call multiple tools in one turn.       |

### Reasoning models

| Parameter          | Type  | Default | Description                                                                                                                      |
| ------------------ | ----- | ------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `reasoning_effort` | `str` | `None`  | Reasoning depth (`"minimal"`, `"low"`, `"medium"`, `"high"`) for reasoning / GPT-5 models. `"none"` requires `gpt-5.1` or later. |
| `verbosity`        | `str` | `None`  | Output verbosity (`"low"`, `"medium"`, `"high"`) for reasoning / GPT-5 models.                                                   |

### Streaming & client

| Parameter   | Type   | Default | Description                                                 |
| ----------- | ------ | ------- | ----------------------------------------------------------- |
| `streaming` | `bool` | `False` | Use the WebSocket Responses API for lower per-turn latency. |
| `store`     | `bool` | `True`  | Persist responses server-side for later retrieval.          |
| `wss_url`   | `str`  | `None`  | Override the WebSocket Responses URL.                       |

## Import paths

| SDK     | Import                                                    | Constructor          |
| ------- | --------------------------------------------------------- | -------------------- |
| Python  | `from zeroruntime.plugins import OpenAILLM`               | `OpenAILLM(...)`     |
| Node JS | `import { OpenAILLM } from '@zeroruntime/js-sdk/plugins'` | `OpenAILLM({ ... })` |

<Note>
  Azure OpenAI is provided by the separate `AzureOpenAILLM` plugin
  (`from zeroruntime.plugins import AzureOpenAILLM`).
</Note>

The reply is streamed to the [text-to-speech](/plugins/tts/cartesia) plugin, which
synthesizes the agent's voice.