Integrate a Voice Agent with JavaScript

Zero Runtime lets you integrate AI agents with real-time voice interaction into your JavaScript application within minutes. In this quickstart, you’ll create an AI agent that joins a meeting room and interacts with users through voice using the Google Gemini Live API.

Prerequisites

Before proceeding, ensure that your development environment meets the following requirements:

A Zero Runtime account (create one from the Zero Runtime Dashboard).
Node.js and Python 3.11+ installed on your device.
A Google API key with Gemini Live API access.

You need a Zero Runtime account to generate a token and a Google API key for the Gemini Live API. Generate a token from the Zero Runtime Dashboard and a Google API key from Google AI Studio.

Project Structure

First, create an empty project using mkdir folder_name in your preferred location for the JavaScript frontend. Your final project structure should look like this:

root

index.html

config.js

index.js

agent-js.py

.env

You will be working on the following files:

index.html: Creates a basic UI for joining the meeting.
config.js: Stores the token and room ID for the JavaScript frontend.
index.js: Renders the meeting view and handles audio functionality.
agent-js.py: The Python AI agent backend using the Google Gemini Live API.
.env: Environment variables for the Python agent’s API keys.

Part 1: JavaScript Frontend

Install Zero Runtime

Import Zero Runtime using the <script> tag or install it using npm/yarn.

<script>
npm
yarn

<html>
  <head>
    <!--.....-->
  </head>
  <body>
    <!--.....-->
    <script src="https://sdk.videosdk.live/js-sdk/0.3.6/videosdk.js"></script>
  </body>
</html>

npm install @videosdk.live/js-sdk

yarn add @videosdk.live/js-sdk

Design the user interface

Create an index.html file containing a join-screen and a grid-screen for audio-only interaction.

index.html

<!DOCTYPE html>
<html>
<head> </head>
<body>
    <div id="join-screen">
        <button id="createMeetingBtn">Join Agent Meeting</button>
    </div>
    <div id="textDiv"></div>
    <div id="grid-screen" style="display: none">
        <h3 id="meetingIdHeading"></h3>
        <button id="leaveBtn">Leave</button>
        <button id="toggleMicBtn">Toggle Mic</button>
        <div id="audioContainer"></div>
    </div>
    <script src="https://sdk.videosdk.live/js-sdk/0.3.6/videosdk.js"></script>
    <script src="config.js"></script>
    <script src="index.js"></script>
</body>
</html>

Configure environment and credentials

Create a meeting room using the Zero Runtime API:

curl -X POST https://api.videosdk.live/v2/rooms \
  -H "Authorization: YOUR_JWT_TOKEN_HERE" \
  -H "Content-Type: application/json"

Copy the roomId from the response and configure it in config.js:

config.js

TOKEN = "YOUR_ZRT_AUTH_TOKEN";
ROOM_ID = "YOUR_MEETING_ID"; // Static room ID shared between frontend and agent

Implement meeting logic

In index.js, retrieve DOM elements, declare variables, and add the core meeting functionality.

index.js

// getting Elements from Dom
const leaveButton = document.getElementById("leaveBtn");
const toggleMicButton = document.getElementById("toggleMicBtn");
const createButton = document.getElementById("createMeetingBtn");
const audioContainer = document.getElementById("audioContainer");
const textDiv = document.getElementById("textDiv");

// declare Variables
let meeting = null;
let meetingId = "";
let isMicOn = false;

// Join Agent Meeting Button Event Listener
createButton.addEventListener("click", async () => {
  document.getElementById("join-screen").style.display = "none";
  textDiv.textContent = "Please wait, we are joining the meeting";
  meetingId = ROOM_ID;
  initializeMeeting();
});

// Initialize meeting
function initializeMeeting() {
  window.VideoSDK.config(TOKEN);

  meeting = window.VideoSDK.initMeeting({
    meetingId: meetingId,
    name: "C.V.Raman",
    micEnabled: true,
    webcamEnabled: false,
  });

  meeting.join();

  meeting.localParticipant.on("stream-enabled", (stream) => {
    if (stream.kind === "audio") {
      setAudioTrack(stream, meeting.localParticipant, true);
    }
  });

  meeting.on("meeting-joined", () => {
    textDiv.textContent = null;
    document.getElementById("grid-screen").style.display = "block";
    document.getElementById("meetingIdHeading").textContent = `Meeting Id: ${meetingId}`;
  });

  meeting.on("meeting-left", () => {
    audioContainer.innerHTML = "";
  });

  meeting.on("participant-joined", (participant) => {
    let audioElement = createAudioElement(participant.id);
    participant.on("stream-enabled", (stream) => {
      if (stream.kind === "audio") {
        setAudioTrack(stream, participant, false);
        audioContainer.appendChild(audioElement);
      }
    });
  });

  meeting.on("participant-left", (participant) => {
    let aElement = document.getElementById(`a-${participant.id}`);
    if (aElement) aElement.remove();
  });
}

// Create audio elements for participants
function createAudioElement(pId) {
  let audioElement = document.createElement("audio");
  audioElement.setAttribute("autoPlay", "false");
  audioElement.setAttribute("playsInline", "true");
  audioElement.setAttribute("controls", "false");
  audioElement.setAttribute("id", `a-${pId}`);
  audioElement.style.display = "none";
  return audioElement;
}

// Set audio track
function setAudioTrack(stream, participant, isLocal) {
  if (stream.kind === "audio") {
    if (isLocal) {
      isMicOn = true;
    } else {
      const audioElement = document.getElementById(`a-${participant.id}`);
      if (audioElement) {
        const mediaStream = new MediaStream();
        mediaStream.addTrack(stream.track);
        audioElement.srcObject = mediaStream;
        audioElement.play().catch((err) => console.error("audioElem.play() failed", err));
      }
    }
  }
}

// Implement controls
leaveButton.addEventListener("click", async () => {
  meeting?.leave();
  document.getElementById("grid-screen").style.display = "none";
  document.getElementById("join-screen").style.display = "block";
});

toggleMicButton.addEventListener("click", async () => {
  if (isMicOn) meeting?.muteMic();
  else meeting?.unmuteMic();
  isMicOn = !isMicOn;
});

Part 2: Python AI Agent

Setup the project

Use Python 3.11 or higher, then install the Zero Runtime Agents SDK. Create and activate a virtual environment, then install zrt and python-dotenv. Every provider plugin ships with zrt.

uv venv --python 3.11
source .venv/bin/activate     # macOS/Linux
.\.venv\Scripts\activate      # Windows

python3.11 -m venv venv
source venv/bin/activate      # macOS/Linux
.\venv\Scripts\activate       # Windows

New to uv? See the uv install guide.

uv pip install zrt python-dotenv

pip install zrt python-dotenv

Configure environment variables

Create a .env file in your project root to store your keys and token securely for the Python agent:

.env

# Google API Key for the Gemini Live API
GOOGLE_API_KEY=your_google_api_key_here

# Option A: pre-generated ZRT auth token
ZRT_AUTH_TOKEN=your_zrt_auth_token_here

# Option B: SDK auto-generates the token (omit ZRT_AUTH_TOKEN)
# ZRT_API_KEY=your_zrt_api_key
# ZRT_SECRET_KEY=your_zrt_secret_key

Create the AI agent backend

Create the Python AI agent (agent-js.py) that joins the same meeting room and interacts with users through voice.

agent-js.py

import os
import zrt
from zrt import Agent, Pipeline, Room
from zrt.plugins import GeminiRealtime, GeminiLiveConfig
from zrt.inference import AICousticsDenoise
from dotenv import load_dotenv

load_dotenv()

AGENT_ID = "game-show-agent"

pipeline = Pipeline(
    llm=GeminiRealtime(
        api_key=os.getenv("GOOGLE_API_KEY"),  # optional when GOOGLE_API_KEY is in .env
        config=GeminiLiveConfig(model="gemini-3.1-flash-live-preview", voice="Leda", response_modalities=["AUDIO"]),
    ),
    denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)


class MyVoiceAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            name="MyVoiceAgent",
            agent_id=AGENT_ID,
            instructions="You are a high-energy game-show host guiding the caller to guess a secret number from 1 to 100 to win 1,000,000$.",
            pipeline=pipeline,
        )

    async def on_enter(self) -> None:
        await self.session.say(
            "Welcome to the Zero Runtime AI Agent game show! I'm your host, and we're about to play for 1,000,000$. Are you ready to play?"
        )

    async def on_exit(self) -> None:
        await self.session.say("Goodbye!")


def invoke_agent() -> None:
    # Join the same room the frontend created (its meeting ID).
    zrt.invoke(AGENT_ID, room=Room(room_id="YOUR_MEETING_ID"))


if __name__ == "__main__":
    zrt.serve(MyVoiceAgent, on_ready=invoke_agent)

Part 3: Run the Application

Start the frontend

Once you have completed all the steps above, serve your frontend files:

# Using Python's built-in server
python3 -m http.server 8000

# Or using the Node.js http-server
npx http-server -p 8000

Open http://localhost:8000 in your web browser.

Run the AI agent

Open a new terminal, activate your virtual environment, and run the Python agent. It reads GOOGLE_API_KEY from the .env file.

uv run python agent-js.py

python agent-js.py

Connect and interact

Join the meeting from the frontend:
- Click the Join Agent Meeting button in your browser.
- Allow microphone permissions when prompted.
Agent connection:
- Once you join, the Python agent detects your participation.
- You should see “Participant joined” in the terminal.
- The AI agent greets you and initiates the game.
Start playing:
- The agent guides you through a number guessing game (1–100).
- Use your microphone to interact with the AI host.

Troubleshooting

'Waiting for participant...' but no connection

Ensure both the frontend and the agent are running.
Check that the ROOM_ID matches in config.js and agent-js.py.
Verify your Zero Runtime token is valid.

Audio not working

Check browser permissions for microphone access.
Ensure your Google API key has Gemini Live API access enabled.

Agent not responding

Verify your GOOGLE_API_KEY is set in the .env file.
Check that the Gemini Live API is enabled in your Google Cloud Console.

Next Steps

Quickstart Example

Complete working example with source code.

Build Your First Voice Agent

Learn the Cascade and Realtime pipelines in depth.

Get started

Quickstarts

SDKs

Integrate a Voice Agent with JavaScript

Prerequisites

Project Structure

Part 1: JavaScript Frontend

Part 2: Python AI Agent

Part 3: Run the Application

Troubleshooting

Next Steps

Quickstart Example

Build Your First Voice Agent

​Prerequisites

​Project Structure

​Part 1: JavaScript Frontend

​Part 2: Python AI Agent

​Part 3: Run the Application

​Troubleshooting

​Next Steps

Quickstart Example

Build Your First Voice Agent

Prerequisites

Project Structure

Part 1: JavaScript Frontend

Part 2: Python AI Agent

Part 3: Run the Application

Troubleshooting

Next Steps