AI Voice Agent with iOS

Zero Runtime lets you integrate AI agents with real-time voice interaction into your native iOS app within minutes. In this quickstart, you’ll create an AI agent that joins a meeting room and interacts with users through voice using the Google Gemini Live API.

Prerequisites

Before proceeding, ensure that your development environment meets the following requirements:

A Zero Runtime account (create one from the Zero Runtime Dashboard).
iOS 13.0+, Xcode 13.0+, and Swift 5.0+.
Python 3.11+ installed on your device.

You need a Zero Runtime account to generate a token. The agent reaches the Gemini Live API through the Zero Runtime Inference Gateway, so you don’t need a separate Google API key. Generate a token from the Zero Runtime Dashboard.

Project Structure

Your final Xcode project structure should look like this:

zrt-agents-quickstart-ios

JoinScreenView.swift

MeetingView.swift

MeetingViewController.swift

RoomsStruct.swift

zrt_agents_quickstart_iosApp.swift

zrt-agents-quickstart-ios.xcodeproj

agent-ios.py

.env

You will be working on the following files:

JoinScreenView.swift: Join screen UI.
MeetingView.swift: Meeting interface with audio controls.
MeetingViewController.swift: Handles meeting logic and events.
agent-ios.py: The Python AI agent backend using the Google Gemini Live API.
.env: Environment variables for the Python agent’s API keys.

1. iOS Frontend

Step 1: Create the app and install the RTC SDK

Create a new iOS app in Xcode:

Create a new Xcode project.
Choose the App template.
Add a Product Name and save the project.

Install the RTC SDK using Swift Package Manager:

In Xcode, go to File > Add Packages...
Enter the repository URL: https://github.com/videosdk-live/videosdk-rtc-ios-sdk.git
Choose the latest version and click Add Package.

Add microphone and camera permissions to Info.plist:

Info.plist

<key>NSCameraUsageDescription</key>
<string>Camera permission description</string>
<key>NSMicrophoneUsageDescription</key>
<string>Microphone permission description</string>

Step 2: Create models and views

Create the Swift models and views for the meeting interface.

RoomsStruct.swift

struct RoomsStruct: Codable {
    let createdAt, updatedAt, roomID: String?
    let links: Links?
    let id: String?

    enum CodingKeys: String, CodingKey {
        case createdAt, updatedAt
        case roomID = "roomId"
        case links, id
    }
}

struct Links: Codable {
   let getRoom, getSession: String?

    enum CodingKeys: String, CodingKey {
        case getRoom = "get_room"
        case getSession = "get_session"
    }
}

JoinScreenView.swift

import SwiftUI

struct JoinScreenView: View {

    // State variables for
    let meetingId: String = "YOUR_MEETING_ID"
    @State var name: String

    var body: some View {
        NavigationView {
            VStack {
                Text("Zero Runtime")
                    .font(.largeTitle)
                    .fontWeight(.bold)
                Text("AI Agent Quickstart")
                    .font(.largeTitle)
                    .fontWeight(.semibold)
                    .padding(.bottom)

                TextField("Enter Your Name", text: $name)
                    .foregroundColor(Color.black)
                    .autocorrectionDisabled()
                    .font(.headline)
                    .overlay(
                        Image(systemName: "xmark.circle.fill")
                            .padding()
                            .offset(x: 10)
                            .foregroundColor(Color.gray)
                            .opacity(name.isEmpty ? 0.0 : 1.0)
                            .onTapGesture {
                                UIApplication.shared.endEditing()
                                name = ""
                            }
                        , alignment: .trailing)
                    .padding()
                    .background(
                        RoundedRectangle(cornerRadius: 25)
                            .fill(Color.secondary.opacity(0.5))
                            .shadow(color: Color.gray.opacity(0.10), radius: 10))
                    .padding()

                NavigationLink(destination: MeetingView(meetingId: self.meetingId, userName: name ?? "Guest")
                    .navigationBarBackButtonHidden(true)) {
                        Text("Join Meeting")
                            .foregroundColor(Color.white)
                            .padding()
                            .background(
                                RoundedRectangle(cornerRadius: 25.0)
                                    .fill(Color.blue))
                    }
            }
        }
    }
}

extension UIApplication {

    func endEditing() {
        sendAction(#selector(UIResponder.resignFirstResponder), to: nil, from: nil, for: nil)
    }
}

MeetingView.swift

import SwiftUI
import VideoSDKRTC

struct MeetingView: View{

@Environment(\.presentationMode) var presentationMode

@ObservedObject var meetingViewController = MeetingViewController()
@State var meetingId: String?
@State var userName: String?
@State var isUnMute: Bool = true

var body: some View {
    VStack {
        if meetingViewController.participants.count == 0 {
            Text("Meeting Initializing")
        } else {
            VStack {
                VStack(spacing: 20) {
                    Text("Meeting ID: \(meetingViewController.meetingID)")
                        .padding(.vertical)

                    List {
                        ForEach(meetingViewController.participants.indices, id: \.self) { index in
                            Text("Participant Name: \(meetingViewController.participants[index].displayName)")
                        }
                    }
                }

                VStack {
                    HStack(spacing: 15) {
                        // mic button
                        Button {
                            if isUnMute {
                                isUnMute = false
                                meetingViewController.meeting?.muteMic()
                            }
                            else {
                                isUnMute = true
                                meetingViewController.meeting?.unmuteMic()
                            }
                        } label: {
                            Text("Toggle Mic")
                                .foregroundStyle(Color.white)
                                .font(.caption)
                                .padding()
                                .background(
                                    RoundedRectangle(cornerRadius: 25)
                                        .fill(Color.blue))
                        }
                        // end meeting button
                        Button {
                            meetingViewController.meeting?.end()
                            presentationMode.wrappedValue.dismiss()
                        } label: {
                            Text("End Call")
                                .foregroundStyle(Color.white)
                                .font(.caption)
                                .padding()
                                .background(
                                    RoundedRectangle(cornerRadius: 25)
                                        .fill(Color.red))
                        }
                    }
                    .padding(.bottom)
                }
            }
        }
    }
    .onAppear() {
        /// MARK :- configuring the videoSDK
        VideoSDK.config(token: meetingViewController.token)
        print(meetingId)
        if meetingId?.isEmpty == false {
            print("i ff meeting isd is emty \(meetingId)")
            // join an existing meeting with provided meeting Id
            meetingViewController.joinMeeting(meetingId: meetingId!, userName: userName!)
        }
    }
}
}

Step 3: Implement meeting logic

Create the main meeting view controller to handle meeting events.

MeetingViewController.swift

import Foundation
import VideoSDKRTC

class MeetingViewController: ObservableObject {

    var token = "YOUR_ZERORUNTIME_AUTH_TOKEN" // Add Your token here
    var meetingId: String = ""
    var name: String = ""

    @Published var meeting: Meeting? = nil
    @Published var participants: [Participant] = []
    @Published var meetingID: String = ""

    func initializeMeeting(meetingId: String, userName: String) {
        meeting = VideoSDK.initMeeting(
            meetingId: meetingId,
            participantName: userName,
            micEnabled: true,
            webcamEnabled: false
        )

        meeting?.addEventListener(self)
        meeting?.join()
    }
}

extension MeetingViewController: MeetingEventListener {

    func onMeetingJoined() {
        guard let localParticipant = self.meeting?.localParticipant else { return }

        // add to list
        participants.append(localParticipant)

        localParticipant.addEventListener(self)

    }

    func onParticipantJoined(_ participant: Participant) {
        participants.append(participant)

        // add listener
        participant.addEventListener(self)

    }

    func onParticipantLeft(_ participant: Participant) {
        participants = participants.filter({ $0.id != participant.id })
    }

    func onMeetingLeft() {
        meeting?.localParticipant.removeEventListener(self)
        meeting?.removeEventListener(self)
    }

    func onMeetingStateChanged(meetingState: MeetingState) {
        switch meetingState {
        case .DISCONNECTED:
            participants.removeAll()
        default:
            print("")
        }
    }
}

extension MeetingViewController: ParticipantEventListener {

}

extension MeetingViewController {

    func joinMeeting(meetingId: String, userName: String) {
        if !token.isEmpty {
            self.meetingID = meetingId
            self.initializeMeeting(meetingId: meetingId, userName: userName)
        }
        else {
            print("Auth token required")
        }
    }
}

Step 4: Configure the app entry point

Configure the main app entry point.

zrt_agents_quickstart_iosApp.swift

import SwiftUI

@main
struct zrt_agents_quickstart_iosApp: App {
    var body: some Scene {
        WindowGroup {
            JoinScreenView(name: "")
        }
    }
}

Step 5: Configure environment and credentials

Create a meeting room using the Zero Runtime API:

curl -X POST https://api.videosdk.live/v2/rooms \
  -H "Authorization: YOUR_JWT_TOKEN_HERE" \
  -H "Content-Type: application/json"

Copy the roomId from the response and set it as the meetingId in JoinScreenView.swift, and set your token in MeetingViewController.swift (YOUR_ZERORUNTIME_AUTH_TOKEN).

2. Python AI Agent

Step 1: Setup the project

Use Python 3.11 or higher, then install the Zero Runtime Agents SDK. Create and activate a virtual environment, then install zrt and python-dotenv. Every provider plugin ships with zrt.

uv venv --python 3.11
source .venv/bin/activate     # macOS/Linux
.\.venv\Scripts\activate      # Windows

python3.11 -m venv venv
source venv/bin/activate      # macOS/Linux
.\venv\Scripts\activate       # Windows

New to uv? See the uv install guide.

uv pip install zeroruntime python-dotenv

pip install zeroruntime python-dotenv

Step 2: Configure environment variables

Create a .env file in your project root to store your token securely for the Python agent:

.env

# pre-generated Zero Runtime auth token
ZERORUNTIME_AUTH_TOKEN=your_zrt_auth_token_here

Step 3: Create the AI agent backend

Create the Python AI agent (agent-ios.py) that joins the same meeting room and interacts with users through voice.

agent-ios.py

import zeroruntime
from zeroruntime import Agent, Pipeline, Room
from zeroruntime.inference import AICousticsDenoise, GeminiRealtime
from dotenv import load_dotenv

load_dotenv()

pipeline = Pipeline(
    # Realtime speech-to-speech model goes in the pipeline's llm slot -
    # no separate STT or TTS needed.
    llm=GeminiRealtime(
        model="gemini-3.1-flash-live-preview",
        config={
            "voice": "Leda",  # Puck, Charon, Kore, Fenrir, Aoede, Leda, Orus, Zephyr
            "response_modalities": ["AUDIO"],
        },
    ),
    denoise=AICousticsDenoise(model_id="quail-vf-2.2-l-16khz"),
)


class MyVoiceAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            name="MyVoiceAgent",
            instructions="You are a high-energy game-show host guiding the caller to guess a secret number from 1 to 100 to win 1,000,000$.",
            pipeline=pipeline,
        )

    async def on_enter(self) -> None:
        await self.session.say(
            "Welcome to the Zero Runtime AI Agent game show! I'm your host, and we're about to play for 1,000,000$. Are you ready to play?"
        )

    async def on_exit(self) -> None:
        await self.session.say("Goodbye!")


def invoke_agent() -> None:
    # Join the same room the frontend created (its meeting ID).
    zeroruntime.invoke(room=Room(room_id="YOUR_MEETING_ID"))


if __name__ == "__main__":
    zeroruntime.serve(MyVoiceAgent, on_ready=invoke_agent)

3. Run the Application

Step 1: Run the iOS app

Build and run the app from Xcode on a simulator or physical device.

Step 2: Run the AI agent

Open a new terminal, activate your virtual environment, and run the Python agent. It reads ZERORUNTIME_AUTH_TOKEN from the .env file.

uv run python agent-ios.py

python agent-ios.py

Step 3: Connect and interact

Run the iOS app on a simulator or device.
Join the meeting and allow microphone permissions.
When you join, the Python agent detects your participation and starts speaking.
Talk to the agent in real time and play the number guessing game.

Troubleshooting

'Waiting for participant...' but no connection

Ensure the same room_id is set in both the iOS app (JoinScreenView.swift) and the agent’s Room(room_id=...).
Verify microphone permissions in iOS Settings > Privacy & Security > Microphone.
For simulator issues, ensure you’re using a physical device for microphone testing.

Audio not working

Check that microphone permissions were granted when prompted.

Agent not responding

Confirm your Zero Runtime token is valid and ZERORUNTIME_AUTH_TOKEN is set in the .env file.

Overview

Build

Plugins

Deploy

AI Voice Agent with iOS

Prerequisites

Project Structure

1. iOS Frontend

2. Python AI Agent

3. Run the Application

Troubleshooting

Next Steps

Quickstart Example

Build Your First Voice Agent

​Prerequisites

​Project Structure

​1. iOS Frontend

​2. Python AI Agent

​3. Run the Application

​Troubleshooting

​Next Steps

Quickstart Example

Build Your First Voice Agent

Prerequisites

Project Structure

1. iOS Frontend

2. Python AI Agent

3. Run the Application

Troubleshooting

Next Steps