WalidAlHassan/Avery-ff5

LiveKit Agent

A voice AI project built with LiveKit Agents for Python and LiveKit Cloud. This project is designed to work with coding agents like Claude Code, Cursor, and Codex — see Coding agent support for setup tips.

This project was converted to code from the LiveKit Agent Builder. The code is identical to production deployments from the builder. Follow the steps below to make it your own and deploy it to LiveKit Cloud. once you do so, you can delete the version in the builder.

Next steps

Run and deploy your agent

Get your agent running locally and in production:

1. Run locally: Follow the Quickstart section below to set up your environment and test the agent
2. Deploy to production: See the Deploy to production section for deployment options and best practices

Quickstart

Get up and running so you can start customizing:

1. Install dependencies:

   uv sync
   

2. Set up your LiveKit credentials:

   Sign up for LiveKit Cloud, then configure your environment. You can either:

   • Manual setup: Copy .env.example to .env.local and fill in:

     • LIVEKIT_URL
     • LIVEKIT_API_KEY
     • LIVEKIT_API_SECRET
     • any provider API key listed in .env.example (realtime models are bring-your-own-key)

   • Automatic setup (recommended): Use the LiveKit CLI:

     lk cloud auth
     lk app env -w -d .env.local
     

3. Download required models:

   uv run python src/agent.py download-files
   

   This downloads the model files used by the audio enhancement and noise cancellation plugins. VAD and turn detection run in LiveKit Inference, so they need no local models.

4. Test your agent:

   uv run python src/agent.py console
   

   This lets you speak to your agent directly in your terminal.

5. Run for development:

   uv run python src/agent.py dev
   

   Use this when connecting to a frontend or telephony. This puts your agent into your LiveKit Cloud project, so use a different project if you don't want to affect production traffic.

Local Self-Hosted Development

This project can run entirely on your own machine, with a self-hosted LiveKit Server, and without a LiveKit Cloud account. The agent still talks to external AI providers (Deepgram, Google Gemini, Cartesia) directly, using your own API keys, instead of routing through LiveKit Inference.

Local LiveKit Server (livekit-server --dev)
        │  ws://localhost:7880
        ▼
LiveKit Agent Worker (src/agent.py)
        │
        ├── STT  → Deepgram API      (DEEPGRAM_API_KEY)
        ├── LLM  → Google Gemini API (GOOGLE_API_KEY)
        └── TTS  → Cartesia API      (CARTESIA_API_KEY)

1. Install LiveKit Server

macOS: brew update && brew install livekit Linux: curl -sSL https://get.livekit.io | bash Windows: download from the latest release page

Alternatively, if you'd rather not install anything system-wide, run it via Docker instead (no docker compose needed):

docker run --rm -d --name livekit-dev \
  -p 7880:7880 -p 7881:7881 \
  -p 50000-50100:50000-50100/udp \
  livekit/livekit-server --dev --bind 0.0.0.0

• The UDP range is narrowed to 50000-50100 (instead of the full 50000-60000) to avoid clashing with other apps on your machine (VPNs, remote-desktop tools, etc.) that may already be using a port somewhere in that range.
• --bind 0.0.0.0 is required in Docker: --dev alone binds the server to the container's loopback address only, which Docker's port-forwarding can't reach, causing connection resets from the host.

2. Start LiveKit Server

livekit-server --dev

This starts a signaling server at ws://localhost:7880 with fixed dev credentials (devkey / secret) — no cloud account involved.

3. Configure .env.local

Copy .env.example to .env.local and fill in your own Deepgram, Google, and Cartesia API keys. The LiveKit values already default to the local dev server:

cp .env.example .env.local

4. Install Python dependencies

uv sync

5. Download local model files

VAD and turn detection run locally (not via LiveKit Inference), so their model files need to be downloaded once:

uv run python src/agent.py download-files

6. Start the agent

uv run python src/agent.py dev

With LIVEKIT_URL=ws://localhost:7880 in .env.local, the worker registers with your local server instead of LiveKit Cloud.

7. Connect a client

The agent worker only joins rooms — something still needs to create a room and connect a participant to it. Use uv run python src/agent.py console for a quick terminal-based mic/speaker test without any server or frontend, or point one of the frontend starter templates at ws://localhost:7880 with the local devkey/secret to test a full room-based session.

Troubleshooting

• docker: ... ports are not available / bind: address already in use: another app on your machine already holds a port inside the UDP range you asked Docker to publish. Narrow the range (e.g. 50000-50100) instead of publishing the full 50000-60000.
• Console mode connects but the server resets the connection: if running LiveKit Server in Docker, make sure --bind 0.0.0.0 is passed — otherwise the server only listens on the container's loopback address, which Docker's port-forwarding can't reach.
• console mode fails with PortAudioError: Invalid sample rate: your OS's default audio input device is a raw ALSA hardware device that only supports its native sample rate (no resampling), while LiveKit needs a different rate. List devices and pick your system's audio server device (e.g. pipewire or pulse) instead, which resamples automatically:

  uv run python src/agent.py console --list-devices
  uv run python src/agent.py console --input-device <id> --output-device <id>
  

  --input-device/--output-device are optional — omit them to use the system default, or pass device IDs from --list-devices if the default fails (e.g. uv run python src/agent.py console --input-device 7 --output-device 7 for a pipewire device).

What's not covered here

• ai-coustics noise cancellation is billed either through a LiveKit Cloud project or your own ai-coustics license. Locally, it's skipped unless you set AI_COUSTICS_LICENSE_KEY in .env.local.
• This setup is for local development only — no TLS, TURN, load balancing, or production hardening. See Deploy to production for that.

Customize your agent

Once your agent is running, enhance it for your use case:

• Customize AI models: Your agent uses a voice AI pipeline built on LiveKit Inference. More than 50 model providers are supported, including Realtime models.

• Add tests: You can add a full test suite to your agent. See the testing documentation for more information.

• Build reliable workflows: For complex agents, use tasks and handoffs instead of long instruction prompts. This minimizes latency and improves reliability by structuring your agent into focused, reusable components.

Get help from AI coding assistants

Supercharge your development with AI coding assistants that understand LiveKit. This project works seamlessly with Claude Code, Cursor, Codex, and other AI coding tools.

For your convenience, LiveKit offers both a CLI and an MCP server that can be used to browse and search its documentation. The LiveKit CLI (lk docs) works with any coding agent that can run shell commands. Install it for your platform:

macOS:

brew install livekit-cli

Linux:

curl -sSL https://get.livekit.io/cli | bash

Windows:

winget install LiveKit.LiveKitCLI

The lk docs subcommand requires version 2.15.0 or higher. Check your version with lk --version and update if needed. Once installed, your coding agent can search and browse LiveKit documentation directly from the terminal:

lk docs search "voice agents"
lk docs get-page /agents/start/voice-ai-quickstart

See the Using coding agents guide for more details, including MCP server setup.

Customize the AI assistant context: The project includes an AGENTS.md file that guides AI assistants on how to work with this codebase. Edit this file to add your own project-specific context, patterns, and preferences. Learn more at https://agents.md.

Frontend development

If you don't alread have a frontend, use the following templates and guides to get started on one:

Platform	Starter Template	What to customize
Web	livekit-examples/agent-starter-react	React & Next.js—customize UI, add features, integrate with your backend
iOS/macOS	livekit-examples/agent-starter-swift	Native apps for iOS, macOS, visionOS—add platform-specific features
Flutter	livekit-examples/agent-starter-flutter	Cross-platform—customize for Android, iOS, web, desktop
React Native	livekit-examples/voice-assistant-react-native	Mobile with Expo—add native modules, customize navigation
Android	livekit-examples/agent-starter-android	Kotlin & Jetpack Compose—build Material Design UI
Web Embed	livekit-examples/agent-starter-embed	Widget for any website—customize styling, add to your site
Telephony	Documentation	Add phone calling—configure SIP, add call routing, customize prompts

Observability

LiveKit provides deep session insights for your agents through Agent Observability. Monitor conversation quality, track latency metrics, and debug agent behavior in production.

Deploy to production

To deploy your agent to production, you can use the LiveKit CLI:

lk agent create

See the deploying to production guide for detailed instructions and optimization tips.

Join the LiveKit community

Join the LiveKit Slack Community to get help from the LiveKit team and other developers.