LiveKit Agents vs Pipecat vs OpenAI Realtime vs ElevenAgents vs Vapi: building voice agents

Five ways to give an agent a voice compared: two open-source realtime frameworks (LiveKit Agents, Pipecat), a speech-to-speech model API (OpenAI Realtime) and two hosted voice-agent platforms (ElevenAgents, Vapi). Covers architecture, hosting, telephony, MCP tools, testing, pricing and license.

Type

Comparison

Author

Agentica Author

Published

2026-10-23

Last verified

2026-10-23

• voice
• realtime
• telephony
• speech-to-speech
• webrtc
• mcp
• agents

Comparison

Verdict: Use LiveKit Agents or Pipecat if you want to own the code and the deployment, swap STT/LLM/TTS vendors freely and self-host; LiveKit bundles the media server and a free cloud tier, Pipecat has the widest telephony-carrier list and composable multi-agent pipelines. Use the OpenAI Realtime API (directly or as a plugin inside either framework) when one speech-to-speech model with tool calling is enough. Use ElevenAgents or Vapi when you want a hosted voice agent on a phone number quickly, with testing and analytics included, and accept per-minute platform fees.

A voice agent is an ordinary tool-using agent with two hard extras: audio has to move in real time (WebRTC, WebSocket or a phone line), and the agent has to know when the person has stopped talking. The five options here split along two questions.

Do you want a framework, a model API or a platform?

• Frameworks you run: LiveKit Agents and Pipecat. Both are open source and vendor-neutral. You write Python, pick providers for each stage and deploy wherever you like. Each has an optional managed cloud (LiveKit Cloud, Pipecat Cloud by Daily).
• A model API: the OpenAI Realtime API is one speech-to-speech model you call over WebRTC or WebSocket. You still host the app, mint ephemeral keys and run your own tool backend. Both frameworks can also use it as their model.
• Hosted platforms: ElevenAgents and Vapi. You configure an assistant in a dashboard or API, attach a phone number and pay per minute. Testing, analytics and telephony are built in.

Cascaded pipeline or speech-to-speech?

A cascaded agent runs speech-to-text, then an LLM, then text-to-speech. You can pick the best vendor for each stage, read every transcript and use any text LLM, but each hop adds delay. A speech-to-speech model (OpenAI Realtime, Eleven Realtime, the realtime plugins in LiveKit and Pipecat) handles audio end to end, which removes the hand-offs between stages, but you are tied to that model's voices and pricing. LiveKit, Pipecat and ElevenAgents support both. Vapi's docs and pricing are built around a transcriber, a model and a voice (cascaded), and OpenAI Realtime voice sessions are speech-to-speech.

flowchart LR
  caller((Caller)) -- WebRTC / SIP --> T[Transport]
  T --> STT --> LLM --> TTS --> T
  T -. speech-to-speech .-> S2S[Realtime model] -.-> T
  LLM -- tool calls --> MCP[(MCP servers)]
  S2S -- tool calls --> MCP

Tools: all five speak MCP

Every option can now call tools on a Model Context Protocol server, but in different ways:

• LiveKit and Pipecat connect from your agent process. Pipecat's MCPClient turns a server's tools into a ToolsSchema for the LLM, with filters for which tools and outputs to expose.
• OpenAI Realtime connects for you when you add an mcp tool with a server_url. Use a function tool instead when your own code should run the action or check approvals.
• Vapi imports every tool the server exposes at the start of each call and opens a new MCP session for each tool call. Its docs warn that large tool lists add latency and timeout risk, so point it at a narrowly scoped server (for example a Zapier MCP or Composio server limited to a few actions).
• ElevenAgents attaches MCP servers as agent tools and publishes a security page for third-party servers.

On a live call a slow tool means silence. Keep tool responses small and fast, and put long jobs behind a quick acknowledgement.

Cost

Per-minute platform fees stack on top of model costs. As of 2026-10-23: Vapi charges $0.05/min hosting plus provider costs, ElevenAgents $0.08 per additional minute plus LLM and telephony at cost, Pipecat Cloud $0.01 per active agent minute (agent-1x) with providers billed separately, and LiveKit Cloud includes 1,000 agent session minutes free, then $0.01/min on paid plans. OpenAI Realtime bills by audio and text tokens, so idle open sessions and long system prompts cost money. OpenAI's cost guide recommends closing idle sessions. Check the pricing pages before you commit, because these numbers change often.

Caveats

• Latency and accuracy numbers on vendor pages (time to first audio, turn-taking scores) are vendor claims. Measure them on your own calls.
• Voice adds new prompt-injection paths: a caller can say instructions out loud, and tool results get read back. The guide Test an agent for prompt injection applies here too.
• Phone numbers, call recording and data retention bring legal duties (consent, HIPAA, data residency). The plans differ in retention and in zero-data-retention (ZDR) options.

Directory entries in this comparison

• LiveKit Agents: Open-source Python (and JS) framework for realtime voice and video agents over WebRTC, with STT/LLM/TTS plugins, telephony, turn detection, MCP tools and tests.
• Pipecat: Open-source Python framework from Daily for realtime voice and multimodal agents: composable pipelines, 80+ AI service integrations, multi-agent handoffs and client SDKs.
• OpenAI Realtime API: OpenAI's speech-to-speech API for low-latency voice agents over WebRTC, WebSocket or SIP, with tool calling, MCP servers, interruptions and handoffs.
• ElevenAgents (ElevenLabs Agents Platform): ElevenLabs' hosted platform for voice and multimodal agents: ASR, any LLM, 10,000+ voices and a turn-taking model, with workflows, tools, telephony, tests and evals.
• Vapi: Hosted voice-agent API: build phone and web voice assistants that combine your chosen STT, LLM and TTS providers, with tools, MCP servers, telephony and testing.
• OpenAI Agents SDK: Lightweight framework from OpenAI for multi-agent workflows in Python (with a JS/TS sibling SDK).
• Model Context Protocol (MCP): Open protocol that standardizes how AI applications connect to external tools, data sources and prompts via MCP servers.
• Zapier MCP: Hosted MCP server connecting MCP clients to 9,000+ app integrations with OAuth handling and dynamic tool discovery.
• Composio: Tool platform for agents: 1,000+ toolkits with tool search, delegated authentication and sandboxed execution.

Related

• Terminal coding agents compared (Comparison)
• Connect an agent to a remote MCP server (Guide)
• Evaluate an AI agent with open-source eval and observability tools (Guide)
• MCP gateways compared (Comparison)
• Give your coding agent up-to-date library docs over MCP (Context7, GitMCP, DeepWiki, Microsoft Learn) (Guide)

Sources

• LiveKit Agents README, accessed 2026-10-23
• LiveKit pricing, accessed 2026-10-23
• LiveKit Agents docs, accessed 2026-10-23
• Pipecat README, accessed 2026-10-23
• Pipecat docs index (llms.txt), accessed 2026-10-23
• Pipecat Cloud pricing, accessed 2026-10-23
• OpenAI: Getting started with the Realtime API, accessed 2026-10-23
• OpenAI: Realtime with tools, accessed 2026-10-23
• OpenAI: Voice cost optimization, accessed 2026-10-23
• ElevenAgents overview, accessed 2026-10-23
• ElevenAgents pricing, accessed 2026-10-23
• ElevenLabs docs index (llms.txt), accessed 2026-10-23
• Vapi pricing, accessed 2026-10-23
• Vapi docs: MCP tools, accessed 2026-10-23

Machine-readable

• JSON
• Markdown
• Source on GitHub