Index Agentica

LLM gateways compared: OpenRouter vs Vercel AI Gateway vs Cloudflare AI Gateway vs Portkey vs LiteLLM

Five ways to put one endpoint between your agents and many model providers: two hosted model marketplaces (OpenRouter, Vercel AI Gateway), Cloudflare's proxy, and two open-source gateways you can self-host (Portkey, LiteLLM). Covers API, fallback, caching, spend controls, MCP, pricing and license.

Type
Comparison
Author
Agentica Author
Published
Last verified

Comparison

LLM gateways compared: OpenRouter vs Vercel AI Gateway vs Cloudflare AI Gateway vs Portkey vs LiteLLM
EntryWhat it isRun it whereAPIRouting and fallbackCachingSpend and access controlsMCP toolsPricing (2026-10-25)License
OpenRouterHosted marketplace API: 500+ models from 80+ providers behind one key, billed from prepaid creditsOpenRouter-hosted only; eu.openrouter.ai and us.openrouter.ai keep inference in-region on Business and EnterpriseOpenAI-compatible (change base URL and model names); tool calling passes through when the model supports itmodels array tries the next model on error; openrouter/auto auto-routing; provider fallback on rate limits. Failed attempts are not billed (Zero Completion Insurance)Provider prompt caching on Standard and above; no response cache documented on the pricing pageBudgets and spend controls on Standard and above; per-key credit caps; workspaces (5, or 1,000 on Business); SSO/SAML on EnterpriseNot part of the gateway pages we checkedTokens at provider list price. Fee on credit purchases: 5.5% Standard, 8% Business. BYOK free up to $25,000/month of list-price inference, then 5%. Free tier: 25+ free models, 50 requests/dayProprietary
Vercel AI GatewayVercel's managed gateway for calling models across providers from any infrastructure (your app need not run on Vercel)Vercel-hosted; authenticate with an AI Gateway API key, or OIDC from Vercel deploymentsAI SDK, OpenAI Chat Completions, OpenAI Responses and Anthropic Messages formats at ai-gateway.vercel.shRoutes a model across healthy providers; ordered provider and model fallbacks; routing rules; request logs record every routing attempt'Automatic caching' listed under Models & ProvidersBudgets per team, project, API key or member (cover system-credential spend only, not BYOK); request logs; paid add-ons for team-wide provider allowlist and ZDRNot part of the gateway pages we checkedNo markup or platform fee on tokens, including BYOK; prepaid credits; monthly free credit limited to a subset of models with lower rate limits. BYOK needs purchased credits. Add-ons: Custom Reporting $0.075/1,000 writes and $5/1,000 queries; team-wide ZDR or provider allowlist $0.10/1,000 requestsProprietary
Cloudflare AI GatewayProxy in front of your AI providers that adds analytics, logs, caching, rate limiting, retries and model fallbackCloudflare's network; available on all plans, created automatically on first request to the default gatewayProvider-native endpoints through the gateway, plus an OpenAI-compatible REST API (/ai/v1/chat/completions); the older /compat endpoint is deprecated for single-model callsRequest retry and model fallback via dynamic routesResponse caching served from Cloudflare's cache (core feature)Rate limiting (core feature); free DLP scanning; Guardrails using Llama Guard on Workers AINot part of the gateway pages we checkedCore features (analytics, caching, rate limiting) free. Logs: Workers Logs pricing for accounts whose first gateway dates from 2026-09-24 on, else Legacy Logs limits. Unified Billing: 5% fee on credits, tokens without markup. Guardrails billed as Workers AI inferenceProprietary
Portkey AI GatewayOpen-source gateway (npx @portkey-ai/gateway) plus a hosted platform with logs, guardrails and prompt management. Palo Alto Networks completed its acquisition of Portkey and is integrating it as Prisma AIRS AI GatewaySelf-host (Node; local on :8787 with a console), Portkey-hosted SaaS, or enterprise private-cloud/VPC deploymentsOpenAI-compatible client; provider chosen per request or through configsConfigs: retries, fallbacks, load balancing, conditional routing, timeouts (all in the open-source gateway)Simple caching on the hosted Developer plan; simple and semantic caching on ProductionVirtual keys with budgeting; RBAC and service-account keys on Production; granular budgets, rate limits and SSO on Enterprise; guardrailsYes: Portkey MCP Gateway for central auth, per-server/per-tool access, identity forwarding and tool-call logsOpen source free. Developer free (10k recorded logs/month, 3-day log retention). Production $49/month (100k logs, +$9 per extra 100k up to 3M). Enterprise customMIT (open-source gateway)
LiteLLM MCP GatewayLiteLLM AI Gateway (the LiteLLM Proxy): self-hosted server with one OpenAI-compatible endpoint for 100+ providers, MCP tools and A2A agents. Listed here as LiteLLM MCP GatewaySelf-hosted only: Docker, Helm or Terraform with a Postgres database; Admin UI at /uiOpenAI format for every provider; existing OpenAI clients work by changing base URL and keyLoad balancing, fallbacks and retries across deploymentsRedis/Valkey, semantic, and S3/GCS cachesVirtual keys per user, team or project, each with models, budgets and rate limits; spend tracking by key, team, user and tag; guardrails; SSO for the Admin UIYes: MCP Gateway on the same keys and spend records (HTTP, SSE and stdio servers; REST tool API)Free to self-host; you pay providers directly and run the infrastructure. Enterprise features are under a separate licenseMIT, except the enterprise/ directory

Verdict: Want hundreds of models on one key with no infrastructure? OpenRouter or Vercel AI Gateway. Both charge provider list prices: OpenRouter takes a fee when you buy credits, and Vercel charges for add-ons. Already on Cloudflare and mostly need caching, rate limits and logs in front of your own provider keys? Cloudflare AI Gateway's core features are free. Need to self-host with per-team virtual keys and budgets, and want MCP tools governed by the same keys? LiteLLM, or Portkey's open-source gateway with its hosted or enterprise tiers.

An LLM gateway sits between your agents and the model providers. Your code talks to one endpoint, and the gateway handles keys, routing, retries, caching, logging and spend limits. For agents that loop dozens of times per task, it is where you stop runaway costs and survive a provider outage without code changes.

This page is about model traffic. To put one controlled endpoint in front of many MCP tool servers, see MCP gateways compared. Two of the gateways here (Portkey and LiteLLM) now do both.

Three kinds of gateway

flowchart LR
  A((agent)) --> M[Hosted marketplace<br/>OpenRouter, Vercel]
  M -- their accounts or your BYOK --> P1[(providers)]
  A --> C[Proxy in front of your keys<br/>Cloudflare AI Gateway]
  C -- your keys or Unified Billing --> P2[(providers)]
  A --> S[Self-hosted gateway<br/>LiteLLM, Portkey OSS]
  S -- your keys --> P3[(providers)]
Mermaid diagram (source shown; not rendered client-side)

What matters for agents

Fallback that doesn't bill you twice. Agent loops hit rate limits. OpenRouter retries other models from a models array and says failed attempts aren't charged. Vercel records every routing attempt in its request logs. Cloudflare, Portkey and LiteLLM all have configurable retries and fallbacks.

Hard spend caps. Budgets come in different shapes. Vercel budgets apply per team, project, API key or member, but only to spend on Vercel's own credentials: BYOK spend doesn't count, and a failed BYOK request can fall back to system credentials. LiteLLM and Portkey budgets attach to virtual keys, so you can give each agent or user its own capped key. OpenRouter allows a credit cap per API key. Cloudflare's documented control is rate limiting rather than budgets.

Caching. Cloudflare, Portkey and LiteLLM can return cached responses for identical requests, and Portkey (Production) and LiteLLM also offer semantic caching. Be careful with caching for agents: a cached tool-planning step can replay a stale decision. Scope caches to read-only, deterministic calls.

One key layer for models and tools. LiteLLM's MCP gateway and Portkey's MCP Gateway put tool calls behind the same keys, logs and access rules as model calls. Want that without adopting either? Kong AI Gateway has an AI MCP Proxy plugin, which needs Kong's AI license. Or bridge stdio servers yourself (see Bridge stdio and remote MCP servers).

Observability. Every gateway here logs requests. For traces of whole agent runs, evals and prompt management, pair it with a tracing tool from Agent observability platforms. Helicone, also listed in the directory, combines a gateway with observability.

Ownership and licensing notes

Picking models

A gateway only helps if you route to the right model. OpenRouter Rankings shows real usage by model, and models.dev lists model specs and prices across providers. Re-check prices on the gateway's own model list before you set budgets.

Directory entries in this comparison

Related

Sources

Machine-readable