LLM Gateway Pricing Is Where Teams Get Quietly Wrecked

A team picks an LLM gateway when traffic is tiny. The integration is a base URL change. The pricing page says something like "5%" or "$49/month," so the decision gets made quickly and everyone ships.

Then the feature works. Token spend goes up, which is expected. But the gateway line item goes up too, sometimes far more than expected. Finance asks why the "proxy" is taking a bigger cut than the logging stack. What looked like a flat fee during evaluation turns out to be a percentage that compounds with every unit of success, and a vendor decision made at low volume ends up getting re-litigated months later.

Most of the "hidden costs" in LLM gateways aren't actually hidden. They're just easy to misread, because vendors mix token spend, platform fees, and request-based quotas under the same "one API, many models" pitch. What follows is the pricing behavior that matters in production: the four pricing models that keep showing up, the failure modes that trigger surprise bills, and the math worth doing before signing anything.

TL;DR

• Percentage-of-token-spend pricing is a growth tax. It scales forever and usually has no ceiling.
• Flat platform fee plus zero token markup is the least surprising model long-term. Token spend grows; the gateway fee doesn't automatically compound with it.
• Request- or seat-volume SaaS tiers live or die on the definition of "request." Agent loops, retries, and embedding pipelines can blow through quotas long before token spend looks large.
• "Free" self-hosted gateways aren't free. They move the cost into on-call, upgrades, scaling, and incident response.
• Price at ten times current volume before committing to anything. The evaluation period is the worst possible time to judge a pricing model.

The four pricing models teams actually run into

Model 1: Percentage markup on token spend (and token-adjacent fees)

This is the easiest model to underestimate because it looks harmless at low spend. A gateway takes what the underlying provider charges and adds a percentage on top. Some vendors advertise "no markup" while charging percentage fees on adjacent surfaces instead, funding credits or BYOK overages. Different label, same behavior: the gateway bill rises in lockstep with model spend.

Requesty runs a flat 5% markup on model cost, covering routing, caching, failover, guardrails, and analytics, dropping to 0% with bring-your-own-key if the customer holds its own provider contracts. OpenRouter states no markup on inference pricing itself, but charges 5.5% on card-funded credit purchases, 5% on crypto purchases, and 5% on bring-your-own-key usage above the plan's free monthly allowance.

The production behavior that matters: there's no natural ceiling. If a product succeeds, the gateway takes a larger absolute cut every month, and leaner prompts don't fix it, since success increases total volume regardless of per-call efficiency. At $2,000 a month in token spend, a 5% fee is $100 and nobody notices. At $80,000 a month, it's $4,000 a month, and everyone notices. Percentage pricing isn't automatically the wrong call, if a vendor is genuinely saving money elsewhere through negotiated provider rates or aggressive caching, the math can still work, but it should be a deliberate decision rather than a default a team falls into during a low-volume trial.

The gateway underneath a percentage fee usually isn't doing more work as volume grows. It's still proxying JSON, streaming bytes, and logging metadata. The product didn't get more complex; it got more users. A percentage fee turns that success into an open-ended commission on it.

Model 2: Flat platform fee with zero token markup

This is what most engineers assume they're buying when they buy a gateway: pay for the platform, pass provider pricing straight through. FastRouter runs this model: published tiers (Starter, Pro, Business, Enterprise) priced by request volume and feature limits, with token costs passed through at provider-listed rates at zero markup, on every request, regardless of model or provider. The platform fee and the token spend are two separate numbers that don't multiply against each other.

curl \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek/deepseek-v4.1-flash",
    "messages": [{ "role": "user", "content": "List the 4 common LLM gateway pricing models." }]
  }'

The upside is predictability: token spend grows because the application grows, but the gateway doesn't take a percentage cut of that growth, and forecasting stays sane because tier limits are known in advance. The tradeoff is that flat-fee vendors almost always gate something by tier, request volume, log retention, SSO or RBAC, private networking, SLA and support, so cost increases happen at tier boundaries instead of compounding continuously. That's usually easier to plan around than a percentage that grows forever, but it still matters for spiky traffic or chatty architectures like agents, tool routers, and evaluation runs.

Model 3: Request- or seat-volume SaaS tiers

A number of products in this space are closer to logging, analytics, or governance tools than full routing infrastructure. They may support routing, but they price like an observability platform: requests logged, retention window, seats, or "units." Portkey publishes a free Developer tier (10K logged requests/month, 3-day retention), a $49/month Production tier (100K requests, $9 per additional 100K, 30-day retention), and a custom Enterprise tier with SSO and private or VPC deployment. Helicone publishes a free Hobby tier (10K requests/month), a $79/month Pro tier with usage-based overage, and a $799/month Team tier adding SOC 2 and HIPAA coverage. Langfuse is observability and tracing only, it doesn't route model calls, and publishes a free Hobby tier (50K units/month), a $29/month Core tier, a $199/month Pro tier, and a $2,499/month Enterprise tier, priced on usage volume and retention.

The part that bites teams is that "request" isn't a universal unit. Depending on the vendor's own definition, it might cover one chat completion, one embedding call, one tool-invocation step, one retry, one fallback attempt to a second provider, or a streamed response counted as one request, or oddly, more than one. Token spend can look perfectly fine while request counts quietly explode: an embedding ingestion pipeline turns one document into hundreds of cheap-per-call requests that still count fully against a quota, and an agent loop turns one user action into a plan, a tool selection, a tool execution, a summarization, and a retry, each cheap in tokens but each its own request. Request-based pricing works well for low-QPS, high-token workloads like long generations and heavy RAG, and works badly for high-QPS, low-token patterns like routers, classifiers, embeddings, and agents. No team should sign a request-capped plan without getting the vendor's exact definition of "request" or "unit" in writing.

Model 4: Open-source self-hosted plus enterprise add-ons

This bucket means running the gateway software directly, optionally paying for enterprise features and support on top. LiteLLM's open-source proxy is free to self-host indefinitely, supporting 100+ providers with virtual keys, budgets, and fallbacks built in; its Enterprise tier adds SSO/SCIM, audit logs, multi-region deployment, and 24/7 support, custom-priced and explicitly not per-token. Kong AI Gateway runs a sales-led process with no published tiered pricing. TrueFoundry publishes SaaS tiers alongside a self-host option, with self-hosted infrastructure running roughly $600 to $1,000 a month on top of the platform fee. Cloudflare AI Gateway keeps core routing, caching, and analytics free with any Cloudflare account, charging instead for log volume beyond Workers plan limits and a 5% fee specifically on credit purchases made through its Unified Billing option, with underlying provider inference rates passed through unchanged; guardrails are billed separately as Workers AI inference.

Self-hosting removes a vendor invoice for the gateway software. It doesn't remove cost, it moves it into engineering time and operational risk: uptime for the proxy layer, scaling and load shedding, upgrades when upstream providers change schemas, secret management and key rotation, incident response when the gateway itself becomes the bottleneck, and compliance requirements around audit logs and access control. A commonly reported failure mode is self-hosted infrastructure that runs fine until it doesn't: the proxy becomes a critical dependency, and an on-call engineer ends up debugging timeouts and backpressure at 2 a.m. even though every underlying model provider is healthy. Self-hosting can be the right call for teams with strong platform operations and a genuine need for full control, but it should be a deliberate trade, not an accident that comes from comparing only license cost.

The ten platforms, at a glance

Five questions to ask before trusting any gateway pricing page

• Is the fee a percentage of token spend, or a flat number? A percentage has no ceiling; a flat fee does. A "0% markup" claim paired with a 5% funding fee still behaves like a percentage fee at scale. The label doesn't matter; the math does.
• What happens above the published tier? Some vendors offer smooth overage pricing, others force a jump to the next fixed tier, and a few impose a hard cap that's usually unacceptable in production.
• Is routing bundled with observability, or is observability the entire product? Langfuse is observability, not routing. Several platforms are priced and built like logging products while looking, at a glance, like gateways.
• If it's open source, what becomes enterprise-only later? SSO/SCIM, audit logs, multi-region deployment, and support are the recurring ones. Teams tend to assume they won't need it until the point they suddenly do.
• What counts as a "request" or "unit"? Whether retries count, whether failover counts as multiple requests, how tool calls and embeddings are counted, and how streaming affects the count all belong in writing before a plan is signed.

The arithmetic that matters at scale

Take a team spending $50,000 a month on model tokens, a realistic number for a mid-size production product. At a 5% markup, the gateway fee on that spend is $2,500 a month. If spend grows to $150,000 a month, the fee becomes $7,500 a month, with nothing new happening on the gateway's side beyond proxying more traffic and taking a bigger cut of it.

Compare that to a flat platform fee: FastRouter's Business tier, for example, is $799 a month covering up to 50M requests, a number that doesn't move whether the token spend behind those requests is $50,000 or $150,000. The crossover point where a percentage fee starts costing more than a given flat fee is simply the flat fee divided by the percentage: $799 divided by 0.05 is roughly $15,980 a month in token spend. Above that line, a 5% model costs more than the flat $799 fee, ignoring tier changes and overages. The exact number moves by vendor, but the shape of the curve doesn't: a percentage fee is a tax on growth, and a flat fee is not.

Failure modes pricing pages don't warn about

• Failover doubles or triples billable events. One end-user action can turn into multiple provider attempts during an incident, and request-based plans get expensive exactly when reliability is already bad.
• Retries inflate logged requests. Observability-first products often count retries as separate requests, so a dashboard shows more usage precisely when the underlying system is unhealthy.
• Agent loops multiply requests. A simple chat feature can turn into a dozen or more internal calls: routing, tool selection, tool execution, summarization, retries, and fallback. Request quotas disappear fast against that pattern.
• Embedding ingestion is request-heavy. Chunking turns one document into hundreds of calls. Token spend stays modest while request volume explodes.
• Self-hosting fails at the seams. The proxy goes down, not the model provider, which means debugging connection pools, timeouts, and backpressure rather than a provider status page.

A gateway decision isn't really about what it costs on a good day. It's about what it costs, and how it behaves, on the worst day of the quarter.

What to do this week

• Pull real traffic numbers from the last 7 to 30 days: total token spend, total request count, retry rate, and average LLM calls per user action, since agent and tool chains change this math substantially.
• Price out 10x current volume using the actual traffic shape. If requests scale faster than tokens, agent- and embedding-heavy systems, request-capped tiers will hurt first. If tokens scale faster than requests, percentage markup will hurt first.
• Ask vendors the annoying counting questions and get the answers in writing. What counts as a request or unit, whether retries and failover count separately, and what happens at the cap: overage, tier jump, or cutoff.
• Decide what kind of pain is acceptable. Percentage pricing costs more when the product succeeds; flat fees cost more when tier limits are outgrown; request-based SaaS costs more when the system is chatty or unhealthy; self-hosting costs in engineering time and on-call load.
• Run a staging spike to measure switching cost, even without switching yet: point a service at a gateway's endpoint, update model identifiers, run the existing test suite plus a small load test, and confirm logging, retention, and quotas hold up under retries and failover.

Getting started

FastRouter's own published pricing, including its zero-markup structure across every tier, is documented at fastrouter.ai/pricing .

Pricing figures above reflect each vendor's own published pages as of September 2026. Gateway pricing changes frequently; reverify every number against the live vendor page before publishing or making a purchasing decision.