BYOK on FastRouter: Route Your Own Keys and Custom Models Through One Gateway

The first time you ship an LLM feature, you hardcode an API key into your backend and call it a day. It works. Then you hit production scale, and the reality of managing multiple providers across multiple environments hits you.

You fine-tune a model on Azure to cut costs on a classification workload. You need gpt-5.4 for complex reasoning but want a fallback to claude-sonnet-4-6 when OpenAI drops connections. You deploy a compliance-mandated instance on Vertex AI. Suddenly your application code is littered with different SDKs, custom retry logic, and split telemetry. Your fine-tuned model sits outside your gateway — invisible to your monitoring, excluded from your fallback chains, ungoverned by your access controls.

We ran into this exact problem. A fine-tuned model handling a significant chunk of our inference traffic was completely invisible to our routing and monitoring infrastructure. When it went down, we found out from users, not from alerts. Two dashboards, two sets of logs, manual correlation at 2am.

FastRouter's BYOK (Bring Your Own Key) feature fixes this. You attach your own provider credentials directly to your FastRouter organization, and all traffic — standard models and custom models — routes through a single gateway with unified observability, governance, and security controls. Your pricing, your rate limits, your compliance posture. FastRouter handles the routing, fallback, and monitoring layer on top.

TL;DR

• BYOK routes traffic through your own provider accounts — your negotiated pricing, your rate limits, your compliance posture. FastRouter adds routing, observability, and governance on top.
• Custom model endpoints (fine-tuned, self-hosted) are now first-class citizens. Register them alongside catalog models and get the same fallback chains, cost tracking, and Activity Log coverage.
• Provider Slug is the key concept — it's the identifier that traces every request back to a specific credential set in the Activity Log. Name it well; you'll see it constantly.
• Only Organization Owners can create or modify integrations. Developers and Project Admins can consume them within project scope but cannot touch credentials.
• Catalog constraint matters: only models in the FastRouter Model Catalog are routable, even with your own key. Custom models must be explicitly registered.

What BYOK Actually Does (and Doesn't Do)

Let's be precise. When you create a BYOK integration, you're telling FastRouter: "Here are credentials for my own provider account. Route requests through my account, but give me your routing, observability, and governance layer on top."

What you get:

• Unified observability dashboard across all models (standard and custom)
• Fallback chains that can include your custom models
• Cost tracking per integration, per model
• Access governance — project-scoped credentials with role-based access
• Activity logging with Provider Slug attribution on every request
• Gateway configs for load balancing and failover

What you don't get:

• FastRouter doesn't manage your provider account. If your API key gets rate-limited or suspended, that's between you and the provider.
• FastRouter doesn't subsidize costs. You're paying your own negotiated rates.
• FastRouter doesn't magically make non-catalog models routable. The model must either be in the FastRouter catalog or you need to register it as a custom model with the right schema mapping.

This is a real tradeoff. You're trading the simplicity of FastRouter's shared key pools for full control over your provider relationship. For most production teams, that's the right call — especially if you have negotiated enterprise pricing or compliance requirements that demand your own accounts.

How the Integration Is Structured

Every BYOK integration is a named, project-scoped record. Three things define it:

1. A provider (Anthropic, OpenAI, Azure, Vertex AI, etc.)
2. Credentials (API key, service account JSON, or auth config depending on provider)
3. A model selection — which catalog models this integration serves, plus any custom models you register

You can have multiple integrations for the same provider. Separate Anthropic integrations for production and research, each scoped to different projects, each with different model sets — that's a normal configuration.

The Provider Slug is the identifier you actually use day-to-day. It shows up in:

• Virtual Model configurations
• Gateway Configs (fallback and load-balancing setups)
• Every entry in the Activity Log

When you're debugging a production issue, the Activity Log shows you exactly which Provider Slug — which credential set — was used for every request. That's your traceability layer. Name it something meaningful: anthropic-prod-classifier is useful at 2am. integration-3 is not.

Supported Providers

Most providers use a simple API key. Two have extra requirements worth knowing about:

Google Vertex AI requires a full Service Account JSON key file upload — the same JSON you'd download from the GCP IAM console. Make sure the service account has the right IAM roles before you try to configure this.

Microsoft Azure requires your API key plus your Azure Resource Name and deployment configuration. Azure's deployment model is more complex than most providers, so expect to spend a few minutes getting the deployment config right. The thing that actually worked for us was keeping a separate document mapping FastRouter model slugs to Azure deployment names — they drift apart faster than you'd expect.

Setting Up an Integration

Only Organization Owners can create integrations. If you're a Project Admin or Developer, you can consume them — you cannot modify credentials.

Navigate to Setup → External Keys and click New Integration. Three steps.

Step 1 — Select the Provider

Pick from the list. There's a search filter if you have a lot of providers to sort through. Nothing complicated here.

Step 2 — Integration Details

This is where the important decisions happen.

The Provider Slug matters more than you think. Every request that uses this integration gets tagged with this slug in the Activity Log. When you're doing incident response, the slug is how you figure out which credential and which provider account was involved. Establish a naming convention before you have 15 integrations with inconsistent slugs. Something like {provider}-{environment}-{purpose} works: anthropic-prod-classifier, openai-research-general.

Project scoping is your access boundary. If you scope an integration to "Project Alpha," only users with access to Project Alpha can route traffic through those credentials. This is how you prevent your research team from accidentally burning through your production API budget. Don't default to "All Projects" unless you genuinely mean it.

Credential Fields

For most providers, you paste an API key. For Vertex AI, you upload a Service Account JSON file. For Azure, you provide an API key, Resource Name, and deployment configuration.

Advanced Options (Worth Knowing About)

Two advanced options are collapsed by default:

• Custom Host — Override the provider's default base URL. When set, you choose an API format: OpenAI-compatible, Anthropic-compatible, or Cohere-compatible. This is how you point to custom endpoints, proxies, or self-hosted inference servers.
• Custom Auth Headers — Additional headers forwarded to your endpoint. These are not logged by FastRouter, which is the right security decision for sensitive auth material but also means there's no audit trail for what's in those headers.

Step 3 — Model Provisioning

This step controls which models are available through this integration.

Important constraint: FastRouter only routes to catalog models. Even with your own API key, you can only enable models that exist in FastRouter's model catalog. If a model isn't in the catalog, contact support to request it. Toggle models on/off individually, or use All/None shortcuts.

Auto-enable (where new catalog models from this provider are automatically enabled on the integration) is coming soon but not available yet.

A note on model choices here: be deliberate. claude-sonnet-4-6 returns structured JSON reliably without prompt hacks, whereas gemini-3.1-pro-preview often needs explicit format instructions injected into the system prompt to avoid breaking your parsers. When you're toggling models on for a BYOK integration, think about which models your downstream code actually handles correctly.

Registering Custom Models

This is the part that changes the game for teams running fine-tuned or custom-hosted models. Inside Step 3, click Add Custom Model.

The Base Model field deserves attention. If your fine-tuned model is built on claude-sonnet-4-6 and uses the Anthropic request/response schema, set that as the base model. FastRouter uses it to know how to format requests and parse responses — serialization, token counting, and response parsing all depend on getting this right. If you set the wrong base model, you'll get schema mismatches that are painful to debug.

Custom models can be edited or deleted at any time from the model list. The provider cannot be changed after an integration is created — if you need a different provider, create a new integration.

Routing Traffic Through BYOK

Once your BYOK integration is active, you can make requests through FastRouter using the standard OpenAI-compatible API. The routing decision — whether to use your BYOK credentials or FastRouter's shared pool — happens on FastRouter's side based on your integration configuration and project scope.

cURL

1bash
2curl https://api.fastrouter.ai/api/v1/chat/completions \
3  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
4  -H "Content-Type: application/json" \
5  -d '{
6    "model": "anthropic/claude-sonnet-4-6",
7    "messages": [
8      {"role": "user", "content": "Explain BYOK in one sentence."}
9    ]
10  }'
Copy

Python (OpenAI SDK)

1python
2from openai import OpenAI
3
4client = OpenAI(
5    base_url="https://api.fastrouter.ai/api/v1",
6    api_key="your-fastrouter-api-key",
7)
8
9response = client.chat.completions.create(
10    model="anthropic/claude-sonnet-4-6",
11    messages=[
12        {"role": "user", "content": "Explain BYOK in one sentence."}
13    ],
14)
15
16print(response.choices[0].message.content)
17
Copy

TypeScript (OpenAI SDK)

1typescript
2import OpenAI from "openai";
3
4const client = new OpenAI({
5  baseURL: "https://api.fastrouter.ai/api/v1",
6  apiKey: "your-fastrouter-api-key",
7});
8
9const response = await client.chat.completions.create({
10  model: "anthropic/claude-sonnet-4-6",
11  messages: [
12    { role: "user", content: "Explain BYOK in one sentence." },
13  ],
14});
15
16console.log(response.choices[0].message.content);
Copy

The model string uses FastRouter's provider/model format. When your BYOK integration is active and scoped to the project your API key belongs to, FastRouter routes this request through your own provider credentials instead of shared key pools. The request format doesn't change — the routing decision happens server-side based on your integration configuration.

Once active, you reference your integration via its Provider Slug in Virtual Models (associate a model alias with a specific integration), Gateway Configs (use in fallback or load-balancing configurations), and the Activity Log (every request shows the Provider Slug).

What This Actually Enables

Once your custom model is registered, it's a first-class routing target. Concretely:

• Fallback chains that include your fine-tuned model. Configure a Gateway Config where traffic hits your custom model first, and falls back to a standard catalog model if it fails. That's something you simply cannot do when your custom endpoint is managed separately.
• Unified observability. Your fine-tuned model's calls appear in the same Activity Log as every other model. Same cost dashboard. No more correlating logs across separate systems.
• Dynamic routing without code changes. Use Dynamic Tags to route specific workloads to your custom model. The application code doesn't change — the routing rule does.
• Governance consistency. The same access controls that apply to your catalog models apply to your custom model. Project Admins and Developers can call it; only Organization Owners can modify the configuration.

Security: What You Should Actually Think About

Credential Storage and Access

• Credentials are encrypted at rest. After saving, API keys are never returned in API responses. You can't retrieve a key you've stored — only replace it.
• Only Organization Owners can create, modify, or delete integrations. Project Admins and Developers can use integrations within their project scope but cannot see or modify credentials.
• Custom Auth Headers are forwarded without logging. Correct security decision, but it means no audit trail for what's in those headers.

What I'd Actually Worry About

Key rotation cadence. Since you can't retrieve stored keys, you need a process for rotating keys that includes updating the integration in FastRouter. Build this into your key rotation runbook. Document the process: rotate the key at the provider, update the integration in FastRouter, verify traffic is flowing. Test this before you need it.

Overly broad project scoping. Setting every integration to "All Projects" defeats the purpose of project-level access control. Be deliberate about scoping, especially for production credentials.

Custom Host without TLS. If you're using Custom Host to point at a self-hosted model, make sure it's behind HTTPS. FastRouter doesn't enforce this for you.

Orphaned integrations. When a team member leaves, audit which integrations they created and whether the underlying API keys should be rotated.

Compliance implications. Traffic routes through your provider account. If your compliance posture requires specific data residency, your provider account configuration controls this, not FastRouter's. Your negotiated BAAs, DPAs, and other agreements remain intact because traffic flows through your account. The Activity Log with Provider Slug attribution gives you request-level audit trails — relevant for SOC 2 and similar frameworks.

Real Failure Modes

Here's what actually goes wrong with BYOK in production:

Stale credentials. You rotate your provider API key but forget to update the integration in FastRouter. Requests start failing. The Activity Log shows the Provider Slug, but the error comes from the provider, not FastRouter. If you have fallback chains configured, traffic silently routes to your fallback model — which might be more expensive. You won't notice until you check the bill.

Scope mismatch. You create an integration scoped to "Project A" but your API key is associated with "Project B." Requests from Project B can't use the integration. This is by design, but it catches people off guard.

Azure deployment config errors. Azure is notoriously pedantic. The model slug in FastRouter must match the actual deployment name configured in your Azure resource. If you name your FastRouter model gpt-5.4 but your Azure deployment is named my-gpt-deployment, Azure will return a generic 404, causing FastRouter to register a downstream failure. Cryptic errors from Azure, not helpful errors from FastRouter.

Custom model schema mismatch. You register a custom model but set the wrong Base Model for schema compatibility. The request serializes correctly but the response parsing breaks because the actual model returns a different format than expected. Always verify your base model schema mappings match the actual API format your custom model speaks.

Rate limit attribution confusion. Your BYOK integration uses your provider account's rate limits. If you have multiple integrations pointing to the same provider account, they share those rate limits — and you might hit limits faster than expected. This is invisible from within FastRouter; you need to monitor it on the provider side.

Vertex service account expirations. Google Vertex AI relies on Service Account JSON keys. If your SecOps team automatically rotates or revokes those service accounts in GCP, your BYOK integration will instantly start throwing 401 Unauthorized errors. Align your FastRouter integration updates with your GCP key rotation lifecycle.

What to Do This Week

1. Audit your current model integrations. List every custom endpoint your application calls directly, bypassing your gateway. List every LLM API key currently hardcoded or injected into your environment variables. These are your candidates for migration.
2. Establish a slug naming convention. Before you create your first integration, decide on a pattern. {provider}-{environment}-{purpose} works: anthropic-prod-classifier, openai-research-general. Consistency here pays dividends in log readability.
3. Create one BYOK integration. Start with your most important non-gateway model — probably a fine-tuned model or a custom endpoint. Go to Setup → External Keys, pick the provider, choose a meaningful Provider Slug. Scope it to the right project.
4. Register one custom model. Set the Base Model to match its API schema. Add pricing if you want cost tracking to work correctly.
5. Set up a fallback chain. Once the custom model is routing correctly, configure a Gateway Config that falls back to a catalog model if your custom endpoint fails. This alone is worth the setup effort. Test it by temporarily breaking your custom endpoint and confirming requests fall through.
6. Add key rotation to your ops runbook. Document: rotate the key at the provider → update the integration in FastRouter → verify traffic is flowing. Test the process before you need it in an incident.
7. Review project scoping. If you're using "All Projects" for everything, tighten it. Especially for production credentials. Check your Custom Host fields — verify they point where they should and use HTTPS.

After that, migrate the rest of your custom endpoints on the same pattern. BYOK is available now under Setup → External Keys.

Documentation is at docs.fastrouter.ai/add-external-keys-byok.