beyhan/bifrost
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Beyhan Oğur
2026-04-26 21:52:23 +03:00
commit 880f412e2c
2662 changed files with 866266 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

646
docs/migration-guides/v1.5.0.mdx Normal file
View File

@@ -0,0 +1,646 @@
---
title: "Migrating to v1.5.0"
description: "Breaking changes and migration instructions for the v1.5.0 release"
---
v1.5.0 introduces several breaking changes across provider key configuration, Virtual Key semantics, the Go SDK, and the REST API. This page consolidates every breaking change with before/after examples and a migration checklist.
<Warning>
**Make a database backup before upgrading.** Automatic database migrations run on startup and are not revertible. A backup is the only way to restore a previous state if anything goes wrong. A database successfully migrated to v1.5.0 cannot be used to run v1.4.x.
</Warning>
---
## Automatic Database Migration
If you are running Bifrost with a database (SQLite or Postgres), existing data is automatically migrated on startup. You do not need to manually update your database records.
The following automatic migrations run on upgrade:
- Provider keys with `models: []` are converted to `models: ["*"]`
- Virtual Key provider configs with `allowed_models: []` are converted to `allowed_models: ["*"]`
- Virtual Keys with no `provider_configs` are backfilled with all currently configured providers (`allowed_models: ["*"]`, `key_ids: ["*"]`)
- Virtual Keys with no `mcp_configs` are backfilled with all currently connected MCP clients (`tools_to_execute: ["*"]`)
- Per-provider `deployments` maps (Azure, Bedrock, Vertex, Replicate) are migrated into the unified `aliases` field
**The automatic migration only protects your existing data.** Any new configuration created after upgrading — via `config.json` or the REST API — must follow the new semantics described below.
---
## Breaking Change 1: Empty Array Now Means "Deny All"
v1.5.0 flips the meaning of empty arrays across all allow-list fields:
| What you write | v1.4.x meaning | v1.5.0 meaning |
|---|---|---|
| `[]` (empty array) | Allow **all** | Allow **none** |
| `["*"]` (wildcard) | Not applicable | Allow **all** |
| `["a", "b"]` | Only `a` and `b` | Only `a` and `b` (unchanged) |
This affects four fields:
| Field | Where |
|---|---|
| `models` | Provider key |
| `allowed_models` | Virtual Key provider config |
| `key_ids` | Virtual Key provider config |
| `tools_to_execute` | Virtual Key MCP config |
### Provider key `models`
**Before:**
```json
{ "value": "env.OPENAI_API_KEY", "models": [] }
```
`models: []` → key served all models
**After:**
```json
{ "value": "env.OPENAI_API_KEY", "models": ["*"] }
```
### Virtual Key `allowed_models`
**Before:**
```json
{ "provider": "openai", "weight": 1.0 }
```
Missing `allowed_models` → all models allowed
**After:**
```json
{ "provider": "openai", "allowed_models": ["*"], "key_ids": ["*"], "weight": 1.0 }
```
### Virtual Key MCP `tools_to_execute`
**Before:**
```json
{ "mcp_client_name": "my-tools", "tools_to_execute": [] }
```
**After:**
```json
{ "mcp_client_name": "my-tools", "tools_to_execute": ["*"] }
```
---
## Breaking Change 2: `allowed_keys` Renamed to `key_ids`
The field used to restrict which provider API keys a Virtual Key can use has been renamed from `allowed_keys` to `key_ids`. The deny-by-default rule also applies — omitting the field or setting it to `[]` now blocks all keys.
<Note>
Unlike `allowed_models`, there is no automatic database migration for `key_ids`. An empty or omitted `key_ids` disables all key selection. You must explicitly use `["*"]` to restore allow-all behavior.
</Note>
**Before:**
```json
{ "provider": "openai", "allowed_keys": ["key-prod-001"], "weight": 1.0 }
```
**After:**
```json
{ "provider": "openai", "key_ids": ["key-prod-001"], "allowed_models": ["*"], "weight": 1.0 }
```
To allow all keys:
```json
{ "provider": "openai", "key_ids": ["*"], "allowed_models": ["*"], "weight": 1.0 }
```
---
## Breaking Change 3: Virtual Key `provider_configs` is Deny-by-Default
In v1.4.x, a Virtual Key with no `provider_configs` had access to all providers. In v1.5.0, it blocks all providers.
**Before:** `"provider_configs": []` → access to all providers
**After:** `"provider_configs": []` → no provider access
To allow all providers, list each one explicitly:
```json
{
"provider_configs": [
{ "provider": "openai", "allowed_models": ["*"], "key_ids": ["*"], "weight": 1.0 },
{ "provider": "anthropic", "allowed_models": ["*"], "key_ids": ["*"], "weight": 1.0 }
]
}
```
The automatic migration backfills all currently configured providers into any VK that has an empty `provider_configs`. However, any VK created after upgrading must include explicit provider configs.
---
## Breaking Change 4: WhiteList Validation
Two new validation rules are enforced on all allow-list fields. The API returns **HTTP 400** if either is violated.
**Rule 1: Wildcard cannot be mixed with other values**
```json
// ❌ Invalid
{ "allowed_models": ["*", "gpt-4o"] }
// ✅ Valid
{ "allowed_models": ["*"] }
```
**Rule 2: No duplicate values**
```json
// ❌ Invalid
{ "allowed_models": ["gpt-4o", "gpt-4o"] }
```
Applies to: `models`, `allowed_models`, `key_ids`, `tools_to_execute`, `tools_to_auto_execute`, `allowed_extra_headers`.
---
## Breaking Change 5: `weight` is Now Nullable
The `weight` field on a Virtual Key provider config was previously a required `float64`. It is now an optional `*float64`.
- `weight: 0.5` — provider participates in weighted load balancing
- `weight: null` / omitted — provider is accessible for direct routing but excluded from weighted selection
**API response change:** `weight` may now be `null`. Update any client code that assumes it is always a number.
---
## Breaking Change 6: Provider Keys API Separated
Provider key management now has dedicated endpoints. The `keys` field has been removed from all provider API requests and responses.
### What changed
| Before (v1.4.x) | After (v1.5.0) |
|---|---|
| `GET /api/providers/{p}` returns `keys` | `keys` field removed from provider response |
| `POST /api/providers` accepts `keys` | `keys` field ignored — create keys separately |
| `PUT /api/providers/{p}` accepts `keys` | `keys` field ignored — update keys via dedicated endpoints |
### New endpoints
| Method | Endpoint | Description |
|---|---|---|
| `GET` | `/api/providers/{provider}/keys` | List all keys |
| `GET` | `/api/providers/{provider}/keys/{key_id}` | Get a single key |
| `POST` | `/api/providers/{provider}/keys` | Create a key |
| `PUT` | `/api/providers/{provider}/keys/{key_id}` | Update a key |
| `DELETE` | `/api/providers/{provider}/keys/{key_id}` | Delete a key |
### How to update
**Creating a provider with keys:**
**Before:**
```bash
curl -X POST localhost:8080/api/providers -d '{
"provider": "openai",
"keys": [{"name": "main", "value": "sk-..."}]
}'
```
**After:** Create provider first, then add keys:
```bash
curl -X POST localhost:8080/api/providers -d '{"provider": "openai"}'
curl -X POST localhost:8080/api/providers/openai/keys -d '{"name": "main", "value": "sk-..."}'
```
**Reading keys:**
**Before:** `curl localhost:8080/api/providers/openai | jq '.keys'`
**After:** `curl localhost:8080/api/providers/openai/keys | jq '.keys'`
**Updating / deleting keys:**
**Before:** Bulk replace via provider update:
```bash
curl -X PUT localhost:8080/api/providers/openai -d '{"keys": [{"id": "key-1", "value": "sk-new"}]}'
```
**After:** Individual key operations:
```bash
curl -X PUT localhost:8080/api/providers/openai/keys/key-1 -d '{"name": "updated", "value": "sk-new"}'
curl -X DELETE localhost:8080/api/providers/openai/keys/key-2
```
---
## Breaking Change 7: Compact Plugin Restructured
The `enable_litellm_fallbacks` option has been removed and replaced with three granular options.
**Before:**
```json
{ "compat": { "enable_litellm_fallbacks": true } }
```
**After:**
```json
{
"compat": {
"convert_text_to_chat": true,
"convert_chat_to_responses": true,
"should_drop_params": true
}
}
```
| Old option | New option | Description |
|---|---|---|
| `enable_litellm_fallbacks` | `convert_text_to_chat` | Text completion → chat completion fallback |
| _(new)_ | `convert_chat_to_responses` | Chat completion → Responses API fallback |
| _(new)_ | `should_drop_params` | Drop unsupported OpenAI-compatible params |
**Response field changes:**
| Field | Change |
|---|---|
| `extra_fields.litellm_compat` | **Removed** |
| `extra_fields.dropped_compat_plugin_params` | **Added** — lists params dropped by this plugin |
| `extra_fields.converted_request_type` | **Added** — the request type it was converted to |
---
## Breaking Change 8: Replicate Image Edits Removed from Generations Endpoint
The `/v1/images/generations` endpoint on the Replicate provider no longer accepts image editing parameters (source image, mask). It now only handles text-to-image generation.
If you were passing image editing parameters to `/v1/images/generations` on Replicate, switch to `/v1/images/edits`.
<Note>
Support for image editing via `/v1/images/edits` on Replicate is also being removed in a follow-up release. Plan to migrate to an alternative provider.
</Note>
---
## Breaking Change 9: Provider `deployments` removed — migrate to `aliases`
Provider deployment mappings should now live in the top-level `aliases` field on each key. Aliases work across all providers and map any model name to a provider-specific identifier (deployment name, inference profile ARN, fine-tuned model ID, etc.).
The database migration runs automatically on startup, migrating existing deployment data into `aliases`. Only `config.json` files need to be updated manually.
### Azure
**Before:**
```json
{
"providers": {
"azure": {
"keys": [{
"value": "env.AZURE_API_KEY",
"azure_key_config": {
"endpoint": "env.AZURE_ENDPOINT",
"deployments": {
"gpt-4o": "my-gpt4o-deployment",
"gpt-4o-mini": "my-mini-deployment"
}
}
}]
}
}
}
```
**After:**
```json
{
"providers": {
"azure": {
"keys": [{
"value": "env.AZURE_API_KEY",
"azure_key_config": {
"endpoint": "env.AZURE_ENDPOINT"
},
"aliases": {
"gpt-4o": "my-gpt4o-deployment",
"gpt-4o-mini": "my-mini-deployment"
}
}]
}
}
}
```
### Bedrock
<Warning>
`bedrock_key_config.deployments` is a legacy field and is **removed** in v1.5.0 config semantics. Some setups/builds may still accept it for backward compatibility, but do not rely on it — migrate to `aliases` to avoid silent breakage and future removal.
</Warning>
**Before:**
```json
{
"bedrock_key_config": {
"region": "env.AWS_REGION",
"deployments": {
"claude-3-5-sonnet": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
}
}
}
```
**After:**
```json
{
"bedrock_key_config": {
"region": "env.AWS_REGION"
},
"aliases": {
"claude-3-5-sonnet": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
}
}
```
### Vertex
**Before:**
```json
{
"vertex_key_config": {
"project_id": "env.VERTEX_PROJECT_ID",
"project_number": "env.VERTEX_PROJECT_NUMBER",
"region": "env.VERTEX_REGION",
"auth_credentials": "env.VERTEX_AUTH_CREDENTIALS",
"deployments": {
"gemini-2.0-flash": "projects/my-project/locations/us-central1/endpoints/123456"
}
}
}
```
**After:**
```json
{
"vertex_key_config": {
"project_id": "env.VERTEX_PROJECT_ID",
"project_number": "env.VERTEX_PROJECT_NUMBER",
"region": "env.VERTEX_REGION",
"auth_credentials": "env.VERTEX_AUTH_CREDENTIALS"
},
"aliases": {
"gemini-2.0-flash": "projects/my-project/locations/us-central1/endpoints/123456"
}
}
```
### Replicate
The Replicate key config is also restructured. The `deployments` map is gone. A new boolean `use_deployments_endpoint` controls whether requests are routed through the [Deployments API](https://replicate.com/docs/reference/http#deployments.predictions.create) (private, fixed hardware) or the standard Models API.
**Before:**
```json
{
"replicate_key_config": {
"deployments": {
"my-model": "owner/model-name/version-hash"
}
}
}
```
**After:**
```json
{
"replicate_key_config": {
"use_deployments_endpoint": true
},
"aliases": {
"my-model": "owner/model-name"
}
}
```
| Old field | New field | Notes |
|---|---|---|
| `replicate_key_config.deployments` | Removed | Use top-level `aliases` |
| _(new)_ | `replicate_key_config.use_deployments_endpoint` | `bool`, default `false` |
---
## Breaking Change 10: Go SDK — `ExtraFields` Model Fields Renamed
`ModelRequested string` has been replaced by two fields on `BifrostResponseExtraFields` and `BifrostErrorExtraFields`.
**Before:**
```go
model := response.ExtraFields.ModelRequested
```
**After:**
```go
// The alias the caller passed as "model" in the request
original := response.ExtraFields.OriginalModelRequested
// The actual identifier sent to the provider API
// Equals OriginalModelRequested when no alias is configured
resolved := response.ExtraFields.ResolvedModelUsed
```
The same rename applies to `BifrostErrorExtraFields`.
**JSON tag changes:**
| Old | New |
|---|---|
| `"model_requested"` | `"original_model_requested"` + `"resolved_model_used"` |
---
## Breaking Change 11: Go SDK — `StreamAccumulatorResult` Field Renamed
`Model string` has been replaced by two fields on `StreamAccumulatorResult` (returned by tracer streaming accumulation methods).
**Before:**
```go
result.Model
```
**After:**
```go
result.RequestedModel // original alias from the caller
result.ResolvedModel // actual model identifier used by the provider
```
---
## Breaking Change 12: `selected_key_id` Cleared on Terminal Retry Failures
With the introduction of multi-key retry rotation, `selected_key_id` (and `selected_key_name`) in the request context are **cleared when all retry attempts fail**. Previously, these fields always reflected the key that was selected for the request, even on error.
The `attempt_trail` is now the authoritative record of every key tried and why each attempt failed.
### What changed
| Field | Before | After |
|---|---|---|
| `selected_key_id` | Always set, even on error | Empty string when all retries exhausted |
| `selected_key_name` | Always set, even on error | Empty string when all retries exhausted |
| `attempt_trail` | Not present | Array of `{ key_id, key_name, fail_reason }` per attempt |
### Impact on logging and telemetry plugins
The built-in **logging plugin** writes `selected_key_id` and `selected_key_name` directly to each log record. For multi-key requests that exhaust all retries, both fields will be empty in the stored log entry. The `attempt_trail` column captures the full per-attempt key history and is the correct field to use for failure attribution.
The built-in **telemetry plugin** emits `selected_key_id` and `selected_key_name` as span attributes. For exhausted-retry failures these attributes will be empty strings on the error span. The `attempt_trail` span attribute contains the full rotation history.
If you run a custom plugin or downstream log consumer that filters or groups by `selected_key_id` to track which key caused a failure, you must update it to handle the empty-string case and read from `attempt_trail` when attribution is needed.
### How to update
**If you read `selected_key_id` from plugin context to attribute failed requests:**
**Before:**
```go
keyID, _ := ctx.Value(schemas.BifrostContextKeySelectedKeyID).(string)
// keyID was always populated, even on error
```
**After:**
```go
// Populated on success (or for single-key / pinned / sticky flows on error):
keyID, _ := ctx.Value(schemas.BifrostContextKeySelectedKeyID).(string)
// For full attribution across all retry attempts (including failures):
if trail, ok := ctx.Value(schemas.BifrostContextKeyAttemptTrail).([]schemas.KeyAttemptRecord); ok {
for _, record := range trail {
// record.KeyID, record.KeyName, record.FailReason
}
}
```
**If you consume `selected_key_id` from the logging REST API:**
The `selected_key_id` field on a `LogEntry` may now be an empty string when the request failed after exhausting all retries. Use `attempt_trail` for the full per-attempt key history.
<Note>
Single-key, pinned (`x-bf-key-id` / `x-bf-key-name`), and session-sticky requests are unaffected — they never rotate keys, so `selected_key_id` remains populated on failure for those flows.
</Note>
---
## Opting Out: `version: 1` Compatibility Mode
If you are not ready to adopt the new deny-by-default semantics, you can add a single field to `config.json` to restore v1.4.x behavior for all allow-list fields loaded from that file:
```json
{
"version": 1,
"providers": { ... }
}
```
| Value | Behavior |
|---|---|
| `2` (default, omitted) | v1.5.0 semantics — empty = deny all, `["*"]` = allow all |
| `1` | v1.4.x semantics — empty = allow all |
**What `version: 1` normalizes at startup** (before any other processing):
| Field | Without `version: 1` | With `version: 1` |
|---|---|---|
| Provider key `models: []` | Deny all models | Allow all models (→ `["*"]`) |
| VK `provider_configs: []` | No providers allowed | All configured providers added with `allowed_models: ["*"]` |
| VK provider config `allowed_models: []` | Deny all models | Allow all models (→ `["*"]`) |
| VK provider config `key_ids: []` | No keys allowed | All keys allowed (→ `key_ids: ["*"]`) |
| VK `mcp_configs: []` | No MCP tools allowed | All configured MCP clients added with `tools_to_execute: ["*"]` |
<Note>
`version: 1` only applies to configuration loaded from `config.json`. Virtual Keys created or updated via the REST API always use v1.5.0 semantics regardless of this setting. The automatic database migration that runs on startup is also unaffected.
</Note>
<Warning>
`version: 1` is a temporary compatibility shim. Plan to migrate your `config.json` to explicit `["*"]` wildcards and remove the `version` field before the next major release.
</Warning>
---
## Complete Migration Checklist
<Steps>
<Step title="Backup your database">
Make a copy of your config store database before starting the upgrade.
</Step>
<Step title="Update provider key models in config.json">
Replace `"models": []` or missing `models` fields with `"models": ["*"]` on every provider key.
</Step>
<Step title="Add allowed_models and key_ids to every VK provider config">
Add `"allowed_models": ["*"]` and `"key_ids": ["*"]` to every `provider_configs` entry (or list specific values). Rename any `allowed_keys` fields to `key_ids`.
</Step>
<Step title="Ensure every VK has at least one provider config">
Any Virtual Key with `"provider_configs": []` or no `provider_configs` will block all traffic.
</Step>
<Step title="Update tools_to_execute for MCP configs">
Replace `"tools_to_execute": []` with `"tools_to_execute": ["*"]`. Ensure every VK that needs MCP access has at least one `mcp_configs` entry.
</Step>
<Step title="Handle nullable weight in API consumers">
Update any client code that processes `weight` to accept `null` in addition to numbers.
</Step>
<Step title="Fix invalid WhiteList values">
Ensure no list mixes `"*"` with specific values (e.g., `["*", "gpt-4o"]`) and no list has duplicate entries.
</Step>
<Step title="Migrate key management to dedicated endpoints">
Stop sending `keys` in provider create/update payloads and stop reading `keys` from provider responses. Use `/api/providers/{provider}/keys` for all key operations.
</Step>
<Step title="Update compact plugin config">
Replace `enable_litellm_fallbacks` with the appropriate combination of `convert_text_to_chat`, `convert_chat_to_responses`, and `should_drop_params`.
</Step>
<Step title="Migrate provider deployments to aliases">
Move deployment mappings from provider-specific `deployments` fields into the top-level `aliases` field on each key. For Replicate, set `use_deployments_endpoint: true` if you were using the deployments endpoint.
</Step>
<Step title="Update Go SDK references to ExtraFields.ModelRequested">
Replace `ExtraFields.ModelRequested` with `ExtraFields.OriginalModelRequested` (and optionally read `ExtraFields.ResolvedModelUsed`). Update JSON consumers reading `"model_requested"` to use `"original_model_requested"` and `"resolved_model_used"`.
</Step>
<Step title="Update Go SDK references to StreamAccumulatorResult.Model">
Replace `.Model` with `.RequestedModel` (and optionally `.ResolvedModel`) on any `StreamAccumulatorResult` usage.
</Step>
<Step title="Handle empty selected_key_id on terminal retry failures">
If your code reads `selected_key_id` / `selected_key_name` from the request context or log entries to attribute failed requests, add a null/empty check and fall back to `attempt_trail` for the full per-attempt key history.
</Step>
</Steps>
---
## Troubleshooting
**All requests returning 403/blocked after upgrade**
A provider key has `models: []`, a Virtual Key has no `provider_configs`, or a provider config has `allowed_models: []`. Check Bifrost logs — a blocked request logs which rule denied it. Fix: add `"models": ["*"]` on provider keys, `"allowed_models": ["*"]` on VK provider configs.
**MCP tools not being injected / tool calls blocked**
The VK needs an `mcp_configs` entry for the MCP client with `"tools_to_execute": ["*"]` (or specific tools).
**API returning 400 on VK create/update**
A whitelist validation failure — either mixing `"*"` with specific values, or duplicate values in a list.
**"No keys available" or key selection errors**
A provider config with `key_ids` omitted or `[]` now blocks all keys (`allow_all_keys: false`). Add `"key_ids": ["*"]`.
**Provider create/update errors about `keys` field**
The `keys` field has been removed. Remove it from provider payloads and use `/api/providers/{provider}/keys` instead.
**Replicate requests failing after upgrade**
If you used `replicate_key_config.deployments`, move the mappings to the top-level `aliases` field and set `use_deployments_endpoint: true` if you were targeting the Deployments API.
**Go SDK compilation errors on `ModelRequested` or `StreamAccumulatorResult.Model`**
Rename to `OriginalModelRequested`/`ResolvedModelUsed` on ExtraFields, and `RequestedModel`/`ResolvedModel` on StreamAccumulatorResult.