first commit
This commit is contained in:
Beyhan Oğur
242
docs/cli-agents/claude-code.mdx
Normal file
242
docs/cli-agents/claude-code.mdx
Normal file
@@ -0,0 +1,242 @@
|
||||
---
|
||||
title: "Claude Code"
|
||||
description: "Use Claude Code with Bifrost to route through any provider and unlock advanced features like MCP tools and observability."
|
||||
icon: "star-of-life"
|
||||
---
|
||||
|
||||
[Claude Code](https://www.claude.com/product/claude-code) brings AI-powered coding capabilities directly to your terminal.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost
|
||||
with Claude Code, try switching to `*` or adding the specific headers required by your client. By default, Bifrost
|
||||
whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## To install Claude code
|
||||
|
||||
```bash
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
```
|
||||
|
||||
## Configuring Claude Code to work with Bifrost
|
||||
|
||||
Claude Code supports multiple authentication methods. Choose the one that matches your account type.
|
||||
|
||||
1. **Set environment variables**
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=your-bifrost-virtual-key # or "dummy" if virtual keys are not enabled
|
||||
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic
|
||||
```
|
||||
|
||||
2. **Run Claude Code**
|
||||
|
||||
```bash
|
||||
claude
|
||||
```
|
||||
|
||||
## Setup with the VS Code extension
|
||||
|
||||
If you prefer using Claude Code inside VS Code, install the [Claude Code extension](https://marketplace.visualstudio.com/items?itemName=anthropic.claude-code) and point it to Bifrost using the same environment variables you use for the CLI.
|
||||
|
||||
### 1. Install the extension
|
||||
|
||||
1. Open **Extensions** in VS Code (`Cmd+Shift+X` on macOS)
|
||||
2. Search for **Claude Code**
|
||||
3. Install **Claude Code** by Anthropic
|
||||
|
||||
### 2. Configure Bifrost endpoint
|
||||
|
||||
Add Bifrost variables in your editor settings:
|
||||
|
||||
```json
|
||||
{
|
||||
"claudeCode.disableLoginPrompt": true,
|
||||
"claudeCode.environmentVariables": [
|
||||
{
|
||||
"name": "ANTHROPIC_BASE_URL",
|
||||
"value": "http://localhost:8080/anthropic"
|
||||
},
|
||||
{
|
||||
"name": "ANTHROPIC_DEFAULT_SONNET_MODEL",
|
||||
"value": "vertex/claude-sonnet-4-6"
|
||||
},
|
||||
{
|
||||
"name": "ANTHROPIC_DEFAULT_HAIKU_MODEL",
|
||||
"value": "vertex/claude-haiku-4-5"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If virtual keys are enabled, also add `ANTHROPIC_API_KEY` in the same list (use your Bifrost virtual key value). If not, set it to `dummy`.
|
||||
|
||||
## Amazon Bedrock via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
Configure environment variables
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=bifrost-virtual-key
|
||||
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic
|
||||
```
|
||||
|
||||
### Pin model versions (recommended)
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_DEFAULT_SONNET_MODEL="bedrock/global.anthropic.claude-sonnet-4-6"
|
||||
export ANTHROPIC_DEFAULT_HAIKU_MODEL="bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
```
|
||||
|
||||
If you don't pin using CLI - you can pin these in UI. Go to Dashboard > Models > Model Providers > AWS Bedrock > Key. And add deployments
|
||||
|
||||
<Frame>
|
||||
<img src="/media/cli/claude-bedrock-pinning.png" alt="Claude bedrock pinning" />
|
||||
</Frame>
|
||||
|
||||
### Start Claude Code
|
||||
|
||||
```
|
||||
claude
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Always pin model versions with `ANTHROPIC_DEFAULT_*_MODEL` when using Bedrock. Without pinning, Claude Code aliases
|
||||
resolve to the latest version, which may not be enabled in your Bedrock account.
|
||||
</Warning>
|
||||
|
||||
## Google Vertex AI via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
Configure environment variables
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=bifrost-virtual-key
|
||||
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic
|
||||
```
|
||||
|
||||
### Pin model versions (recommended)
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_DEFAULT_SONNET_MODEL="vertex/claude-sonnet-4-6"
|
||||
export ANTHROPIC_DEFAULT_HAIKU_MODEL="vertex/claude-haiku-4-5"
|
||||
```
|
||||
|
||||
### Start Claude Code
|
||||
|
||||
```
|
||||
claude
|
||||
```
|
||||
|
||||
## Azure via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
Configure environment variables
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_API_KEY=bifrost-virtual-key
|
||||
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic
|
||||
```
|
||||
|
||||
### Pin model versions (recommended)
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_DEFAULT_SONNET_MODEL="azure/claude-sonnet-4-6"
|
||||
export ANTHROPIC_DEFAULT_HAIKU_MODEL="azure/claude-haiku-4-5"
|
||||
```
|
||||
|
||||
You will also have to map these model names to actual deployment names on Bifrost dashboard.
|
||||
|
||||
<Frame>
|
||||
<img src="/media/cli/claude-azure-pinning.png" alt="Claude bedrock pinning" />
|
||||
</Frame>
|
||||
|
||||
### Start Claude Code
|
||||
|
||||
```
|
||||
claude
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Azure-hosted models must support **tool use capabilities** for Claude Code to function properly. Verify tool calling
|
||||
support before configuring Azure models.
|
||||
</Warning>
|
||||
|
||||
## Model Configuration
|
||||
|
||||
Claude Code uses three model tiers: **Sonnet** (default), **Opus** (complex tasks), and **Haiku** (fast, lightweight). With Bifrost, you can override these defaults to use any model from any provider.
|
||||
|
||||
**Override Default Models:**
|
||||
|
||||
Set environment variables to replace Claude Code's default model tiers with any Bifrost-configured model:
|
||||
|
||||
```bash
|
||||
# Replace Sonnet tier with GPT-5
|
||||
export ANTHROPIC_DEFAULT_SONNET_MODEL="openai/gpt-5"
|
||||
|
||||
# Replace Opus tier with Claude Opus 4.5
|
||||
export ANTHROPIC_DEFAULT_OPUS_MODEL="anthropic/claude-opus-4-5-20251101"
|
||||
|
||||
# Replace Haiku tier with Azure-hosted Claude
|
||||
export ANTHROPIC_DEFAULT_HAIKU_MODEL="azure/claude-haiku-4-5"
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Alternative models must support **tool use capabilities** for file operations, terminal commands, and code editing to
|
||||
work properly with Claude Code.
|
||||
</Warning>
|
||||
|
||||
**Start with a Specific Model:**
|
||||
|
||||
Launch Claude Code with a specific model using the `--model` flag:
|
||||
|
||||
```bash
|
||||
# Start with Opus
|
||||
claude --model claude-opus-4-5-20251101
|
||||
|
||||
# Start with Haiku for lightweight tasks
|
||||
claude --model claude-haiku-4-5-20251001
|
||||
```
|
||||
|
||||
## Switching Models Mid-Session
|
||||
|
||||
Use the `/model` command to switch models during an active session:
|
||||
|
||||
```bash
|
||||
# Using full model names
|
||||
/model claude-opus-4-5-20251101 # This will be used from anthropic provider
|
||||
/model claude-sonnet-4-5-20250929 # This will be used from anthropic provider
|
||||
|
||||
# Using different providers dynamically via Bifrost
|
||||
/model vertex/claude-haiku-4-5
|
||||
/model azure/claude-sonnet-4-5
|
||||
/model bedrock/claude-sonnet-4-5
|
||||
```
|
||||
|
||||
<Tip>
|
||||
Run `/model` without arguments to check your current model. The switch is instantaneous and Claude Code seamlessly
|
||||
continues your conversation context with the new model.
|
||||
</Tip>
|
||||
|
||||
<Warning>
|
||||
If you use Claude-specific features like **web search**, **computer use**, or **citations**, ensure the model you
|
||||
switch to also supports these capabilities. Non-Claude models or Claude models on certain providers may not support
|
||||
all features.
|
||||
</Warning>
|
||||
|
||||
## Provider Compatibility
|
||||
|
||||
<Warning>
|
||||
**Not all providers work well with Claude Code**. Since Claude Code heavily relies on tool calling for file operations, terminal commands, and code editing, providers must properly support and stream tool call arguments.
|
||||
|
||||
**Known Issues:**
|
||||
|
||||
- **OpenRouter**: Does not stream function call arguments properly. Tool calls return with empty `arguments` fields, causing Claude Code to fail when attempting file operations or other tool-based actions.
|
||||
- **Some proxy providers**: May not fully implement the Anthropic API streaming specification for tool calls.
|
||||
|
||||
If you experience issues with tool calls not executing properly, try switching to a different provider in your Bifrost configuration.
|
||||
|
||||
</Warning>
|
||||
199
docs/cli-agents/claude-desktop.mdx
Normal file
199
docs/cli-agents/claude-desktop.mdx
Normal file
@@ -0,0 +1,199 @@
|
||||
---
|
||||
title: "Claude Desktop"
|
||||
description: "Route Claude Desktop App traffic through Bifrost for multi-provider routing, virtual keys, and observability."
|
||||
icon: "desktop"
|
||||
---
|
||||
|
||||
The [Claude Desktop App](https://claude.ai/download) brings Claude to your desktop with a chat interface and a built-in Code tab powered by [Claude Code](https://claude.com/product/claude-code). By routing the Code tab through Bifrost, you get multi-provider routing, virtual keys, budget controls, and full observability.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Claude Desktop, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## How It Works
|
||||
|
||||
The Claude Desktop App has two tabs with different routing behavior:
|
||||
|
||||
| Tab | Traffic | Bifrost Integration |
|
||||
|-----|---------|-------------------|
|
||||
| **Code tab** | Uses the Anthropic Messages API (Claude Code under the hood) | Route inference through Bifrost's `/anthropic` endpoint |
|
||||
| **Chat tab** | Connects directly to claude.ai | Connect to Bifrost's `/mcp` endpoint for MCP tools |
|
||||
|
||||
This guide covers configuring the **Code tab** for inference routing and the **Chat tab** for MCP tool access.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Configure settings.json
|
||||
|
||||
Open `~/.claude/settings.json` and add the Bifrost endpoint and API key under the `env` key:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "your-bifrost-virtual-key",
|
||||
"ANTHROPIC_BASE_URL": "http://localhost:8080/anthropic"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For production deployments:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "your-bifrost-virtual-key",
|
||||
"ANTHROPIC_BASE_URL": "https://bifrost.example.com/anthropic"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Tip>
|
||||
You can open this file by running `/config` inside Claude Code, or by editing `~/.claude/settings.json` directly. These settings are shared between the Desktop app and the CLI.
|
||||
</Tip>
|
||||
|
||||
### 2. Start a New Session
|
||||
|
||||
Close any active session in the Code tab and start a new one for the settings to take effect.
|
||||
|
||||
## Amazon Bedrock via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
Add the Bifrost endpoint and pin model versions in `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "bifrost-virtual-key",
|
||||
"ANTHROPIC_BASE_URL": "http://localhost:8080/anthropic",
|
||||
"ANTHROPIC_DEFAULT_SONNET_MODEL": "bedrock/global.anthropic.claude-sonnet-4-6",
|
||||
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "bedrock/global.anthropic.claude-haiku-4-5-20251001-v1:0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If you don't pin using settings.json - you can pin these in UI. Go to Dashboard > Models > Model Providers > AWS Bedrock > Key. And add deployments.
|
||||
|
||||
<Frame>
|
||||
<img src="/media/cli/claude-bedrock-pinning.png" alt="Claude bedrock pinning" />
|
||||
</Frame>
|
||||
|
||||
<Warning>
|
||||
Always pin model versions when using Bedrock. Without pinning, Claude Code aliases resolve to the latest version, which may not be enabled in your Bedrock account.
|
||||
</Warning>
|
||||
|
||||
## Google Vertex AI via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "bifrost-virtual-key",
|
||||
"ANTHROPIC_BASE_URL": "http://localhost:8080/anthropic",
|
||||
"ANTHROPIC_DEFAULT_SONNET_MODEL": "vertex/claude-sonnet-4-6",
|
||||
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "vertex/claude-haiku-4-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Azure via Bifrost
|
||||
|
||||
### Setup
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_API_KEY": "bifrost-virtual-key",
|
||||
"ANTHROPIC_BASE_URL": "http://localhost:8080/anthropic",
|
||||
"ANTHROPIC_DEFAULT_SONNET_MODEL": "azure/claude-sonnet-4-6",
|
||||
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "azure/claude-haiku-4-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You will also have to map these model names to actual deployment names on Bifrost dashboard.
|
||||
|
||||
<Frame>
|
||||
<img src="/media/cli/claude-azure-pinning.png" alt="Claude Azure pinning" />
|
||||
</Frame>
|
||||
|
||||
<Warning>
|
||||
Azure-hosted models must support **tool use capabilities** for Claude Code to function properly. Verify tool calling support before configuring Azure models.
|
||||
</Warning>
|
||||
|
||||
## Model Configuration
|
||||
|
||||
The Code tab uses three model tiers: **Sonnet** (default), **Opus** (complex tasks), and **Haiku** (fast, lightweight). With Bifrost, you can override these defaults to use any model from any provider.
|
||||
|
||||
**Override Default Models:**
|
||||
|
||||
Add to `~/.claude/settings.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_DEFAULT_SONNET_MODEL": "openai/gpt-5",
|
||||
"ANTHROPIC_DEFAULT_OPUS_MODEL": "anthropic/claude-opus-4-5-20251101",
|
||||
"ANTHROPIC_DEFAULT_HAIKU_MODEL": "azure/claude-haiku-4-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Alternative models must support **tool use capabilities** for file operations, terminal commands, and code editing to work properly with Claude Code.
|
||||
</Warning>
|
||||
|
||||
## MCP Integration (Chat Tab)
|
||||
|
||||
The Chat tab supports MCP servers configured in `claude_desktop_config.json`. Connect to Bifrost's MCP endpoint to give the Chat tab access to all your aggregated MCP tools:
|
||||
|
||||
**Config file locations:**
|
||||
- **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
- **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"bifrost": {
|
||||
"url": "http://localhost:8080/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer your-bifrost-virtual-key"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
MCP servers in `claude_desktop_config.json` are for the **Chat tab only**. For MCP in the Code tab, configure servers in `~/.claude.json` or your project's `.mcp.json` file. See [MCP Gateway URL](/mcp/gateway-url) for full setup details.
|
||||
</Note>
|
||||
|
||||
## Enterprise Deployment
|
||||
|
||||
For organization-wide Bifrost routing, deploy a `managed-settings.json` file via MDM (Jamf, Kandji, Intune):
|
||||
|
||||
- **macOS**: `/Library/Application Support/ClaudeCode/managed-settings.json`
|
||||
- **Windows**: `C:\Program Files\ClaudeCode\managed-settings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"ANTHROPIC_BASE_URL": "https://bifrost.example.com/anthropic",
|
||||
"ANTHROPIC_API_KEY": "org-bifrost-virtual-key"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Managed settings cannot be overridden by individual users, ensuring all Code tab traffic routes through Bifrost.
|
||||
|
||||
## Observability
|
||||
|
||||
All Claude Desktop Code tab requests through Bifrost are logged. Monitor them at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage patterns across your organization.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
- [Built-in Observability](/features/observability/default) — Monitor all AI traffic
|
||||
- [MCP Gateway URL](/mcp/gateway-url) — Full MCP server setup and tool filtering
|
||||
117
docs/cli-agents/claude-for-office.mdx
Normal file
117
docs/cli-agents/claude-for-office.mdx
Normal file
@@ -0,0 +1,117 @@
|
||||
---
|
||||
title: "Claude for Office"
|
||||
description: "Use Claude for Office (Microsoft 365 add-in) with Bifrost to route requests through any provider with virtual keys, budget controls, and observability."
|
||||
icon: "file-word"
|
||||
---
|
||||
|
||||
[Claude for Office](https://marketplace.microsoft.com/en-us/product/saas/wa200009404?tab=overview) is Anthropic's Microsoft 365 add-in that brings Claude directly into Word, Excel, PowerPoint, and Outlook. By routing Claude for Office through Bifrost, you get governance features like virtual keys, budget controls, rate limits, and built-in observability for all office-based AI usage across your organization.
|
||||
|
||||
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install the Add-in
|
||||
|
||||
Install the [Claude for Office add-in](https://marketplace.microsoft.com/en-us/product/saas/wa200009404?tab=overview) from the Microsoft AppSource marketplace within any Office application (Word, Excel, PowerPoint, or Outlook). Organization admins can also deploy it centrally via the Microsoft 365 admin center.
|
||||
|
||||
### 2. Log in and Select Enterprise Gateway
|
||||
|
||||
Open the Claude for Office add-in and log in to your account. On the login screen, select **Enterprise Gateway** as your connection method.
|
||||
|
||||
### 3. Configure the API Endpoint
|
||||
|
||||
Enter your Bifrost endpoint as the **Base URL**:
|
||||
|
||||
```
|
||||
https://bifrost.example.com/anthropic
|
||||
```
|
||||
|
||||
For local development:
|
||||
|
||||
```
|
||||
http://localhost:8080/anthropic
|
||||
```
|
||||
|
||||
### 4. Set Your API Key
|
||||
|
||||
Enter your Bifrost [virtual key](/features/governance/virtual-keys) or Anthropic API key in the **API Key** field.
|
||||
|
||||
### 5. Whitelist Required Headers
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this step.
|
||||
</Note>
|
||||
|
||||
Claude for Office requires the following headers to be in the Allowed Headers list. Go to **Settings > Client Settings** and add them as a comma-separated list:
|
||||
|
||||
```
|
||||
anthropic-dangerous-direct-browser-access, anthropic-version, content-type, user-agent, x-api-key, x-stainless-arch, x-stainless-helper-method, x-stainless-lang, x-stainless-os, x-stainless-package-version, x-stainless-retry-count, x-stainless-runtime, x-stainless-runtime-version, x-stainless-timeout
|
||||
```
|
||||
|
||||
### 6. Enable CORS
|
||||
|
||||
<Note>
|
||||
If your Allowed Origins are already set to `*`, you can skip this step.
|
||||
</Note>
|
||||
|
||||
Claude for Office makes requests from the `pivot.claude.ai` domain. Go to **Settings > Client Settings** and add the following origin to your Allowed Origins list:
|
||||
|
||||
```
|
||||
https://pivot.claude.ai
|
||||
```
|
||||
|
||||
You're all set — Claude for Office will now route all requests through Bifrost.
|
||||
|
||||
## How It Works
|
||||
|
||||
Claude for Office uses the Anthropic Messages API natively. Bifrost exposes a fully compatible Anthropic API at the `/anthropic` path, so the add-in works without any additional configuration beyond pointing it at your Bifrost instance.
|
||||
|
||||
Bifrost automatically handles:
|
||||
|
||||
- **Model routing** — requests are routed to the correct provider based on the model name
|
||||
- **Tool stripping** — server-side tools like `code_execution`, `web_search`, and `web_fetch` are automatically stripped to prevent API conflicts
|
||||
- **Model metadata** — token limits and capabilities are returned in list models responses for proper model selection in the add-in
|
||||
|
||||
## Using Other Providers (Azure, Vertex, Bedrock)
|
||||
|
||||
By default, Claude for Office sends requests to the Anthropic API. To route requests through Azure, Vertex AI, or Amazon Bedrock instead, you have two options:
|
||||
|
||||
### Option 1: Set the Model Name Directly
|
||||
|
||||
In the Claude for Office model selector, enter the provider-specific model name or deployment name. Bifrost will automatically route the request to the correct provider based on the model identifier.
|
||||
|
||||
For example:
|
||||
- **Azure**: Enter your Azure deployment name (e.g., `claude-opus-4-0`)
|
||||
- **Vertex AI**: Enter the Vertex model ID (e.g., `claude-opus-4@20250514`)
|
||||
- **Bedrock**: Enter the Bedrock model ID (e.g., `anthropic.claude-opus-4-0-20250514-v1:0`)
|
||||
|
||||
### Option 2: Use Routing Rules
|
||||
|
||||
If you prefer to keep the default model names in Claude for Office and control routing server-side, create a [routing rule](/providers/routing-rules) in Bifrost. This lets you route requests to any provider — with fallbacks — without changing anything in the add-in.
|
||||
|
||||
For example, you can create a rule that matches requests from Claude for Office (based on the `origin` header) and routes them to Azure with a Vertex AI fallback:
|
||||
|
||||
|
||||
|
||||
This approach is ideal for organization-wide deployments where you want centralized control over which provider handles Claude for Office traffic.
|
||||
|
||||
## Using Virtual Keys
|
||||
|
||||
Bifrost [Virtual Keys](/features/governance/virtual-keys) can be used as the API key in Claude for Office. This lets you:
|
||||
|
||||
- Enforce per-user or per-team budgets and rate limits
|
||||
- Control which models and providers each user can access
|
||||
- Track usage and costs across your organization
|
||||
- Rotate credentials without updating every user's add-in configuration
|
||||
|
||||
For organization-wide deployments, create separate virtual keys for each team or department to manage AI spend independently.
|
||||
|
||||
## Observability
|
||||
|
||||
All Claude for Office requests through Bifrost are logged. Monitor them at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage patterns across your organization.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
- [Built-in Observability](/features/observability/default) — Monitor all AI traffic
|
||||
81
docs/cli-agents/codex-cli.mdx
Normal file
81
docs/cli-agents/codex-cli.mdx
Normal file
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: "Codex CLI"
|
||||
description: "Use OpenAI's Codex CLI with Bifrost for powerful code generation with any provider."
|
||||
icon: "openai"
|
||||
---
|
||||
|
||||
[Codex CLI](https://developers.openai.com/codex/cli/) provides powerful code generation and completion capabilities directly in your terminal.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not, and you face issues integrating Bifrost with Codex CLI, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Installing Codex CLI
|
||||
|
||||
```bash
|
||||
npm install -g @openai/codex
|
||||
```
|
||||
|
||||
## Configuring Codex CLI with Bifrost
|
||||
|
||||
<Warning>
|
||||
Codex CLI always prefers OAuth over custom API keys. Make sure you run `/logout` before configuring the Bifrost gateway with Codex.
|
||||
</Warning>
|
||||
|
||||
### Update codex.toml
|
||||
|
||||
Add the Bifrost base URL and credentials to your global `~/.codex/config.toml` or project-specific `.codex/config.toml`:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=<bifrost_virtual_key>
|
||||
```
|
||||
|
||||
```toml
|
||||
openai_base_url="http://localhost:8080/openai/v1"
|
||||
env_key="OPENAI_API_KEY"
|
||||
model = "openai/gpt-5.4"
|
||||
```
|
||||
|
||||
Always run `codex` from the same terminal session where you exported variables, or restart the terminal after changing your profile. GUI-launched terminals or IDEs may not pick up shell-profile exports unless the environment is configured there as well.
|
||||
|
||||
## Model Configuration
|
||||
|
||||
Use the `--model` flag to start Codex with a specific model:
|
||||
|
||||
```bash
|
||||
codex --model openai/gpt-5-codex
|
||||
codex --model openai/gpt-5.4-pro
|
||||
```
|
||||
|
||||
You can also switch models mid-session with the `/model` command:
|
||||
|
||||
```bash
|
||||
/model openai/gpt-5.4-pro
|
||||
/model openai/gpt-5-codex
|
||||
```
|
||||
|
||||
## Using Non-OpenAI Models with Codex CLI
|
||||
|
||||
Bifrost automatically translates OpenAI API requests to other providers, so you can use Codex CLI with models from Anthropic, Google, Mistral, and more. Use the `provider/model-name` format to specify any Bifrost-configured model:
|
||||
|
||||
```bash
|
||||
# Start with an Anthropic model
|
||||
codex --model anthropic/claude-sonnet-4-5-20250929
|
||||
|
||||
# Start with a Google model
|
||||
codex --model gemini/gemini-2.5-pro
|
||||
|
||||
# Switch mid-session
|
||||
/model anthropic/claude-sonnet-4-5-20250929
|
||||
/model mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-OpenAI models **must support tool use** for Codex CLI to work properly. Codex CLI relies on tool calling for file operations, terminal commands, and code editing. Models without tool use support will fail on most operations.
|
||||
</Warning>
|
||||
90
docs/cli-agents/cursor.mdx
Normal file
90
docs/cli-agents/cursor.mdx
Normal file
@@ -0,0 +1,90 @@
|
||||
---
|
||||
title: "Cursor"
|
||||
description: "Add Bifrost as a custom model in Cursor, configure MCP tools, and use virtual keys for team access control."
|
||||
icon: "arrow-pointer"
|
||||
---
|
||||
|
||||
[Cursor](https://cursor.com) is an AI-powered IDE that supports OpenAI-compatible APIs and MCP (Model Context Protocol). By connecting Cursor to Bifrost, you get access to any provider/model in your Bifrost configuration, plus MCP tools and governance features like virtual keys.
|
||||
|
||||
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Cursor, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Open Cursor Settings**
|
||||
|
||||
Press <key>Cmd</key>+<key>,</key> (macOS) or <key>Ctrl</key>+<key>,</key> (Windows/Linux) and navigate to **Models**.
|
||||
|
||||
2. **Enter your API key**
|
||||
|
||||
In the **OpenAI API Key** field, enter your Bifrost virtual key or provider API key.
|
||||
|
||||
3. **Override the base URL**
|
||||
|
||||
Toggle **Override OpenAI Base URL** to ON and enter your Bifrost endpoint:
|
||||
|
||||
<Note>
|
||||
For cursor you need publicly accessible link for Bifrost.
|
||||
</Note>
|
||||
|
||||
For deployed instances, use your Bifrost deployment URL (e.g., `https://bifrost.example.com/cursor`).
|
||||
|
||||
4. **Add custom models** (optional)
|
||||
|
||||
Type a model name in the **Add or search model** field using the `provider/model-name` format:
|
||||
|
||||
|
||||
|
||||
Examples: `anthropic/claude-sonnet-4-5-20250929`, `openai/gpt-5`, `gemini-2.5-pro`
|
||||
|
||||
| Provider | Format | Example |
|
||||
|----------|--------|---------|
|
||||
| Anthropic | `anthropic/model-name` | `anthropic/claude-sonnet-4-5-20250929` |
|
||||
| Gemini | `model-name` | `gemini-2.5-pro` |
|
||||
| OpenAI | `openai/model-name` | `openai/gpt-5` |
|
||||
| Bedrock | `bedrock/model-name` | `bedrock/anthropic.claude-3` |
|
||||
| Vertex (non-Gemini) | `vertex/model-name` | `vertex/text-bison` |
|
||||
| Other providers | `provider/model-name` | `groq/llama-3.3-70b-versatile` |
|
||||
|
||||
## Using Virtual Keys
|
||||
|
||||
Bifrost [Virtual Keys](/features/governance/virtual-keys) can be used as the OpenAI API Key in Cursor. Virtual keys let you enforce budgets, rate limits, and provider access controls for each user or team.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Cursor assigns models to different features — **Chat**, **Agent**, **Inline Edit**, and **Tab Completion**. After configuring Bifrost, you can assign any `provider/model-name` to each feature for optimal cost and performance:
|
||||
|
||||
- Use a powerful model like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for Agent mode
|
||||
- Use a fast model like `groq/llama-3.3-70b-versatile` for Tab completion
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single OpenAI-compatible endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `anthropic`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-native models **must support tool use** for Cursor's agent mode and inline editing to work properly. Models without tool use support will only work for basic chat.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
Cursor's "Override OpenAI Base URL" is a global setting that applies to all OpenAI-compatible models. This works well with Bifrost since Bifrost handles routing to the correct provider based on the model name.
|
||||
</Note>
|
||||
|
||||
## Observability
|
||||
|
||||
All Cursor requests through Bifrost are logged. Monitor them at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content.
|
||||
106
docs/cli-agents/gemini-cli.mdx
Normal file
106
docs/cli-agents/gemini-cli.mdx
Normal file
@@ -0,0 +1,106 @@
|
||||
---
|
||||
title: "Gemini CLI"
|
||||
description: "Use Google's Gemini CLI with Bifrost for advanced reasoning capabilities with any provider."
|
||||
icon: "diamond"
|
||||
---
|
||||
|
||||
[Gemini CLI](https://github.com/google-gemini/gemini-cli) is Google's powerful coding assistant with advanced reasoning capabilities.
|
||||
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Gemini CLI, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## To install Gemini CLI
|
||||
|
||||
```bash
|
||||
npm install -g @google/gemini-cli
|
||||
```
|
||||
|
||||
## Configuring Gemini CLI to work with Bifrost
|
||||
|
||||
Gemini CLI supports multiple authentication methods. Choose the one that matches your account type.
|
||||
|
||||
### Google account (OAuth)
|
||||
|
||||
Log in with your Google account for free-tier access (60 requests/min, 1,000 requests/day).
|
||||
|
||||
1. **Set the Bifrost base URL**
|
||||
```bash
|
||||
export GOOGLE_GEMINI_BASE_URL=http://localhost:8080/genai
|
||||
```
|
||||
|
||||
2. **Run Gemini CLI and sign in**
|
||||
```bash
|
||||
gemini
|
||||
```
|
||||
Select **Login with Google** and authenticate via your browser. All traffic automatically routes through Bifrost.
|
||||
|
||||
### API key based usage
|
||||
|
||||
For users with a Gemini API key (obtain one from [Google AI Studio](https://aistudio.google.com/apikey)):
|
||||
|
||||
1. **Configure environment variables**
|
||||
```bash
|
||||
export GEMINI_API_KEY=your-api-key # Gemini API key or Bifrost virtual key
|
||||
export GOOGLE_GEMINI_BASE_URL=http://localhost:8080/genai
|
||||
```
|
||||
|
||||
2. **Run Gemini CLI**
|
||||
```bash
|
||||
gemini
|
||||
```
|
||||
Select **Use Gemini API Key** in the CLI prompt for authentication.
|
||||
|
||||
|
||||
|
||||
### Google Cloud / Vertex AI
|
||||
|
||||
For enterprise users with Vertex AI access:
|
||||
|
||||
```bash
|
||||
export GOOGLE_API_KEY=your-api-key # Google API key or Bifrost virtual key
|
||||
export GOOGLE_GENAI_USE_VERTEXAI=true
|
||||
export GOOGLE_GEMINI_BASE_URL=http://localhost:8080/genai
|
||||
gemini
|
||||
```
|
||||
|
||||
<Tip>
|
||||
For paid Code Assist License users, set your Google Cloud project: `export GOOGLE_CLOUD_PROJECT="your-project-id"`
|
||||
</Tip>
|
||||
|
||||
Now all Gemini CLI traffic flows through Bifrost, giving you access to any provider/model configured in your Bifrost setup, plus observability and governance.
|
||||
|
||||
## Model Configuration
|
||||
|
||||
Use the `-m` flag to start Gemini CLI with a specific model:
|
||||
|
||||
```bash
|
||||
gemini -m gemini-2.5-flash
|
||||
gemini -m gemini-2.5-pro
|
||||
```
|
||||
|
||||
## Using Non-Google Models with Gemini CLI
|
||||
|
||||
Bifrost automatically translates GenAI API requests to other providers, so you can use Gemini CLI with models from OpenAI, Anthropic, Mistral, and more. Use the `provider/model-name` format to specify any Bifrost-configured model.
|
||||
|
||||
```bash
|
||||
# Start with an OpenAI model
|
||||
gemini -m openai/gpt-5
|
||||
|
||||
# Start with an Anthropic model
|
||||
gemini -m anthropic/claude-sonnet-4-5-20250929
|
||||
|
||||
# Start with a Groq model
|
||||
gemini -m groq/llama-3.3-70b-versatile
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-Google models **must support tool use** for Gemini CLI to work properly. Gemini CLI relies on tool calling for file operations, terminal commands, and code editing. Models without tool use support will fail on most operations.
|
||||
</Warning>
|
||||
130
docs/cli-agents/librechat.mdx
Normal file
130
docs/cli-agents/librechat.mdx
Normal file
@@ -0,0 +1,130 @@
|
||||
---
|
||||
title: "LibreChat"
|
||||
description: "Integrate LibreChat with Bifrost to access any AI provider through a modern open-source chat interface with virtual keys and observability."
|
||||
icon: "message"
|
||||
---
|
||||
|
||||
[LibreChat](https://github.com/danny-avila/LibreChat) is a modern, open-source chat client that supports multiple AI providers. By adding Bifrost as a custom provider, you get access to any model configured in Bifrost through a familiar chat interface, plus governance features like virtual keys and built-in observability.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with LibreChat, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install LibreChat
|
||||
|
||||
Follow the [LibreChat documentation](https://www.librechat.ai/docs/local) for local setup. There are multiple installation options (Docker, npm, etc.).
|
||||
|
||||
### 2. Add Bifrost as a Custom Provider
|
||||
|
||||
Add the following to your `librechat.yaml` file:
|
||||
|
||||
```yaml
|
||||
custom:
|
||||
- name: "Bifrost"
|
||||
apiKey: "dummy"
|
||||
baseURL: "http://localhost:8080/v1"
|
||||
models:
|
||||
default: ["openai/gpt-4o"]
|
||||
fetch: true
|
||||
titleConvo: true
|
||||
titleModel: "openai/gpt-4o"
|
||||
summarize: false
|
||||
summaryModel: "openai/gpt-4o"
|
||||
forcePrompt: false
|
||||
modelDisplayLabel: "Bifrost"
|
||||
iconURL: https://getbifrost.ai/bifrost-logo.png
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `apiKey` | Bifrost virtual key if authentication is enabled; use `dummy` otherwise |
|
||||
| `baseURL` | Bifrost gateway URL + `/v1` (LibreChat uses OpenAI format) |
|
||||
| `models.default` | Default models to show. Use Bifrost model IDs (`provider/model`) |
|
||||
| `models.fetch` | Set `true` to fetch available models from Bifrost |
|
||||
| `titleConvo` | Use AI for conversation title generation |
|
||||
| `titleModel` | Model for title generation |
|
||||
| `summarize` | Enable chat summary generation |
|
||||
| `summaryModel` | Model for summaries |
|
||||
|
||||
<Tip>
|
||||
Set `models.fetch: true` to automatically discover all models configured in Bifrost. This keeps your LibreChat model list in sync with your Bifrost provider configuration.
|
||||
</Tip>
|
||||
|
||||
<Note>
|
||||
If you're running LibreChat in Docker, it does not automatically use `librechat.yaml`. See [Step 1 of the LibreChat custom endpoints guide](https://www.librechat.ai/docs/quick_start/custom_endpoints#step-1-create-or-edit-a-docker-override-file) for how to mount or override the config.
|
||||
</Note>
|
||||
|
||||
### 3. Docker Networking
|
||||
|
||||
Choose the correct `baseURL` for your setup:
|
||||
|
||||
| Setup | baseURL |
|
||||
|-------|---------|
|
||||
| LibreChat and Bifrost on same host | `http://localhost:8080/v1` |
|
||||
| LibreChat in Docker Desktop, Bifrost on host | `http://host.docker.internal:8080/v1` |
|
||||
| LibreChat in Docker Engine (Linux), Bifrost on host | Add `--add-host=host.docker.internal:host-gateway` to `docker run`, or `extra_hosts: ["host.docker.internal:host-gateway"]` in Compose, then use `http://host.docker.internal:8080/v1` |
|
||||
| Both in same Docker network | `http://bifrost-container-name:8080/v1` |
|
||||
|
||||
### 4. Run LibreChat
|
||||
|
||||
Start LibreChat. Bifrost will appear as a provider with all configured models available.
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, set `apiKey` to your virtual key:
|
||||
|
||||
```yaml
|
||||
apiKey: "bf-your-virtual-key-here"
|
||||
```
|
||||
|
||||
This lets you enforce usage limits, budgets, and access control per user or team. For team deployments, create a separate virtual key for each team or environment — each key can have its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
LibreChat displays models from the `models.default` list or fetches them from Bifrost when `models.fetch` is enabled. Use Bifrost model IDs in `provider/model` format to access any configured provider:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
default:
|
||||
- "openai/gpt-5"
|
||||
- "anthropic/claude-sonnet-4-5-20250929"
|
||||
- "gemini/gemini-2.5-pro"
|
||||
- "groq/llama-3.3-70b-versatile"
|
||||
fetch: true
|
||||
```
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex conversations
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick responses
|
||||
- Set `titleModel` and `summaryModel` to lighter models to reduce cost for metadata generation
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single `/v1` endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Note>
|
||||
LibreChat connects to Bifrost via a single OpenAI-compatible endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed in LibreChat.
|
||||
</Note>
|
||||
|
||||
## Observability
|
||||
|
||||
All LibreChat traffic through Bifrost is logged. Monitor it at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage across your team.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
139
docs/cli-agents/open-webui.mdx
Normal file
139
docs/cli-agents/open-webui.mdx
Normal file
@@ -0,0 +1,139 @@
|
||||
---
|
||||
title: "Open WebUI"
|
||||
description: "Integrate Open WebUI with Bifrost to access any AI provider through a modern open-source chat interface with virtual keys and observability."
|
||||
icon: "globe"
|
||||
---
|
||||
|
||||
[Open WebUI](https://github.com/open-webui/open-webui) is a modern, open-source chat interface that supports OpenAI-compatible APIs. By adding Bifrost as a connection, you get access to any model configured in Bifrost through a familiar ChatGPT-like interface, plus governance features like virtual keys and built-in observability.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Open WebUI, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install Open WebUI
|
||||
|
||||
Follow the [Open WebUI documentation](https://docs.openwebui.com/getting-started/installation/) for installation. Open WebUI can run via Docker, Docker Compose, or Kubernetes.
|
||||
|
||||
### 2. Add Bifrost as a Connection
|
||||
|
||||
<Tip>
|
||||
If running Open WebUI in Docker and Bifrost is on the host machine, use `http://host.docker.internal:8080/v1` instead of `localhost`.
|
||||
</Tip>
|
||||
|
||||
1. Open Open WebUI in your browser
|
||||
2. Go to **⚙️ Admin Settings** → **Connections** → **OpenAI**
|
||||
3. Click **➕ Add Connection**
|
||||
4. Configure the following:
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **URL** | `http://localhost:8080/v1` (or your Bifrost host, e.g. `https://bifrost.yourcompany.com/v1`) |
|
||||
| **API Key** | Your Bifrost virtual key if authentication is enabled; otherwise leave empty or use `dummy` |
|
||||
|
||||
5. Click **Save**
|
||||
|
||||
|
||||
|
||||
### 3. Model Discovery
|
||||
|
||||
Open WebUI fetches available models from Bifrost's `/v1/models` endpoint. If auto-detection fails or you want to filter which models appear, add model IDs to the **Model IDs (Filter)** allowlist in the connection settings. Use Bifrost model IDs in `provider/model` format (e.g. `openai/gpt-5`, `anthropic/claude-sonnet-4-5-20250929`).
|
||||
|
||||
|
||||
|
||||
### 4. Start Chatting
|
||||
|
||||
Select your Bifrost connection's model from the chat model selector and start chatting.
|
||||
|
||||
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, set **API Key** in the connection to your virtual key. This lets you enforce usage limits, budgets, and access control per user or team.
|
||||
|
||||
For team deployments, create separate Open WebUI connections (or use different API keys per connection) — each virtual key can have its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Open WebUI displays models fetched from Bifrost or those you add to the Model IDs allowlist. Use Bifrost model IDs in `provider/model` format to access any configured provider:
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex conversations
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick responses
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single `/v1` endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Note>
|
||||
Open WebUI connects to Bifrost via a single OpenAI-compatible endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed in Open WebUI.
|
||||
</Note>
|
||||
|
||||
## Multimodality
|
||||
|
||||
Open WebUI supports image generation and vision (image understanding). You can use Bifrost for both.
|
||||
|
||||
### Image Generation
|
||||
|
||||
Set a Bifrost provider/model as your **image inference engine** for DALL·E-style image generation:
|
||||
|
||||
1. Go to **⚙️ Admin Settings** → **Settings** → **Images**
|
||||
2. Set **Image Generation Engine** to **Open AI**
|
||||
3. Configure:
|
||||
- **API Endpoint URL**: `http://localhost:8080/v1` (or your Bifrost host + `/v1`)
|
||||
- **API Key**: Your Bifrost virtual key if authentication is enabled
|
||||
- **Model**: Bifrost model ID in `provider/model` format (e.g. `openai/dall-e-3`, `openai/gpt-image-1`)
|
||||
|
||||
Bifrost routes image generation requests to the configured provider. Use any image-capable model in your Bifrost configuration (OpenAI DALL·E, GPT-Image, or other providers that support `/v1/images/generations`).
|
||||
|
||||
|
||||
|
||||
### Vision (Image Understanding)
|
||||
|
||||
Chat models that support vision (e.g. `openai/gpt-4o`, `anthropic/claude-sonnet-4-5`) work through your main Bifrost connection. When you select a vision-capable model in the chat selector, you can attach images to your messages — Open WebUI sends them to Bifrost, which routes to the correct provider.
|
||||
|
||||
## Docker Networking
|
||||
|
||||
Choose the correct URL for your setup:
|
||||
|
||||
| Setup | URL |
|
||||
|-------|-----|
|
||||
| Open WebUI and Bifrost on same host | `http://localhost:8080/v1` |
|
||||
| Open WebUI in Docker, Bifrost on host | `http://host.docker.internal:8080/v1` |
|
||||
| Both in same Docker network | `http://bifrost-container-name:8080/v1` |
|
||||
|
||||
## Environment Variables (Alternative)
|
||||
|
||||
You can also configure Bifrost via environment variables when running Open WebUI:
|
||||
|
||||
```bash
|
||||
# Single connection
|
||||
OPENAI_API_BASE_URLS="http://localhost:8080/v1"
|
||||
OPENAI_API_KEYS="your-bifrost-virtual-key"
|
||||
|
||||
# Multiple connections (semicolon-separated)
|
||||
OPENAI_API_BASE_URLS="http://localhost:8080/v1;https://other-gateway.com/v1"
|
||||
OPENAI_API_KEYS="key1;key2"
|
||||
```
|
||||
|
||||
## Observability
|
||||
|
||||
All Open WebUI traffic through Bifrost is logged. Monitor it at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage across your team.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
176
docs/cli-agents/opencode.mdx
Normal file
176
docs/cli-agents/opencode.mdx
Normal file
@@ -0,0 +1,176 @@
|
||||
---
|
||||
title: "Opencode"
|
||||
description: "Use Opencode with Bifrost to access any AI provider through a terminal-based coding assistant with virtual keys and observability."
|
||||
icon: "codepen"
|
||||
---
|
||||
|
||||
[Opencode](https://github.com/sst/opencode) is an AI-powered coding application that supports OpenAI-compatible APIs. By pointing it at Bifrost, you get access to any provider/model in your Bifrost configuration, plus governance features like virtual keys, built-in observability, and per-model options for reasoning effort, thinking budget, and more.
|
||||
|
||||
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with OpenCode, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Configure OpenCode to work with Bifrost
|
||||
|
||||
OpenCode uses a JSON config file (`opencode.json`) to configure providers. Point your provider's `baseURL` to Bifrost.
|
||||
|
||||
### Using OpenAI-compatible endpoint
|
||||
|
||||
Route OpenAI and other providers through Bifrost's OpenAI endpoint:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"openai": {
|
||||
"name": "Bifrost",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8080/openai",
|
||||
"apiKey": "your-bifrost-key"
|
||||
},
|
||||
"models": {
|
||||
"openai/gpt-5": {},
|
||||
"anthropic/claude-sonnet-4-5-20250929": {},
|
||||
"gemini/gemini-2.5-pro": {}
|
||||
}
|
||||
}
|
||||
},
|
||||
"model": "openai/gpt-5"
|
||||
}
|
||||
```
|
||||
|
||||
### Using Anthropic endpoint
|
||||
|
||||
Route Anthropic models through Bifrost's Anthropic endpoint:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"provider": {
|
||||
"anthropic": {
|
||||
"name": "Bifrost",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8080/anthropic/v1",
|
||||
"apiKey": "your-bifrost-key"
|
||||
},
|
||||
"models": {
|
||||
"anthropic/claude-sonnet-4-5-20250929": {}
|
||||
}
|
||||
}
|
||||
},
|
||||
"model": "anthropic/claude-sonnet-4-5-20250929"
|
||||
}
|
||||
```
|
||||
|
||||
<Tip>
|
||||
You can also use the `/connect` command in the OpenCode TUI to configure credentials interactively, then update the `baseURL` in your config file.
|
||||
</Tip>
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, set `apiKey` in your provider options to your virtual key:
|
||||
|
||||
```jsonc
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8080/openai",
|
||||
"apiKey": "bf-your-virtual-key-here"
|
||||
}
|
||||
```
|
||||
|
||||
This lets you enforce usage limits, budgets, and access control per user or environment. For team deployments, create a separate virtual key for each team — each key can have its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Set your default models in `opencode.json`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"model": "openai/gpt-5",
|
||||
"small_model": "anthropic/claude-haiku-4-5"
|
||||
}
|
||||
```
|
||||
|
||||
Switch models in the TUI with <key>ctrl</key>+<key>p</key>
|
||||
|
||||
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex coding tasks
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick completions
|
||||
- Set `small_model` to a lighter model for faster, lower-cost operations
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single OpenAI endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
You can configure models from different providers with per-model options:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"$schema": "https://opencode.ai/config.json",
|
||||
"theme": "opencode",
|
||||
"autoupdate": true,
|
||||
"provider": {
|
||||
"openai": {
|
||||
"name": "Bifrost",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8080/openai",
|
||||
"apiKey": "your-bifrost-key"
|
||||
},
|
||||
"models": {
|
||||
"openai/gpt-5": {
|
||||
"options": {
|
||||
"reasoningEffort": "high",
|
||||
"textVerbosity": "low",
|
||||
"reasoningSummary": "auto",
|
||||
"include": [
|
||||
"reasoning.encrypted_content"
|
||||
]
|
||||
}
|
||||
},
|
||||
"anthropic/claude-sonnet-4-5-20250929": {
|
||||
"options": {
|
||||
"thinking": {
|
||||
"type": "enabled",
|
||||
"budgetTokens": 16000
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-native models **must support tool use** for OpenCode to work properly. OpenCode relies on tool calling for file operations, terminal commands, and code editing. Models without tool use support will fail on most operations.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
OpenCode connects to Bifrost via a single endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed.
|
||||
</Note>
|
||||
|
||||
## Observability
|
||||
|
||||
All OpenCode traffic through Bifrost is logged. Monitor it at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
135
docs/cli-agents/overview.mdx
Normal file
135
docs/cli-agents/overview.mdx
Normal file
@@ -0,0 +1,135 @@
|
||||
---
|
||||
title: "Overview"
|
||||
description: "Use Bifrost with LibreChat, Claude Code, Codex CLI, Gemini CLI, Qwen Code, and more by pointing each tool at the correct Bifrost endpoint."
|
||||
icon: "Book"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Bifrost provides **100% compatible endpoints** for OpenAI, Anthropic, and Gemini APIs, making it seamless to integrate with any agent that uses these providers. Point each agent at the Bifrost URL shape it expects (see [Configuration](#configuration) below and the guide for your tool). That unlocks:
|
||||
|
||||
- **Universal Model Access**: Use **any provider/model** configured in Bifrost with any agent (e.g., use GPT-5 with Claude Code, or Claude Sonnet 4.5 with Codex CLI)
|
||||
- **MCP Tools Integration**: All Model Context Protocol tools configured in Bifrost become available to your agents
|
||||
- **Built-in Observability**: Monitor all agent interactions in real-time through Bifrost's logging dashboard
|
||||
- **Load Balancing**: Automatically distribute requests across multiple providers and regions
|
||||
- **Advanced Features**: Governance, caching, failover, and more - all transparent to your agent
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with your CLI agent, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## CLI Agents
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="LibreChat" icon="comments" href="/cli-agents/librechat">
|
||||
Open-source chat client supporting multiple providers
|
||||
</Card>
|
||||
<Card title="Open WebUI" icon="globe" href="/cli-agents/open-webui">
|
||||
ChatGPT-like open-source chat interface
|
||||
</Card>
|
||||
<Card title="Claude Code" icon="terminal" href="/cli-agents/claude-code">
|
||||
AI-powered coding directly in your terminal
|
||||
</Card>
|
||||
<Card title="Claude for Office" icon="file-word" href="/cli-agents/claude-for-office">
|
||||
Claude in Word, Excel, PowerPoint, and Outlook
|
||||
</Card>
|
||||
<Card title="Claude Desktop" icon="desktop" href="/cli-agents/claude-desktop">
|
||||
Route Claude Desktop App's Code tab through any provider
|
||||
</Card>
|
||||
<Card title="Codex CLI" icon="code" href="/cli-agents/codex-cli">
|
||||
OpenAI's powerful code generation CLI
|
||||
</Card>
|
||||
<Card title="Gemini CLI" icon="diamond" href="/cli-agents/gemini-cli">
|
||||
Google's coding assistant with advanced reasoning
|
||||
</Card>
|
||||
<Card title="Qwen Code" icon="q" href="/cli-agents/qwen-code">
|
||||
Alibaba's coding assistant
|
||||
</Card>
|
||||
<Card title="Opencode" icon="square-code" href="/cli-agents/opencode">
|
||||
Terminal-based coding assistant by SST
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Editors
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="Cursor" icon="mouse-pointer" href="/cli-agents/cursor">
|
||||
AI-powered IDE with MCP integration and virtual key support
|
||||
</Card>
|
||||
<Card title="Zed Editor" icon="window" href="/cli-agents/zed-editor">
|
||||
High-performance editor with AI integration
|
||||
</Card>
|
||||
<Card title="Roo Code" icon="code" href="/cli-agents/roo-code">
|
||||
AI-powered VS Code extension with multi-provider support
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Configuration
|
||||
|
||||
<Note>
|
||||
**OpenAI-compatible URL shape** varies by client: tools such as **Codex CLI** need the API base to end with **`/v1`** (e.g. `https://your-gateway/openai/v1`). Official OpenAI SDKs often take `https://your-gateway/openai` and add `/v1` to request paths. Use the page for your agent to avoid 404s or wrong-host errors.
|
||||
</Note>
|
||||
|
||||
Agent integrations work with your existing Bifrost configuration. Ensure you have:
|
||||
|
||||
- **Providers configured**: See [Provider Configuration](/quickstart/gateway/provider-configuration) for setup details
|
||||
- **Optional: MCP tools**: See [MCP Integration](/mcp/overview) to enhance agent capabilities
|
||||
|
||||
## Monitoring Agent Traffic
|
||||
|
||||
All agent interactions are automatically logged and can be monitored at `http://localhost:8080/logs`. You can filter by provider, model, or search through conversation content to track your agents' performance.
|
||||
|
||||
|
||||
For complete monitoring capabilities, see [Built-in Observability](/features/observability/default).
|
||||
|
||||
## MCP Tools Integration
|
||||
|
||||
<Note>
|
||||
This feature is only available on `v1.4.0-prerelease1` and above.
|
||||
</Note>
|
||||
|
||||
Bifrost exposes all configured MCP tools through a dedicated MCP server endpoint at `/mcp`. This allows external MCP clients (Claude Code, Cursor, and other MCP-compatible applications) to connect directly and access your tools.
|
||||
|
||||
### Connect Claude Code to Bifrost MCP
|
||||
|
||||
Add Bifrost as an MCP server to Claude Code:
|
||||
|
||||
```bash
|
||||
claude mcp add --transport http bifrost http://localhost:8080/mcp
|
||||
```
|
||||
|
||||
**Using Virtual Key Authentication:**
|
||||
|
||||
If you have virtual key authentication enabled in Bifrost, connect using the JSON configuration format:
|
||||
|
||||
```bash
|
||||
claude mcp add-json bifrost '{"type":"http","url":"http://localhost:8080/mcp","headers":{"Authorization":"Bearer bf-virtual-key"}}'
|
||||
```
|
||||
|
||||
Replace `bf-virtual-key` with your actual Bifrost virtual key.
|
||||
|
||||
Claude Code will only have access to the specific MCP tools permitted by the virtual key's configuration. To grant access to additional tools, verify or modify the virtual key's MCP tool permissions in the Bifrost dashboard.
|
||||
|
||||
### Supported Agents
|
||||
|
||||
You can connect any MCP-compatible client to Bifrost's MCP server:
|
||||
|
||||
- **Claude Code** - Add via `claude mcp add` command
|
||||
- **Cursor IDE** - Configure in cursor settings
|
||||
- **Custom Applications** - Any app implementing the MCP protocol
|
||||
|
||||
### Tool Access Control
|
||||
|
||||
Control which tools each client can access using Virtual Keys:
|
||||
|
||||
- Create a Virtual Key for each client/environment
|
||||
- Configure which MCP servers and tools the key can access
|
||||
- Bifrost automatically enforces these permissions
|
||||
|
||||
For complete setup instructions and tool filtering options, see [MCP Gateway URL](/mcp/gateway-url).
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[Provider Configuration](/quickstart/gateway/provider-configuration)** - Configure AI providers for your agents
|
||||
- **[Governance](/features/governance/virtual-keys)** - Set usage limits and policies for your agents
|
||||
- **[Integrations](/integrations/what-is-an-integration)** - Understand how Bifrost works with existing AI provider SDKs
|
||||
130
docs/cli-agents/qwen-code.mdx
Normal file
130
docs/cli-agents/qwen-code.mdx
Normal file
@@ -0,0 +1,130 @@
|
||||
---
|
||||
title: "Qwen Code"
|
||||
description: "Use Alibaba's Qwen Code with Bifrost for AI-powered coding with any provider, virtual keys, and observability."
|
||||
icon: "q"
|
||||
---
|
||||
|
||||
[Qwen Code](https://github.com/QwenLM/qwen-code) is Alibaba's powerful coding assistant with advanced reasoning capabilities. By connecting it to Bifrost, you get access to any provider/model in your Bifrost configuration, plus governance features like virtual keys and built-in observability.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Qwen Code, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install Qwen Code
|
||||
|
||||
```bash
|
||||
npm install -g @qwen-code/qwen-code
|
||||
```
|
||||
|
||||
### 2. Configure Bifrost as a Model Provider
|
||||
|
||||
Qwen Code uses `~/.qwen/settings.json` to configure model providers. Add Bifrost as an OpenAI-compatible provider:
|
||||
|
||||
```json
|
||||
{
|
||||
"modelProviders": {
|
||||
"openai": [
|
||||
{
|
||||
"id": "openai/gpt-5",
|
||||
"name": "GPT-5 (via Bifrost)",
|
||||
"baseUrl": "http://localhost:8080/openai",
|
||||
"envKey": "OPENAI_API_KEY"
|
||||
},
|
||||
{
|
||||
"id": "anthropic/claude-sonnet-4-5-20250929",
|
||||
"name": "Claude Sonnet 4.5 (via Bifrost)",
|
||||
"baseUrl": "http://localhost:8080/openai",
|
||||
"envKey": "OPENAI_API_KEY"
|
||||
}
|
||||
]
|
||||
},
|
||||
"security": {
|
||||
"auth": {
|
||||
"selectedType": "openai"
|
||||
}
|
||||
},
|
||||
"model": {
|
||||
"name": "openai/gpt-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Set Your API Key
|
||||
|
||||
Set the `OPENAI_API_KEY` environment variable to your Bifrost virtual key or provider API key:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY=your-bifrost-virtual-key
|
||||
```
|
||||
|
||||
Alternatively, use a `.env` file in `~/.qwen/.env`:
|
||||
|
||||
```
|
||||
OPENAI_API_KEY=your-bifrost-virtual-key
|
||||
```
|
||||
|
||||
### 4. Run Qwen Code
|
||||
|
||||
```bash
|
||||
qwen
|
||||
```
|
||||
|
||||
<Tip>
|
||||
You can also use environment variables for a minimal setup. Set `OPENAI_BASE_URL` and `OPENAI_API_KEY`, then run `qwen`. Note: some Qwen Code versions prioritize `settings.json` over env vars — use `settings.json` for reliable configuration.
|
||||
</Tip>
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, set `OPENAI_API_KEY` to your virtual key. This lets you enforce usage limits, budgets, and access control per user or environment.
|
||||
|
||||
For team deployments, create a separate virtual key for each team — each key can have its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Use the `/model` command to switch between models at runtime. All models configured in your `modelProviders` appear in the picker.
|
||||
|
||||
```bash
|
||||
# Launch with a specific model
|
||||
qwen --model "anthropic/claude-sonnet-4-5-20250929"
|
||||
```
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex coding tasks
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick completions
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format in your `modelProviders` config to access any configured provider through the single OpenAI endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
Add multiple models to your `modelProviders.openai` array — they all use the same Bifrost `baseUrl` and `envKey`.
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-native models **must support tool use** for Qwen Code to work properly. Qwen Code relies on tool calling for file operations, terminal commands, and code editing. Models without tool use support will fail on most operations.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
Qwen Code connects to Bifrost via a single OpenAI-compatible endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed.
|
||||
</Note>
|
||||
|
||||
## Observability
|
||||
|
||||
All Qwen Code traffic through Bifrost is logged. Monitor it at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
94
docs/cli-agents/roo-code.mdx
Normal file
94
docs/cli-agents/roo-code.mdx
Normal file
@@ -0,0 +1,94 @@
|
||||
---
|
||||
title: "Roo Code"
|
||||
description: "Use Roo Code with Bifrost to access any AI provider through a powerful VS Code extension with virtual keys and observability."
|
||||
icon: "code"
|
||||
---
|
||||
|
||||
[Roo Code](https://roo-code.net/) is an AI-powered VS Code extension that supports OpenAI-compatible APIs. By connecting it to Bifrost, you get access to any provider/model in your Bifrost configuration, plus governance features like virtual keys and built-in observability.
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Roo Code, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Install Roo Code
|
||||
|
||||
Install the [Roo Code extension](https://marketplace.visualstudio.com/items?itemName=RooCodeInc.roo-code) from the VS Code marketplace.
|
||||
|
||||
### 2. Create an API Configuration Profile
|
||||
|
||||
1. Open **Settings** (click the gear icon) → **Providers**
|
||||
2. Click the **+** button to create a new profile
|
||||
3. Select **OpenAI Compatible** as the provider
|
||||
4. Configure the following:
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Base URL** | `http://localhost:8080/openai` (or your Bifrost host, e.g. `https://bifrost.yourcompany.com/openai`) |
|
||||
| **API Key** | Your Bifrost virtual key if authentication is enabled; otherwise use `dummy` or leave empty |
|
||||
| **Model** | Bifrost model ID in `provider/model` format (e.g. `anthropic/claude-sonnet-4-5-20250929`, `openai/gpt-5`) |
|
||||
|
||||
|
||||
|
||||
### 3. Verify the Connection
|
||||
|
||||
Ask Roo which model it's using — it should respond with the Bifrost model ID you configured (e.g. `anthropic/claude-sonnet-4-5`).
|
||||
|
||||
|
||||
|
||||
<Tip>
|
||||
Roo Code supports multiple API configuration profiles. Create separate profiles for different Bifrost virtual keys or model combinations, then switch between them via the profile dropdown in Settings or during chat.
|
||||
</Tip>
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, set **API Key** in your Roo Code profile to your virtual key. This lets you enforce usage limits, budgets, and access control per user or team.
|
||||
|
||||
For team deployments, create a separate configuration profile for each team — each can use a different virtual key with its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Roo Code lets you assign models per mode (Code, Ask, Architect, Debug, Orchestrator). Use Bifrost model IDs in `provider/model` format to access any configured provider:
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex coding tasks
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick completions
|
||||
- Link different profiles to different modes in the **Prompts** tab for optimal cost and performance
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single OpenAI-compatible endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Roo Code requires **native tool calling** (OpenAI-compatible function calling). Models without tool use support cannot be used with Roo Code. Ensure the model you select supports tool calling.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
Roo Code connects to Bifrost via a single OpenAI-compatible endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed.
|
||||
</Note>
|
||||
|
||||
## MCP Server Integration
|
||||
|
||||
Roo Code supports MCP (Model Context Protocol). You can connect it to Bifrost's MCP server to access all tools configured in Bifrost. See [MCP Gateway URL](/mcp/gateway-url) for setup instructions.
|
||||
|
||||
## Observability
|
||||
|
||||
All Roo Code traffic through Bifrost is logged. Monitor it at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
136
docs/cli-agents/zed-editor.mdx
Normal file
136
docs/cli-agents/zed-editor.mdx
Normal file
@@ -0,0 +1,136 @@
|
||||
---
|
||||
title: "Zed Editor"
|
||||
description: "Integrate Zed editor with Bifrost to use any AI provider for code assistance with virtual keys and observability."
|
||||
icon: "z"
|
||||
---
|
||||
|
||||
[Zed](https://zed.dev/) is a high-performance editor with built-in AI assistant support. It can connect to any OpenAI-compatible API, making Bifrost a natural fit for universal model access across providers, plus governance features like virtual keys and built-in observability.
|
||||
|
||||
|
||||
|
||||
<Note>
|
||||
If your Allowed Headers are already set to `*`, you can skip this note. If not and you face issues integrating Bifrost with Zed, try switching to `*` or adding the specific headers required by your client. By default, Bifrost whitelists: `Content-Type`, `Authorization`, `X-Requested-With`, `X-Stainless-Timeout`, and `X-Api-Key`.
|
||||
</Note>
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Configure Bifrost Provider
|
||||
|
||||
Add Bifrost to Zed's `language_models.openai_compatible` configuration. This is typically in your Zed settings (JSON) or workspace config.
|
||||
|
||||
```json
|
||||
"language_models": {
|
||||
"openai_compatible": {
|
||||
"Bifrost": {
|
||||
"api_url": "http://localhost:8080/openai",
|
||||
"available_models": [
|
||||
{
|
||||
"name": "anthropic/claude-sonnet-4.5",
|
||||
"max_tokens": 200000,
|
||||
"max_output_tokens": 4096,
|
||||
"capabilities": {
|
||||
"tools": true,
|
||||
"images": true,
|
||||
"parallel_tool_calls": true,
|
||||
"prompt_cache_key": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "openai/gpt-4o",
|
||||
"max_tokens": 128000,
|
||||
"max_output_tokens": 4096,
|
||||
"capabilities": {
|
||||
"tools": true,
|
||||
"images": true,
|
||||
"parallel_tool_calls": true,
|
||||
"prompt_cache_key": false
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "openai/gpt-5",
|
||||
"max_tokens": 256000,
|
||||
"max_output_tokens": 4096,
|
||||
"capabilities": {
|
||||
"tools": true,
|
||||
"images": true,
|
||||
"parallel_tool_calls": true,
|
||||
"prompt_cache_key": false
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Replace `http://localhost:8080/openai` with your Bifrost gateway URL + `/openai`.
|
||||
|
||||
### 2. Model Capabilities
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `tools` | Enable tool/function calling |
|
||||
| `images` | Enable image input (vision) |
|
||||
| `parallel_tool_calls` | Support multiple tool calls in one response |
|
||||
| `prompt_cache_key` | Enable prompt caching (set `false` if not supported) |
|
||||
|
||||
Use Bifrost model IDs in `provider/model` format (e.g. `openai/gpt-5`, `anthropic/claude-sonnet-4.5`). Ensure these models are configured in Bifrost.
|
||||
|
||||
### 3. Reload Workspace
|
||||
|
||||
After changing the configuration, reload the workspace so Zed recognizes and reloads the provider list.
|
||||
|
||||
## Virtual Keys
|
||||
|
||||
When Bifrost has [virtual key authentication](/features/governance/virtual-keys) enabled, add an `api_key` field to the Bifrost provider config (check Zed's documentation for the exact field name — it may vary by version):
|
||||
|
||||
```json
|
||||
"Bifrost": {
|
||||
"api_url": "http://localhost:8080/openai",
|
||||
"api_key": "bf-your-virtual-key-here",
|
||||
"available_models": [...]
|
||||
}
|
||||
```
|
||||
|
||||
This lets you enforce usage limits, budgets, and access control per user or team. For team deployments, create a separate virtual key for each team or environment — each key can have its own rate limits, budgets, and provider access rules configured in the Bifrost dashboard.
|
||||
|
||||
## Model Selection
|
||||
|
||||
Zed lets you assign models to different AI features. Use Bifrost model IDs in `provider/model` format to access any configured provider:
|
||||
|
||||
- Use powerful models like `openai/gpt-5` or `anthropic/claude-sonnet-4-5-20250929` for complex code generation and refactoring
|
||||
- Use fast models like `groq/llama-3.3-70b-versatile` for quick completions and inline suggestions
|
||||
|
||||
## Using Multiple Providers
|
||||
|
||||
Bifrost routes requests to the correct provider based on the model name. Use the `provider/model-name` format to access any configured provider through the single OpenAI-compatible endpoint:
|
||||
|
||||
```
|
||||
anthropic/claude-sonnet-4-5-20250929
|
||||
openai/gpt-5
|
||||
gemini/gemini-2.5-pro
|
||||
mistral/mistral-large-latest
|
||||
```
|
||||
|
||||
### Supported Providers
|
||||
|
||||
Bifrost supports the following providers with the `provider/model-name` format:
|
||||
|
||||
`openai`, `azure`, `gemini`, `vertex`, `bedrock`, `mistral`, `groq`, `cerebras`, `cohere`, `perplexity`, `xai`, `ollama`, `openrouter`, `huggingface`, `nebius`, `parasail`, `replicate`, `vllm`, `sgl`
|
||||
|
||||
<Warning>
|
||||
Non-native models **must support tool use** for Zed's AI features (code actions, refactoring) to work properly. Models without tool use support will only work for basic chat and completions.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
Zed connects to Bifrost via a single OpenAI-compatible endpoint. Bifrost handles routing to the correct provider based on the model name — no per-provider configuration needed.
|
||||
</Note>
|
||||
|
||||
## Observability
|
||||
|
||||
All Zed requests through Bifrost are logged. Monitor them at `http://localhost:8080/logs` — filter by provider, model, or search through conversation content to track usage.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- [Provider Configuration](/quickstart/gateway/provider-configuration) — Configure AI providers in Bifrost
|
||||
- [Virtual Keys](/features/governance/virtual-keys) — Set up usage limits and access control
|
||||
Reference in New Issue
Block a user