beyhan/bifrost
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Beyhan Oğur
2026-04-26 21:52:23 +03:00
commit 880f412e2c
2662 changed files with 866266 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

618
docs/integrations/anthropic-sdk/files-and-batch.mdx Normal file
View File

@@ -0,0 +1,618 @@
---
title: "Files and Batch API"
tag: "Beta"
description: "Upload files and create batch jobs for asynchronous processing using the Anthropic SDK through Bifrost across multiple providers."
icon: "folder-open"
---
## Overview
Bifrost supports the Anthropic Files API and Batch API (via the `beta` namespace) with **cross-provider routing**. This means you can use the Anthropic SDK to manage files and batch jobs across multiple providers including Anthropic, OpenAI, and Gemini.
The provider is specified using the `x-model-provider` header in `default_headers`.
<Note>
**Bedrock Limitation:** Bedrock batch operations require file-based input with S3 storage, which is not supported via the Anthropic SDK's inline batch API. For Bedrock batch operations, use the [Bedrock SDK](../bedrock-sdk/files-and-batch) directly.
</Note>
---
## Client Setup
<Note>
In API Key section, you can either send virtual key or a dummy key to escape client side validation.
</Note>
### Anthropic Provider (Default)
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
```
### Cross-Provider Client
To route requests to a different provider, set the `x-model-provider` header:
<Tabs group="provider">
<Tab title="OpenAI Provider">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
```
</Tab>
<Tab title="Bedrock Provider">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "bedrock"}
)
```
<Warning>
Bedrock can be used for chat completions via the Anthropic SDK, but **batch operations are not supported**. Bedrock requires file-based batch input with S3 storage. Use the [Bedrock SDK](../bedrock-sdk/files-and-batch) for batch operations.
</Warning>
</Tab>
<Tab title="Gemini Provider">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "gemini"}
)
```
</Tab>
</Tabs>
---
## Files API
The Files API is accessed through the `beta.files` namespace. Note that file support varies by provider.
### Upload a File
<Tabs group="provider">
<Tab title="Anthropic Provider">
Upload a text file for use with Anthropic:
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
# Upload a text file
text_content = b"This is a test file for Files API integration."
response = client.beta.files.upload(
file=("test_upload.txt", text_content, "text/plain"),
)
print(f"File ID: {response.id}")
print(f"Filename: {response.filename}")
```
</Tab>
<Tab title="OpenAI Provider">
Upload a JSONL file for OpenAI batch processing:
```python
import anthropic
# Client configured for OpenAI provider
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
# Create JSONL content in OpenAI batch format
jsonl_content = b'''{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello!"}], "max_tokens": 100}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "How are you?"}], "max_tokens": 100}}'''
response = client.beta.files.upload(
file=("batch_input.jsonl", jsonl_content, "application/jsonl"),
)
print(f"File ID: {response.id}")
```
</Tab>
</Tabs>
### List Files
<Tabs group="provider">
<Tab title="Anthropic Provider">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
# List all files
response = client.beta.files.list()
for file in response.data:
print(f"File ID: {file.id}")
print(f"Filename: {file.filename}")
print(f"Size: {file.size} bytes")
print("---")
```
</Tab>
<Tab title="OpenAI Provider">
```python
import anthropic
# Client configured for OpenAI provider
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
# List all files from OpenAI
response = client.beta.files.list()
for file in response.data:
print(f"File ID: {file.id}, Name: {file.filename}")
```
</Tab>
</Tabs>
### Delete a File
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"} # or omit for anthropic
)
# Delete a file
file_id = "file-abc123"
response = client.beta.files.delete(file_id)
print(f"Deleted file: {file_id}")
```
### Download File Content
Note: Anthropic only allows downloading files created by certain tools (like code execution). OpenAI allows downloading batch output files.
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
# Download file content
file_id = "file-abc123"
response = client.beta.files.download(file_id)
content = response.text()
print(f"File content:\n{content}")
```
---
## Batch API
The Anthropic Batch API is accessed through `beta.messages.batches`. Anthropic's batch API uses **inline requests** rather than file uploads.
### Create a Batch with Inline Requests
<Tabs group="provider">
<Tab title="Anthropic Provider">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
# Create batch with inline requests
batch_requests = [
{
"custom_id": "request-1",
"params": {
"model": "claude-3-sonnet-20240229",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is 2+2?"}
]
}
},
{
"custom_id": "request-2",
"params": {
"model": "claude-3-sonnet-20240229",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}
}
]
batch = client.beta.messages.batches.create(requests=batch_requests)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
```
</Tab>
<Tab title="OpenAI Provider">
When routing to OpenAI, use OpenAI-compatible models:
```python
import anthropic
# Client configured for OpenAI provider
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
# Create batch with inline requests (using OpenAI models)
batch_requests = [
{
"custom_id": "request-1",
"params": {
"model": "gpt-4o-mini",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is 2+2?"}
]
}
},
{
"custom_id": "request-2",
"params": {
"model": "gpt-4o-mini",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}
}
]
batch = client.beta.messages.batches.create(requests=batch_requests)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
```
</Tab>
<Tab title="Gemini Provider">
When routing to Gemini:
```python
import anthropic
# Client configured for Gemini provider
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "gemini"}
)
# Create batch with inline requests (using Gemini models)
batch_requests = [
{
"custom_id": "request-1",
"params": {
"model": "gemini-1.5-flash",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is 2+2?"}
]
}
},
{
"custom_id": "request-2",
"params": {
"model": "gemini-1.5-flash",
"max_tokens": 100,
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}
}
]
batch = client.beta.messages.batches.create(requests=batch_requests)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
```
</Tab>
</Tabs>
<Note>
**Bedrock Note:** Bedrock requires file-based batch creation with S3 storage. When routing to Bedrock from the Anthropic SDK, you'll need to use the Bedrock SDK directly for batch operations. See the [Bedrock SDK documentation](../bedrock-sdk/files-and-batch) for details.
</Note>
### List Batches
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "anthropic"} # or "openai", "gemini"
)
# List batches
response = client.beta.messages.batches.list(limit=10)
for batch in response.data:
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
if batch.request_counts:
print(f"Processing: {batch.request_counts.processing}")
print(f"Succeeded: {batch.request_counts.succeeded}")
print(f"Errored: {batch.request_counts.errored}")
print("---")
```
### Retrieve Batch Status
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "anthropic"} # or "openai", "gemini"
)
# Retrieve batch status
batch_id = "batch-abc123"
batch = client.beta.messages.batches.retrieve(batch_id)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}")
if batch.request_counts:
print(f"Processing: {batch.request_counts.processing}")
print(f"Succeeded: {batch.request_counts.succeeded}")
print(f"Errored: {batch.request_counts.errored}")
```
### Cancel a Batch
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "anthropic"} # or "openai", "gemini"
)
# Cancel batch
batch_id = "batch-abc123"
batch = client.beta.messages.batches.cancel(batch_id)
print(f"Batch ID: {batch.id}")
print(f"Status: {batch.processing_status}") # "canceling" or "ended"
```
### Get Batch Results
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
# Get batch results (only available after batch is completed)
batch_id = "batch-abc123"
results = client.beta.messages.batches.results(batch_id)
# Iterate over results
for result in results:
print(f"Custom ID: {result.custom_id}")
if result.result.type == "succeeded":
message = result.result.message
print(f"Response: {message.content[0].text}")
elif result.result.type == "errored":
print(f"Error: {result.result.error}")
print("---")
```
---
## End-to-End Workflows
### Anthropic Batch Workflow
```python
import time
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key"
)
# Step 1: Create batch with inline requests
print("Step 1: Creating batch...")
batch_requests = [
{
"custom_id": "math-question",
"params": {
"model": "claude-3-sonnet-20240229",
"max_tokens": 100,
"messages": [{"role": "user", "content": "What is 15 * 7?"}]
}
},
{
"custom_id": "geography-question",
"params": {
"model": "claude-3-sonnet-20240229",
"max_tokens": 100,
"messages": [{"role": "user", "content": "What is the largest ocean?"}]
}
}
]
batch = client.beta.messages.batches.create(requests=batch_requests)
print(f" Created batch: {batch.id}, status: {batch.processing_status}")
# Step 2: Poll for completion
print("Step 2: Polling batch status...")
for i in range(20):
batch = client.beta.messages.batches.retrieve(batch.id)
print(f" Poll {i+1}: status = {batch.processing_status}")
if batch.processing_status == "ended":
print(" Batch completed!")
break
if batch.request_counts:
print(f" Processing: {batch.request_counts.processing}")
print(f" Succeeded: {batch.request_counts.succeeded}")
time.sleep(5)
# Step 3: Verify batch is in list
print("Step 3: Verifying batch in list...")
batch_list = client.beta.messages.batches.list(limit=20)
batch_ids = [b.id for b in batch_list.data]
assert batch.id in batch_ids, f"Batch {batch.id} should be in list"
print(f" Verified batch {batch.id} is in list")
# Step 4: Get results (if completed)
if batch.processing_status == "ended":
print("Step 4: Getting results...")
try:
results = client.beta.messages.batches.results(batch.id)
for result in results:
print(f" {result.custom_id}: ", end="")
if result.result.type == "succeeded":
print(result.result.message.content[0].text[:50] + "...")
else:
print(f"Error: {result.result.error}")
except Exception as e:
print(f" Results not yet available: {e}")
print(f"\nSuccess! Batch {batch.id} workflow completed.")
```
### Cross-Provider Batch Workflow (OpenAI via Anthropic SDK)
```python
import time
import anthropic
# Create client with OpenAI provider header
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="virtual-key-or-dummy-key",
default_headers={"x-model-provider": "openai"}
)
# Step 1: Create batch with OpenAI models
print("Step 1: Creating batch for OpenAI provider...")
batch_requests = [
{
"custom_id": "openai-request-1",
"params": {
"model": "gpt-4o-mini",
"max_tokens": 100,
"messages": [{"role": "user", "content": "Explain AI in one sentence."}]
}
},
{
"custom_id": "openai-request-2",
"params": {
"model": "gpt-4o-mini",
"max_tokens": 100,
"messages": [{"role": "user", "content": "What is machine learning?"}]
}
}
]
batch = client.beta.messages.batches.create(requests=batch_requests)
print(f" Created batch: {batch.id}, status: {batch.processing_status}")
# Step 2: Poll for completion
print("Step 2: Polling batch status...")
for i in range(10):
batch = client.beta.messages.batches.retrieve(batch.id)
print(f" Poll {i+1}: status = {batch.processing_status}")
if batch.processing_status in ["ended", "completed"]:
break
time.sleep(5)
print(f"\nSuccess! Cross-provider batch {batch.id} completed via Anthropic SDK.")
```
---
## Provider-Specific Notes
| Provider | Header Value | File Upload | Batch Type | Models |
|----------|--------------|-------------|------------|--------|
| **Anthropic** | `anthropic` or omit | ✅ Beta API | Inline requests | `claude-3-*` |
| **OpenAI** | `openai` | ✅ Beta API | Inline requests | `gpt-4o-*`, `gpt-4-*` |
| **Gemini** | `gemini` | ✅ Beta API | Inline requests | `gemini-1.5-*` |
| **Bedrock** | `bedrock` | ❌ Use Bedrock SDK | File-based (S3) | `anthropic.claude-*` |
---
## Next Steps
- **[Overview](./overview)** - Anthropic SDK integration basics
- **[Configuration](../../quickstart/gateway/provider-configuration)** - Bifrost setup and configuration
- **[Core Features](../../features/)** - Governance, semantic caching, and more

449
docs/integrations/anthropic-sdk/overview.mdx Normal file
View File

@@ -0,0 +1,449 @@
---
title: "Overview"
description: "Use Bifrost as a drop-in replacement for Anthropic API with full compatibility and enhanced features."
icon: "book"
---
## Overview
Bifrost provides complete Anthropic API compatibility through protocol adaptation. The integration handles request transformation, response normalization, and error mapping between Anthropic's Messages API specification and Bifrost's internal processing pipeline.
This integration enables you to utilize Bifrost's features like governance, load balancing, semantic caching, multi-provider support, and more, all while preserving your existing Anthropic SDK-based architecture.
**Endpoint:** `/anthropic`
<Note>
**Enabling the beta header**: Anthropic frequently uses the `anthropic-beta` header to gate access to new features.
Clients like Vercels AI SDK use these. Bifrost will block unrecognized headers by default for security purposes.
To enable the beta header for full compatability, add `anthropic-beta` to the AllowList under Settings -> Client Settings in the UI.
</Note>
---
## Setup
<Tabs group="anthropic-sdk">
<Tab title="Python">
```python {5}
import anthropic
# Configure client to use Bifrost
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="dummy-key" # Keys handled by Bifrost
)
# Make requests as usual
response = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.content[0].text)
```
</Tab>
<Tab title="JavaScript">
```javascript {5}
import Anthropic from "@anthropic-ai/sdk";
// Configure client to use Bifrost
const anthropic = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "dummy-key", // Keys handled by Bifrost
});
// Make requests as usual
const response = await anthropic.messages.create({
model: "claude-3-sonnet-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello!" }],
});
console.log(response.content[0].text);
```
</Tab>
</Tabs>
---
## Provider/Model Usage Examples
Use multiple providers through the same Anthropic SDK format by prefixing model names with the provider:
<Tabs group="anthropic-sdk">
<Tab title="Python">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="dummy-key"
)
# Anthropic models (default)
anthropic_response = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from Claude!"}]
)
# OpenAI models via Anthropic SDK format
openai_response = client.messages.create(
model="openai/gpt-4o-mini",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from OpenAI!"}]
)
# Google Vertex models via Anthropic SDK format
vertex_response = client.messages.create(
model="vertex/gemini-pro",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from Gemini!"}]
)
# Azure models
azure_response = client.messages.create(
model="azure/gpt-4o",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from Azure!"}]
)
# Local Ollama models
ollama_response = client.messages.create(
model="ollama/llama3.1:8b",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from Ollama!"}]
)
```
</Tab>
<Tab title="JavaScript">
```javascript
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "dummy-key",
});
// Anthropic models (default)
const anthropicResponse = await anthropic.messages.create({
model: "claude-3-sonnet-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from Claude!" }],
});
// OpenAI models via Anthropic SDK format
const openaiResponse = await anthropic.messages.create({
model: "openai/gpt-4o-mini",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from OpenAI!" }],
});
// Google Vertex models via Anthropic SDK format
const vertexResponse = await anthropic.messages.create({
model: "vertex/gemini-pro",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from Gemini!" }],
});
// Azure models
const azureResponse = await anthropic.messages.create({
model: "azure/gpt-4o",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from Azure!" }],
});
// Local Ollama models
const ollamaResponse = await anthropic.messages.create({
model: "ollama/llama3.1:8b",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from Ollama!" }],
});
```
</Tab>
</Tabs>
---
## Adding Custom Headers
Pass custom headers required by Bifrost plugins (like governance, telemetry, etc.):
<Tabs group="anthropic-sdk">
<Tab title="Python">
```python
import anthropic
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="dummy-key",
default_headers={
"x-bf-vk": "vk_12345", # Virtual key for governance
}
)
response = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello with custom headers!"}]
)
```
</Tab>
<Tab title="JavaScript">
```javascript
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "dummy-key",
defaultHeaders: {
"x-bf-vk": "vk_12345", // Virtual key for governance
},
});
const response = await anthropic.messages.create({
model: "claude-3-sonnet-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello with custom headers!" }],
});
```
</Tab>
</Tabs>
---
## Using Direct Keys
Pass API keys directly in requests to bypass Bifrost's load balancing. You can pass any provider's API key (OpenAI, Anthropic, Mistral, etc.) since Bifrost only looks for `Authorization` or `x-api-key` headers. This requires the **Allow Direct API keys** option to be enabled in Bifrost configuration.
> **Learn more:** See [Key Management](../../features/keys-management#direct-key-bypass) for enabling direct API key usage.
<Tabs group="anthropic-sdk">
<Tab title="Python">
```python
import anthropic
# Using Anthropic's API key directly
client_with_direct_key = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="sk-your-anthropic-key" # Anthropic's API key works
)
anthropic_response = client_with_direct_key.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello from Claude!"}]
)
# or pass different provider keys per request using headers
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="dummy-key"
)
# Use Anthropic key for Claude
anthropic_response = client.messages.create(
model="claude-3-sonnet-20240229",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello Claude!"}],
extra_headers={
"x-api-key": "sk-ant-your-anthropic-key"
}
)
# Use OpenAI key for GPT models
openai_response = client.messages.create(
model="openai/gpt-4o-mini",
max_tokens=1000,
messages=[{"role": "user", "content": "Hello GPT!"}],
extra_headers={
"Authorization": "Bearer sk-your-openai-key"
}
)
```
</Tab>
<Tab title="JavaScript">
```javascript
import Anthropic from "@anthropic-ai/sdk";
// Using Anthropic's API key directly
const anthropicWithDirectKey = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "sk-your-anthropic-key", // Anthropic's API key works
});
const anthropicResponse = await anthropicWithDirectKey.messages.create({
model: "claude-3-sonnet-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello from Claude!" }],
});
// or pass different provider keys per request using headers
const anthropic = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "dummy-key",
});
// Use Anthropic key for Claude
const anthropicResponse = await anthropic.messages.create({
model: "claude-3-sonnet-20240229",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello Claude!" }],
headers: {
"x-api-key": "sk-ant-your-anthropic-key",
},
});
// Use OpenAI key for GPT models
const openaiResponseWithHeader = await anthropic.messages.create({
model: "openai/gpt-4o-mini",
max_tokens: 1000,
messages: [{ role: "user", content: "Hello GPT!" }],
headers: {
"Authorization": "Bearer sk-your-openai-key",
},
});
```
</Tab>
</Tabs>
---
## Async Inference
Submit inference requests asynchronously and poll for results later using the `x-bf-async` header. This is useful for long-running requests where you don't want to hold a connection open. See [Async Inference](../../features/async-inference) for full details.
<Note>
Async inference requires a [Logs Store](../../features/observability/default) to be configured and is not compatible with streaming.
</Note>
### Messages
<Tabs group="anthropic-sdk">
<Tab title="Python">
```python
import anthropic
import time
client = anthropic.Anthropic(
base_url="http://localhost:8080/anthropic",
api_key="dummy-key"
)
# Submit async request
initial = client.messages.create(
model="anthropic/claude-sonnet-4-20250514",
max_tokens=256,
messages=[{"role": "user", "content": "Tell me a short story."}],
extra_headers={"x-bf-async": "true"}
)
# If content is present, the request completed synchronously
if initial.content:
print(initial.content[0].text)
else:
# Poll until completed
while True:
time.sleep(2)
poll = client.messages.create(
model="anthropic/claude-sonnet-4-20250514",
max_tokens=256,
messages=[{"role": "user", "content": "Tell me a short story."}],
extra_headers={"x-bf-async-id": initial.id}
)
if poll.content:
print(poll.content[0].text)
break
```
</Tab>
<Tab title="JavaScript">
```javascript
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic({
baseURL: "http://localhost:8080/anthropic",
apiKey: "dummy-key",
});
// Submit async request
const initial = await anthropic.messages.create(
{
model: "anthropic/claude-sonnet-4-20250514",
max_tokens: 256,
messages: [{ role: "user", content: "Tell me a short story." }],
},
{ headers: { "x-bf-async": "true" } }
);
// If content is present, the request completed synchronously
if (initial.content?.length > 0) {
console.log(initial.content[0].text);
} else {
// Poll until completed
while (true) {
await new Promise((r) => setTimeout(r, 2000));
const poll = await anthropic.messages.create(
{
model: "anthropic/claude-sonnet-4-20250514",
max_tokens: 256,
messages: [{ role: "user", content: "Tell me a short story." }],
},
{ headers: { "x-bf-async-id": initial.id } }
);
if (poll.content?.length > 0) {
console.log(poll.content[0].text);
break;
}
}
}
```
</Tab>
</Tabs>
### Async Headers
| Header | Description |
|---|---|
| `x-bf-async: true` | Submit the request as an async job. Returns immediately with a job ID. |
| `x-bf-async-id: <job-id>` | Poll for results of a previously submitted async job. |
| `x-bf-async-job-result-ttl: <seconds>` | Override the default result TTL (default: 3600s). |
---
## Supported Features
The Anthropic integration supports all features that are available in both the Anthropic SDK and Bifrost core functionality. If the Anthropic SDK supports a feature and Bifrost supports it, the integration will work seamlessly.
---
## Next Steps
- **[Files and Batch API](./files-and-batch)** - File uploads and batch processing
- **[OpenAI SDK](../openai-sdk/overview)** - GPT integration patterns
- **[Google GenAI SDK](../genai-sdk)** - Gemini integration patterns
- **[Configuration](../../quickstart/README)** - Bifrost setup and configuration
- **[Core Features](../../features/)** - Advanced Bifrost capabilities