---
title: "LiteLLM SDK"
description: "Use Bifrost as a drop-in proxy for LiteLLM applications with zero code changes."
icon: "train"
---

Since LiteLLM already provides multi-provider abstraction, Bifrost adds enterprise features like governance, semantic caching, MCP tools, observability, etc, on top of your existing setup.

**Endpoint:** `/litellm`

<Warning>
 **Provider Compatibility:** This integration only works for AI providers that both LiteLLM and Bifrost support. If you're using a provider specific to LiteLLM that Bifrost doesn't support (or vice versa), those requests will fail.
</Warning>
---

## Setup

<Tabs group="litellm-sdk">
<Tab title="Python">

```python {7}
from litellm import completion

# Configure client to use Bifrost
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm"  # Point to Bifrost
)

print(response.choices[0].message.content)
```

</Tab>
</Tabs>

---

## Provider/Model Usage Examples

Your existing LiteLLM provider switching works unchanged through Bifrost:

<Tabs group="litellm-sdk">
<Tab title="Python">

```python {4}
from litellm import completion

# All your existing LiteLLM patterns work the same
base_url = "http://localhost:8080/litellm"

# OpenAI models
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url=base_url
)

# Anthropic models  
anthropic_response = completion(
    model="claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url=base_url
)

# Google models
google_response = completion(
    model="gemini/gemini-1.5-flash",
    messages=[{"role": "user", "content": "Hello Gemini!"}],
    base_url=base_url
)

# Azure models
azure_response = completion(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello Azure!"}],
    base_url=base_url
)
```

</Tab>
</Tabs>

---

## Adding Custom Headers

Add Bifrost-specific headers for governance and tracking:

<Tabs group="litellm-sdk">
<Tab title="Python">

```python
from litellm import completion

# Add custom headers for Bifrost features
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-bf-vk": "your-virtual-key",          # Virtual key for governance
    }
)

print(response.choices[0].message.content)
```

</Tab>
</Tabs>

---

## Using Direct Keys

Pass API keys directly to bypass Bifrost's key management. You can pass any provider's API key since Bifrost only looks for `Authorization` or `x-api-key` headers. This requires the **Allow Direct API keys** option to be enabled in Bifrost configuration.

> **Learn more:** See [Key Management](../features/keys-management#direct-key-bypass) for enabling direct API key usage.

<Tabs group="litellm-sdk">
<Tab title="Python">

```python
from litellm import completion

# Using OpenAI key directly
openai_response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello GPT!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "Authorization": "Bearer sk-your-openai-key"
    }
)

# Using Anthropic key for Claude models
anthropic_response = completion(
    model="claude-3-sonnet-20240229", 
    messages=[{"role": "user", "content": "Hello Claude!"}],
    base_url="http://localhost:8080/litellm",
    extra_headers={
        "x-api-key": "sk-ant-your-anthropic-key"
    }
)

# Using Azure with direct Azure key
import os

deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT", "my-azure-deployment")
model = f"azure/{deployment}"

azure_response = completion(
    model=model,
    messages=[{"role": "user", "content": "Hello from LiteLLM (Azure demo)!"}],
    base_url="http://localhost:8080/litellm",
    api_key=os.getenv("AZURE_API_KEY", "your-azure-api-key"),
    deployment_id=os.getenv("AZURE_OPENAI_DEPLOYMENT", "gpt-4o-aug"),
    max_tokens=100,
    extra_headers={
        "x-bf-azure-endpoint": "https://your-resource.openai.azure.com",
    }
)
```

</Tab>
</Tabs>

---

## Supported Features

The LiteLLM integration supports all features that are available in both the LiteLLM SDK and Bifrost core functionality. Your existing LiteLLM code works seamlessly with Bifrost's enterprise features. 😄

---

## Next Steps

- **[Governance Features](../features/governance)** - Virtual keys and team management
- **[Semantic Caching](../features/semantic-caching)** - Intelligent response caching
- **[Configuration](../quickstart/README)** - Provider setup and API key management