beyhan/bifrost
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Beyhan Oğur
2026-04-26 21:52:23 +03:00
commit 880f412e2c
2662 changed files with 866266 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

549
docs/deployment-guides/helm/plugins.mdx Normal file
View File

@@ -0,0 +1,549 @@
---
title: "Plugins"
description: "Configure Bifrost plugins in Helm — telemetry, logging, semantic cache, OpenTelemetry, Datadog, governance, and custom plugins"
icon: "puzzle-piece"
---
Plugins are configured under `bifrost.plugins`. Each plugin is independently enabled/disabled. Pre-hooks run in registration order; post-hooks run in reverse order.
<Note>
**Telemetry, logging, and governance are auto-loaded built-ins** — they are always active and do not need to be explicitly enabled. Their configuration lives in `bifrost.client.*` and `bifrost.governance.*`, not in the `plugins` block.
The `plugins` block controls the opt-in plugins: `semanticCache`, `otel`, `datadog`, `maxim`, and custom plugins.
</Note>
```yaml
bifrost:
plugins:
semanticCache:
enabled: false
otel:
enabled: false
datadog:
enabled: false
```
```bash
# Enable an opt-in plugin at install time
helm install bifrost bifrost/bifrost \
--set image.tag=v1.4.11 \
--set bifrost.plugins.otel.enabled=true
# Or upgrade to enable a plugin without touching other values
helm upgrade bifrost bifrost/bifrost \
--reuse-values \
--set bifrost.plugins.semanticCache.enabled=true
```
---
<Tabs>
<Tab title="Telemetry">
### Telemetry (Prometheus)
<Note>
Telemetry is **always active** — it cannot be disabled. You do not need to set `bifrost.plugins.telemetry.enabled`.
</Note>
Exposes Prometheus metrics at `GET /metrics`. Custom labels are set via `bifrost.client.prometheusLabels`:
```yaml
bifrost:
client:
prometheusLabels:
- "environment=production"
- "region=us-east-1"
```
```bash
# Verify metrics are exposed
kubectl port-forward svc/bifrost 8080:8080 &
curl http://localhost:8080/metrics | head -30
```
**With Prometheus Push Gateway** (recommended for multi-replica / HA setups where pull-based scraping can miss pods):
```yaml
bifrost:
plugins:
telemetry:
enabled: true
config:
push_gateway:
enabled: true
push_gateway_url: "http://prometheus-pushgateway.monitoring.svc.cluster.local:9091"
job_name: "bifrost"
instance_id: "" # auto-derived from pod name if empty
push_interval: 15
basic_auth:
username: ""
password: ""
```
**ServiceMonitor for Prometheus Operator:**
```yaml
serviceMonitor:
enabled: true
interval: 30s
scrapeTimeout: 10s
namespace: monitoring # namespace where Prometheus is deployed
```
</Tab>
<Tab title="Logging">
### Request/Response Logging
<Note>
Logging is **auto-loaded** when `bifrost.client.enableLogging: true` and a log store is configured. You do not need to set `bifrost.plugins.logging.enabled`.
</Note>
Configure logging via the `client` block:
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.client.enableLogging` | Enable request/response logging | `true` |
| `bifrost.client.disableContentLogging` | Strip message body from logs (HIPAA/PCI) | `false` |
| `bifrost.client.loggingHeaders` | HTTP headers to capture in log metadata | `[]` |
```yaml
bifrost:
client:
enableLogging: true
disableContentLogging: false # set true for HIPAA/compliance
loggingHeaders:
- "x-request-id"
- "x-user-id"
- "x-team-id"
```
```bash
# Verify logs are being written
kubectl port-forward svc/bifrost 8080:8080 &
curl -s "http://localhost:8080/api/logs?limit=5" | jq .
```
See [Client Configuration](/deployment-guides/helm/client) for the full reference.
</Tab>
<Tab title="Governance">
### Governance
<Note>
Governance is **always active** for OSS deployments. You do not need to set `bifrost.plugins.governance.enabled`.
</Note>
Virtual key enforcement is controlled by the `client` block:
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.client.enforceAuthOnInference` | Require a virtual key (`x-bf-vk`) on every inference request | `false` |
```yaml
bifrost:
client:
enforceAuthOnInference: true # require virtual key on all inference requests
```
Define virtual keys, budgets, rate limits, and routing rules in `bifrost.governance.*`. See the [Governance](/deployment-guides/helm/governance) page.
</Tab>
<Tab title="Semantic Cache">
### Semantic Cache
Caches LLM responses using vector similarity so semantically equivalent prompts return cached answers.
Two modes:
- **Semantic mode** (`dimension > 1`): uses an embedding model + vector store for similarity search
- **Direct / hash mode** (`dimension: 1`): exact-match hash-based caching, no embedding model needed
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.plugins.semanticCache.enabled` | Enable semantic caching | `false` |
| `bifrost.plugins.semanticCache.version` | Plugin config version for DB-backed update tracking (`1` to `32767`) | `1` |
| `bifrost.plugins.semanticCache.config.provider` | Embedding provider | `"openai"` |
| `bifrost.plugins.semanticCache.config.embedding_model` | Embedding model name | `"text-embedding-3-small"` |
| `bifrost.plugins.semanticCache.config.dimension` | Embedding dimension (`1` = direct/hash mode) | `1536` |
| `bifrost.plugins.semanticCache.config.threshold` | Cosine similarity threshold (0–1) | `0.8` |
| `bifrost.plugins.semanticCache.config.ttl` | Cache entry TTL (Go duration) | `"5m"` |
| `bifrost.plugins.semanticCache.config.conversation_history_threshold` | Number of past messages to include in cache key | `3` |
| `bifrost.plugins.semanticCache.config.cache_by_model` | Include model name in cache key | `true` |
| `bifrost.plugins.semanticCache.config.cache_by_provider` | Include provider name in cache key | `true` |
| `bifrost.plugins.semanticCache.config.exclude_system_prompt` | Exclude system prompt from cache key | `false` |
| `bifrost.plugins.semanticCache.config.cleanup_on_shutdown` | Delete cache data on pod shutdown | `false` |
**Semantic mode (with OpenAI embeddings + Weaviate):**
```bash
kubectl create secret generic semantic-cache-secret \
--from-literal=openai-key='sk-your-openai-embedding-key'
```
```yaml
# semantic-cache-values.yaml
image:
tag: "v1.4.11"
vectorStore:
enabled: true
type: weaviate
weaviate:
enabled: true
persistence:
size: 20Gi
bifrost:
plugins:
semanticCache:
enabled: true
config:
provider: "openai"
keys:
- value: "env.SEMANTIC_CACHE_OPENAI_KEY"
weight: 1
embedding_model: "text-embedding-3-small"
dimension: 1536
threshold: 0.85
ttl: "1h"
conversation_history_threshold: 5
cache_by_model: true
cache_by_provider: true
providerSecrets:
semantic-cache-key:
existingSecret: "semantic-cache-secret"
key: "openai-key"
envVar: "SEMANTIC_CACHE_OPENAI_KEY"
```
```bash
helm install bifrost bifrost/bifrost -f semantic-cache-values.yaml
```
**Direct / hash mode** (no embedding provider needed):
```yaml
bifrost:
plugins:
semanticCache:
enabled: true
config:
dimension: 1 # triggers hash-based exact matching
ttl: "30m"
cache_by_model: true
cache_by_provider: true
```
<Note>
The vector store (`vectorStore.*`) must be configured and enabled for semantic mode. Direct/hash mode works without a vector store but still requires a storage backend.
</Note>
</Tab>
<Tab title="OpenTelemetry">
### OpenTelemetry (OTel)
Sends distributed traces and push-based metrics to any OTLP-compatible collector (Jaeger, Tempo, Honeycomb, etc.).
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.plugins.otel.enabled` | Enable OTel tracing | `false` |
| `bifrost.plugins.otel.version` | Plugin config version for DB-backed update tracking (`1` to `32767`) | `1` |
| `bifrost.plugins.otel.config.service_name` | Service name in traces | `"bifrost"` |
| `bifrost.plugins.otel.config.collector_url` | OTLP collector endpoint | `""` |
| `bifrost.plugins.otel.config.trace_type` | Trace type (`genai_extension`, `vercel`, or `open_inference`) | `"genai_extension"` |
| `bifrost.plugins.otel.config.protocol` | Transport protocol (`grpc` or `http`) | `"grpc"` |
| `bifrost.plugins.otel.config.metrics_enabled` | Enable OTLP push-based metrics | `false` |
| `bifrost.plugins.otel.config.metrics_endpoint` | OTLP metrics endpoint | `""` |
| `bifrost.plugins.otel.config.metrics_push_interval` | Push interval in seconds | `15` |
| `bifrost.plugins.otel.config.headers` | Custom headers for the collector | `{}` |
| `bifrost.plugins.otel.config.insecure` | Skip TLS verification | `false` |
| `bifrost.plugins.otel.config.tls_ca_cert` | Path to CA cert for TLS | `""` |
```yaml
# otel-values.yaml
image:
tag: "v1.4.11"
bifrost:
plugins:
otel:
enabled: true
config:
service_name: "bifrost-production"
collector_url: "otel-collector.observability.svc.cluster.local:4317"
trace_type: "genai_extension"
protocol: "grpc"
insecure: true # set false in production with a proper cert
metrics_enabled: true
metrics_endpoint: "otel-collector.observability.svc.cluster.local:4317"
metrics_push_interval: 15
headers:
x-honeycomb-team: "env.HONEYCOMB_API_KEY"
```
```bash
helm upgrade bifrost bifrost/bifrost --reuse-values -f otel-values.yaml
```
**With authentication headers from a Kubernetes Secret:**
```bash
kubectl create secret generic otel-credentials \
--from-literal=api-key='your-honeycomb-or-grafana-key'
```
```yaml
bifrost:
plugins:
otel:
enabled: true
config:
collector_url: "api.honeycomb.io:443"
protocol: "grpc"
headers:
x-honeycomb-team: "env.OTEL_API_KEY"
providerSecrets:
otel-key:
existingSecret: "otel-credentials"
key: "api-key"
envVar: "OTEL_API_KEY"
```
</Tab>
<Tab title="Datadog">
### Datadog APM
Sends traces to a Datadog Agent running in the cluster.
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.plugins.datadog.enabled` | Enable Datadog tracing | `false` |
| `bifrost.plugins.datadog.version` | Plugin config version for DB-backed update tracking (`1` to `32767`) | `1` |
| `bifrost.plugins.datadog.config.service_name` | Service name | `"bifrost"` |
| `bifrost.plugins.datadog.config.agent_addr` | Datadog Agent address | `"localhost:8126"` |
| `bifrost.plugins.datadog.config.env` | Deployment environment tag | `""` |
| `bifrost.plugins.datadog.config.version` | Version tag | `""` |
| `bifrost.plugins.datadog.config.enable_traces` | Enable trace collection | `true` |
| `bifrost.plugins.datadog.config.custom_tags` | Extra tags on all spans | `{}` |
The Datadog Agent is typically deployed via the [Datadog Helm chart](https://docs.datadoghq.com/containers/kubernetes/installation/) as a DaemonSet, making it available at the node's hostIP.
```yaml
# datadog-values.yaml
image:
tag: "v1.4.11"
bifrost:
plugins:
datadog:
enabled: true
config:
service_name: "bifrost"
agent_addr: "$(HOST_IP):8126" # uses Datadog DaemonSet pattern
env: "production"
version: "v1.4.11"
enable_traces: true
custom_tags:
team: "platform"
region: "us-east-1"
# Inject HOST_IP so Bifrost can reach the DaemonSet agent on the same node
env:
- name: HOST_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
```
```bash
helm upgrade bifrost bifrost/bifrost --reuse-values -f datadog-values.yaml
```
</Tab>
<Tab title="Maxim">
### Maxim Observability
Sends LLM request/response data to [Maxim](https://getmaxim.ai) for tracing, evaluation, and observability.
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.plugins.maxim.enabled` | Enable Maxim plugin | `false` |
| `bifrost.plugins.maxim.version` | Plugin config version for DB-backed update tracking (`1` to `32767`) | `1` |
| `bifrost.plugins.maxim.config.api_key` | Maxim API key (plain text, prefer secret) | `""` |
| `bifrost.plugins.maxim.config.log_repo_id` | Maxim log repository ID | `""` |
| `bifrost.plugins.maxim.secretRef.name` | Kubernetes Secret name for API key | `""` |
| `bifrost.plugins.maxim.secretRef.key` | Key within the secret | `"api-key"` |
```bash
kubectl create secret generic maxim-credentials \
--from-literal=api-key='your-maxim-api-key'
```
```yaml
# maxim-values.yaml
image:
tag: "v1.4.11"
bifrost:
plugins:
maxim:
enabled: true
config:
log_repo_id: "your-log-repo-id"
secretRef:
name: "maxim-credentials"
key: "api-key"
```
```bash
helm upgrade bifrost bifrost/bifrost --reuse-values -f maxim-values.yaml
```
</Tab>
<Tab title="Custom Plugin">
### Custom / Dynamic Plugins
Load a custom Go plugin (compiled `.so` file) at runtime.
| Parameter | Description | Default |
|-----------|-------------|---------|
| `bifrost.plugins.custom[].name` | Unique plugin name | `""` |
| `bifrost.plugins.custom[].enabled` | Enable custom plugin | `false` |
| `bifrost.plugins.custom[].path` | Path to compiled `.so` file in the container | `""` |
| `bifrost.plugins.custom[].version` | Plugin config version (`1` to `32767`) | `1` |
| `bifrost.plugins.custom[].config` | Arbitrary plugin-specific configuration | `{}` |
```yaml
bifrost:
plugins:
custom:
- name: "my-custom-plugin"
enabled: true
path: "/plugins/my-plugin.so"
version: 1
config:
api_endpoint: "https://my-service.example.com"
timeout: 5000
```
Mount the `.so` file via a volume:
```yaml
volumes:
- name: custom-plugins
configMap:
name: bifrost-custom-plugins
volumeMounts:
- name: custom-plugins
mountPath: /plugins
```
Or use an init container to download the plugin binary:
```yaml
initContainers:
- name: download-plugin
image: curlimages/curl:8.6.0
command:
- sh
- -c
- |
curl -fsSL https://plugins.example.com/my-plugin.so \
-o /plugins/my-plugin.so
volumeMounts:
- name: plugin-dir
mountPath: /plugins
volumes:
- name: plugin-dir
emptyDir: {}
volumeMounts:
- name: plugin-dir
mountPath: /plugins
```
```bash
helm upgrade bifrost bifrost/bifrost --reuse-values -f custom-plugin-values.yaml
```
</Tab>
</Tabs>
---
## All Plugins Together
```yaml
# all-plugins-values.yaml
image:
tag: "v1.4.11"
bifrost:
encryptionKeySecret:
name: "bifrost-encryption"
key: "encryption-key"
plugins:
telemetry:
enabled: true
config:
custom_labels:
- name: "environment"
value: "production"
logging:
enabled: true
config:
disable_content_logging: false
logging_headers:
- "x-request-id"
governance:
enabled: true
config:
is_vk_mandatory: true
semanticCache:
enabled: true
config:
provider: "openai"
keys:
- value: "env.CACHE_OPENAI_KEY"
weight: 1
embedding_model: "text-embedding-3-small"
dimension: 1536
threshold: 0.85
ttl: "1h"
otel:
enabled: true
config:
service_name: "bifrost"
collector_url: "otel-collector.observability.svc.cluster.local:4317"
protocol: "grpc"
insecure: true
```
```bash
helm install bifrost bifrost/bifrost -f all-plugins-values.yaml
```