first commit

This commit is contained in:
Beyhan Oğur
2026-04-26 21:52:23 +03:00
commit 880f412e2c
2662 changed files with 866266 additions and 0 deletions

View File

@@ -0,0 +1,705 @@
---
title: "Building Dynamically Linked Bifrost Binary"
description: "Learn how to build a dynamically linked Bifrost binary required for custom plugin support"
icon: "hammer"
---
## Why Dynamic Linking?
Go's plugin system requires **dynamic linking** to load `.so` files at runtime. By default, Bifrost builds are **statically linked** for maximum portability across Linux distributions - they bundle all dependencies including the C standard library (libc). However, statically linked binaries **cannot load Go plugins**.
To use custom plugins with Bifrost, you must build a dynamically linked binary that links against the system's libc at runtime.
<Warning>
Dynamic plugins only work on **Linux** and **macOS** (Darwin). Windows is not supported by Go's plugin system.
</Warning>
## Static vs Dynamic Builds
### Static Builds (Default)
Bifrost's default build configuration creates statically linked binaries:
```bash
go build \
-ldflags="-w -s -extldflags '-static' -X main.Version=v1.3.30" \
-tags "sqlite_static" \
-o bifrost-http
```
**Characteristics:**
- ✅ Portable across all Linux distributions (musl, glibc, etc.)
- ✅ No external dependencies required at runtime
- ✅ Smaller deployment surface area
- ❌ **Cannot load Go plugins**
**Use static builds when:** You don't need custom plugins and want maximum portability.
### Dynamic Builds (For Plugins)
To enable plugin support, build without static linking flags:
```bash
go build \
-ldflags="-w -s -X main.Version=v1.3.30" \
-o bifrost-http
```
**Characteristics:**
- ✅ **Can load Go plugins** (`.so` files)
- ✅ Slightly faster compilation
- ⚠️ Must match the target system's libc (musl vs glibc)
- ⚠️ Less portable across different Linux distributions
**Use dynamic builds when:** You need custom plugin support.
## Building with Makefile
The easiest way to build a dynamic binary is using the `DYNAMIC=1` flag with the Makefile:
### Local Build
```bash
# Build dynamically linked binary for your current platform
make build DYNAMIC=1
# With version tag
make build DYNAMIC=1 VERSION=1.3.30
```
This creates `tmp/bifrost-http` as a dynamically linked binary.
### Cross-Compilation
```bash
# Build for Linux AMD64 (uses Docker if cross-compiling)
make build DYNAMIC=1 GOOS=linux GOARCH=amd64
# Build for Linux ARM64
make build DYNAMIC=1 GOOS=linux GOARCH=arm64
```
### How It Works
The `DYNAMIC=1` flag automatically:
- ✅ Removes `-extldflags "-static"` from ldflags
- ✅ Removes `-tags "sqlite_static"` build tag
- ✅ Keeps `CGO_ENABLED=1` (required for SQLite and plugins)
- ✅ Uses Docker for cross-compilation when needed
## Building with Docker
For containerized deployments, you'll need to modify the Dockerfile. Here are two complete examples based on your target environment's libc.
### Option A: Alpine Linux (musl libc)
Use this for Alpine-based deployments or when you want minimal image size.
<Accordion title="Complete Dockerfile for Alpine (musl libc)">
```dockerfile
# --- UI Build Stage: Build the React + Vite frontend ---
FROM node:25-alpine3.23 AS ui-builder
WORKDIR /app
# Copy UI package files and install dependencies
COPY ui/package*.json ./
RUN npm ci
# Copy UI source code
COPY ui/ ./
# Build UI (skip the copy-build step)
RUN npm run build-enterprise
# --- Go Build Stage: Compile the Go binary ---
FROM golang:1.26.1-alpine3.23 AS builder
WORKDIR /app
# Install dependencies including gcc for CGO and sqlite
RUN apk add --no-cache gcc musl-dev sqlite-dev
# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
ENV CGO_ENABLED=1 GOOS=linux
COPY transports/go.mod transports/go.sum ./
RUN go mod download
# Copy source code and dependencies
COPY transports/ ./
COPY --from=ui-builder /app/out ./bifrost-http/ui
# Build the binary with CGO enabled for DYNAMIC LINKING
ENV GOWORK=off
ARG VERSION=unknown
RUN go build \
-ldflags="-w -s -X main.Version=v${VERSION}" \
-a -trimpath \
-o /app/main \
./bifrost-http
# Verify build succeeded
RUN test -f /app/main || (echo "Build failed" && exit 1)
# --- Runtime Stage: Minimal runtime image ---
FROM alpine:3.23
WORKDIR /app
# Install runtime dependencies for CGO-enabled dynamic binary
# musl: C standard library (required for CGO binaries)
# libgcc: GCC runtime library
# ca-certificates: For HTTPS connections
# wget: For healthcheck
RUN apk add --no-cache musl libgcc ca-certificates wget
# Create data directory and set up user
COPY --from=builder /app/main .
COPY --from=builder /app/docker-entrypoint.sh .
# Getting arguments
ARG ARG_APP_PORT=8080
ARG ARG_APP_HOST=0.0.0.0
ARG ARG_LOG_LEVEL=info
ARG ARG_LOG_STYLE=json
ARG ARG_APP_DIR=/app/data
# Environment variables with defaults (can be overridden at runtime)
ENV APP_PORT=$ARG_APP_PORT \
APP_HOST=$ARG_APP_HOST \
LOG_LEVEL=$ARG_LOG_LEVEL \
LOG_STYLE=$ARG_LOG_STYLE \
APP_DIR=$ARG_APP_DIR
RUN mkdir -p $APP_DIR/logs && \
adduser -D -s /bin/sh appuser && \
chown -R appuser:appuser /app && \
chmod +x /app/docker-entrypoint.sh
USER appuser
# Declare volume for data persistence
VOLUME ["/app/data"]
EXPOSE $APP_PORT
# Health check for container status monitoring
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1
# Use entrypoint script that handles volume permissions and argument processing
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["/app/main"]
```
</Accordion>
**Key changes from static build:**
- Line 40-44: Removed `-extldflags '-static'` and `-tags "sqlite_static"`
- Removed UPX compression step (optional, but simpler)
- Runtime uses musl libc from Alpine base image
**Build and run:**
```bash
# Build the image
docker build -f transports/Dockerfile -t bifrost:dynamic-alpine .
# Run the container
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-alpine
```
### Option B: Debian (glibc)
Use this for Debian/Ubuntu-based deployments or when deploying to glibc-based systems.
<Accordion title="Complete Dockerfile for Debian (glibc)">
```dockerfile
# --- UI Build Stage: Build the React + Vite frontend ---
FROM node:25-bookworm AS ui-builder
WORKDIR /app
# Copy UI package files and install dependencies
COPY ui/package*.json ./
RUN npm ci
# Copy UI source code
COPY ui/ ./
# Build UI
RUN npm run build-enterprise
# --- Go Build Stage: Compile the Go binary ---
FROM golang:1.26.1-bookworm AS builder
WORKDIR /app
# Install dependencies including gcc for CGO and sqlite
RUN apt-get update && apt-get install -y \
gcc \
libc6-dev \
libsqlite3-dev \
&& rm -rf /var/lib/apt/lists/*
# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
ENV CGO_ENABLED=1 GOOS=linux
COPY transports/go.mod transports/go.sum ./
RUN go mod download
# Copy source code and dependencies
COPY transports/ ./
COPY --from=ui-builder /app/out ./bifrost-http/ui
# Build the binary with CGO enabled for DYNAMIC LINKING
ENV GOWORK=off
ARG VERSION=unknown
RUN go build \
-ldflags="-w -s -X main.Version=v${VERSION}" \
-a -trimpath \
-o /app/main \
./bifrost-http
# Verify build succeeded
RUN test -f /app/main || (echo "Build failed" && exit 1)
# --- Runtime Stage: Minimal runtime image ---
FROM debian:bookworm-slim
WORKDIR /app
# Install runtime dependencies for CGO-enabled dynamic binary
# libc6: GNU C Library (required for glibc-linked binaries)
# ca-certificates: For HTTPS connections
RUN apt-get update && apt-get install -y \
libc6 \
ca-certificates \
wget \
&& rm -rf /var/lib/apt/lists/*
# Create data directory and set up user
COPY --from=builder /app/main .
COPY --from=builder /app/docker-entrypoint.sh .
# Getting arguments
ARG ARG_APP_PORT=8080
ARG ARG_APP_HOST=0.0.0.0
ARG ARG_LOG_LEVEL=info
ARG ARG_LOG_STYLE=json
ARG ARG_APP_DIR=/app/data
# Environment variables with defaults (can be overridden at runtime)
ENV APP_PORT=$ARG_APP_PORT \
APP_HOST=$ARG_APP_HOST \
LOG_LEVEL=$ARG_LOG_LEVEL \
LOG_STYLE=$ARG_LOG_STYLE \
APP_DIR=$ARG_APP_DIR
RUN mkdir -p $APP_DIR/logs && \
useradd -m -s /bin/sh appuser && \
chown -R appuser:appuser /app && \
chmod +x /app/docker-entrypoint.sh
USER appuser
# Declare volume for data persistence
VOLUME ["/app/data"]
EXPOSE $APP_PORT
# Health check for container status monitoring
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1
# Use entrypoint script that handles volume permissions and argument processing
ENTRYPOINT ["/app/docker-entrypoint.sh"]
CMD ["/app/main"]
```
</Accordion>
**Key differences from Alpine version:**
- Uses `bookworm` (Debian 12) base images instead of Alpine
- Installs `apt` packages instead of `apk`
- Runtime uses glibc (libc6) instead of musl
- Uses `useradd` instead of `adduser` for user creation
**Build and run:**
```bash
# Build the image
docker build -f transports/Dockerfile.debian -t bifrost:dynamic-debian .
# Run the container
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-debian
```
## libc Compatibility
Understanding libc (C standard library) compatibility is **critical** when building dynamic binaries and plugins.
### musl vs glibc
Linux distributions use one of two main C standard libraries:
| libc Type | Used By | Characteristics |
|-----------|---------|-----------------|
| **musl** | Alpine Linux | Lightweight, minimal, security-focused |
| **glibc** | Debian, Ubuntu, RHEL, CentOS, Fedora, Amazon Linux | Standard GNU C Library, feature-rich |
### The Golden Rule
<Warning>
- **A binary built with musl will NOT run on glibc systems.**
- **A binary built with glibc will NOT run on musl systems.**
- **Plugins and Bifrost MUST use the same libc.**
</Warning>
### Why This Matters
When you build a dynamic binary:
```bash
# Built on Alpine (musl)
$ ldd bifrost-http
linux-vdso.so.1
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1
# Built on Debian (glibc)
$ ldd bifrost-http
linux-vdso.so.1
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
```
The binary is linked to a **specific** libc implementation. If you try to run it on a system with a different libc, you'll get errors like:
```
error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file
```
### Choosing Your Build Environment
**Decision Matrix:**
| Target Deployment | Build With | Dockerfile Base |
|-------------------|------------|-----------------|
| Alpine containers | musl | `golang:1.26.1-alpine3.23` |
| Debian/Ubuntu containers | glibc | `golang:1.26.1-bookworm` |
| Ubuntu/Debian servers | glibc | `golang:1.26.1-bookworm` |
| RHEL/CentOS servers | glibc | Native build or glibc container |
| Kubernetes (Alpine) | musl | `golang:1.26.1-alpine3.23` |
| Kubernetes (Debian) | glibc | `golang:1.26.1-bookworm` |
**Simple rule:** Build with the same base OS family as your deployment target.
### Building Plugins
Plugins **must** be built with the **exact same environment** as your Bifrost binary:
```bash
# If Bifrost was built with Alpine/musl
docker run --rm \
-v "$PWD:/work" \
-w /work \
golang:1.26.1-alpine3.23 \
sh -c "apk add --no-cache gcc musl-dev && \
go build -buildmode=plugin -o myplugin.so main.go"
# If Bifrost was built with Debian/glibc
docker run --rm \
-v "$PWD:/work" \
-w /work \
golang:1.26.1-bookworm \
sh -c "apt-get update && apt-get install -y gcc && \
go build -buildmode=plugin -o myplugin.so main.go"
```
See the [hello-world plugin Makefile](https://github.com/maximhq/bifrost/blob/main/examples/plugins/hello-world/Makefile) for a complete example.
## Verification
### Verify Dynamic Linking
After building, check that your binary is dynamically linked:
```bash
# Check binary dependencies
ldd tmp/bifrost-http
# Expected output (musl):
linux-vdso.so.1
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1
# Expected output (glibc):
linux-vdso.so.1
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
```
If you see `statically linked`, the binary **will not load plugins**.
### Verify Plugin Compatibility
Test that your plugin loads successfully:
```bash
# Start Bifrost with your plugin configured
./tmp/bifrost-http -config config.json
# Check logs for plugin initialization
# Should see: "Plugin loaded successfully: your-plugin-name"
```
## Go Version and Package Compatibility
### Go Version Requirement
Bifrost is built with **Go 1.26.1**. Your plugin **must** be compiled with the exact same Go version to ensure compatibility.
```bash
# Check your Go version
go version
# Should output: go version go1.26.1 ...
# If you need to install Go 1.26.1
# Visit: https://go.dev/dl/
```
### Key Package Versions
Bifrost uses the following key packages across its three main modules that may affect plugin development:
#### Transport Layer (`transports/go.mod`)
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/bytedance/sonic` | v1.14.1 | High-performance JSON serialization |
| `github.com/valyala/fasthttp` | v1.67.0 | Fast HTTP server/client |
| `github.com/fasthttp/router` | v1.5.4 | HTTP router for fasthttp |
| `github.com/fasthttp/websocket` | v1.5.12 | WebSocket support |
| `github.com/prometheus/client_golang` | v1.23.0 | Prometheus metrics |
| `gorm.io/gorm` | v1.31.1 | Database ORM |
#### Core Layer (`core/go.mod`)
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/bytedance/sonic` | v1.14.1 | High-performance JSON serialization |
| `github.com/valyala/fasthttp` | v1.67.0 | Fast HTTP client for providers |
| `github.com/google/uuid` | v1.6.0 | UUID generation |
| `github.com/rs/zerolog` | v1.34.0 | Zero-allocation JSON logger |
| `github.com/mark3labs/mcp-go` | v0.41.1 | Model Context Protocol support |
| `golang.org/x/oauth2` | v0.32.0 | OAuth2 client |
#### Framework Layer (`framework/go.mod`)
| Package | Version | Purpose |
|---------|---------|---------|
| `github.com/redis/go-redis/v9` | v9.14.0 | Redis client for caching |
| `github.com/weaviate/weaviate-go-client/v5` | v5.5.0 | Weaviate vector store client |
| `github.com/mattn/go-sqlite3` | v1.14.32 | SQLite3 driver (requires CGO) |
| `gorm.io/gorm` | v1.31.1 | Database ORM |
| `gorm.io/driver/sqlite` | v1.6.0 | GORM SQLite driver |
| `gorm.io/driver/postgres` | v1.6.0 | GORM PostgreSQL driver |
| `golang.org/x/crypto` | v0.43.0 | Cryptographic functions |
<Note>
If your plugin imports any of these packages, use compatible versions to avoid runtime issues. Check `transports/go.mod`, `core/go.mod`, and `framework/go.mod` for complete dependency lists.
</Note>
### Checking Bifrost's Dependencies
To see all dependencies used by Bifrost across its three main modules:
```bash
# View transport layer dependencies
cat transports/go.mod
# View core dependencies
cat core/go.mod
# View framework dependencies
cat framework/go.mod
# Or list all dependencies for a specific module
cd transports && go list -m all
cd ../core && go list -m all
cd ../framework && go list -m all
```
### Plugin go.mod Example
When creating a plugin, your `go.mod` should match Bifrost's Go version:
```go
module github.com/example/my-plugin
go 1.26.1
require (
github.com/maximhq/bifrost/core v1.2.38
// Optional: Add framework for advanced features
// github.com/maximhq/bifrost/framework v1.1.48
// Add other dependencies as needed, matching versions from Bifrost's go.mod files
// github.com/bytedance/sonic v1.14.1
// github.com/rs/zerolog v1.34.0
)
```
<Tip>
Import only the Bifrost modules you need. Most plugins only require `core`. Use `framework` if you need access to config stores, vector stores, or other framework features.
</Tip>
## Troubleshooting
### Common Errors
#### 1. Cannot load plugin - Go version mismatch
```
cannot load plugin: plugin was built with a different version of package runtime/internal/sys
```
**Cause:** Plugin and Bifrost were built with different Go versions.
**Solution:** Use the exact same Go version (Go 1.26.1) for both:
```bash
# Check Go version used for Bifrost
./tmp/bifrost-http -version
# Verify your Go version matches
go version # Should output: go version go1.26.1
# See full compatibility requirements
```
Refer to [Go Version and Package Compatibility](#go-version-and-package-compatibility) for details.
#### 2. Shared library not found
```
error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file
```
**Cause:** Binary built with musl trying to run on glibc system (or vice versa).
**Solution:** Rebuild with the correct libc for your target system.
#### 3. Plugin architecture mismatch
```
plugin was built with a different version of package internal/cpu
```
**Cause:** Plugin and Bifrost built for different architectures (amd64 vs arm64).
**Solution:** Ensure `GOARCH` matches for both builds:
```bash
# Check architecture
uname -m # x86_64 = amd64, aarch64 = arm64
# Build with explicit architecture
GOARCH=amd64 go build ...
```
#### 4. Plugin file not found
```
plugin.Open("myplugin.so"): realpath failed: no such file or directory
```
**Cause:** Plugin file path is incorrect in config.
**Solution:** Use absolute paths or verify relative paths:
```json
{
"plugins": [
{
"path": "/app/data/plugins/myplugin.so",
"config": {}
}
]
}
```
## Best Practices
### 1. Document Your Build Environment
Create a `BUILD.md` file documenting:
- Go version used
- Base image (Alpine vs Debian)
- Build commands
- Target deployment platform
### 2. Use Consistent Tooling
Match Bifrost's exact Go version and key dependencies (see [Go Version and Package Compatibility](#go-version-and-package-compatibility)):
```bash
# Pin Go version in Dockerfile
FROM golang:1.26.1-alpine3.23 AS builder
# Pin Go version in Makefile/CI
GO_VERSION=1.26.1
```
### 3. Test Plugin Loading Locally
Before deploying, test plugin loading:
```bash
# Build both Bifrost and plugin
make build DYNAMIC=1
cd examples/plugins/hello-world && make build
# Test loading
./tmp/bifrost-http -config examples/plugins/hello-world/config.json
```
### 4. Version Your Plugins
Tag plugin builds with version and build info:
```bash
go build -buildmode=plugin \
-ldflags="-X main.Version=v1.0.0 -X main.BuildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
-o myplugin-v1.0.0.so
```
### 5. Multi-Stage Dockerfiles for Plugins
Build plugins in the same Dockerfile as Bifrost:
```dockerfile
# Build plugin
FROM golang:1.26.1-alpine3.23 AS plugin-builder
WORKDIR /plugin
COPY plugins/myplugin/ .
RUN apk add --no-cache gcc musl-dev && \
go build -buildmode=plugin -o myplugin.so main.go
# Build Bifrost
FROM golang:1.26.1-alpine3.23 AS bifrost-builder
# ... (bifrost build steps)
# Runtime
FROM alpine:3.23
COPY --from=bifrost-builder /app/main .
COPY --from=plugin-builder /plugin/myplugin.so /app/plugins/
```
This ensures plugins and Bifrost use identical build environments.
## Next Steps
Now that you have a dynamically linked Bifrost binary:
1. **[Write your first plugin](./writing-plugin)** - Learn the plugin API and create custom functionality
2. **[Deploy with plugins](../deployment-guides)** - Best practices for production deployments
3. **[Example plugins](https://github.com/maximhq/bifrost/tree/main/examples/plugins)** - Study working examples
<Note>
For questions or issues with dynamic builds and plugins, visit our [GitHub Discussions](https://github.com/maximhq/bifrost/discussions) or [Discord community](https://discord.gg/exN5KAydbU).
</Note>

View File

@@ -0,0 +1,113 @@
---
title: "Getting Started"
description: "Learn how to extend Bifrost's functionality by creating custom plugins that intercept and modify requests and responses."
icon: "book"
---
<Note>
Dynamic plugins require dynamic builds of Bifrost which are not enabled by default to keep Bifrost setup easier. If you want to build and try custom plugins on OSS read [building dynamically linked Bifrost binary](./building-dynamic-binary)
</Note>
## What are Bifrost Plugins?
Bifrost plugins allow you to extend the gateway's functionality by intercepting requests and responses. Plugins can modify, log, validate, or enrich data as it flows through the system, giving you powerful hooks into Bifrost's request lifecycle.
## Use Cases
Custom plugins enable you to:
- **Transform requests and responses** - Modify data before it reaches providers or after it returns
- **Add custom validation** - Enforce business rules on incoming requests
- **Implement custom caching** - Cache responses based on custom logic
- **Integrate with external systems** - Send data to logging, monitoring, or analytics platforms
- **Apply custom transformations** - Parse, filter, or enrich LLM responses
## Plugin Architecture
![architecture](../media/dynamic-plugins-architecture.png)
Bifrost leverages **Go's native plugin system** to enable dynamic extensibility. Plugins are built as **shared object files** (`.so` files) that are loaded at runtime by the Bifrost gateway.
### How Go Plugins Work
Go plugins use the `plugin` package from the standard library, which allows Go programs to dynamically load code at runtime. Here's what makes this approach powerful:
- **Native Go Integration** - Plugins are written in Go and have full access to Bifrost's type system and interfaces
- **Dynamic Loading** - Plugins can be loaded, unloaded, and reloaded without restarting Bifrost
- **Type Safety** - Go's type system ensures plugin methods match expected signatures
- **Performance** - No IPC overhead; plugins run in the same process as Bifrost
### Building Shared Objects
Plugins must be compiled as shared objects using Go's `-buildmode=plugin` flag:
```bash
go build -buildmode=plugin -o myplugin.so main.go
```
This generates a `.so` file that exports specific functions matching Bifrost's plugin interface:
<Tabs>
<Tab title="v1.4.x+">
- `Init(config any) error` - Initialize the plugin with configuration
- `GetName() string` - Return the plugin name
- `HTTPTransportPreHook()` - Intercept HTTP requests before they enter Bifrost core (HTTP transport only)
- `HTTPTransportPostHook()` - Intercept HTTP responses after they exit Bifrost core (HTTP transport only)
- `PreLLMHook()` - Intercept requests before they reach providers
- `PostLLMHook()` - Process responses after provider calls
- `Cleanup() error` - Clean up resources on shutdown
</Tab>
<Tab title="v1.3.x">
- `Init(config any) error` - Initialize the plugin with configuration
- `GetName() string` - Return the plugin name
- `TransportInterceptor()` - Modify raw HTTP headers/body (HTTP transport only)
- `PreLLMHook()` - Intercept requests before they reach providers
- `PostLLMHook()` - Process responses after provider calls
- `Cleanup() error` - Clean up resources on shutdown
</Tab>
</Tabs>
### Platform Requirements
**Important Limitations:**
- **Supported Platforms**: Linux and macOS (Darwin) only
- **No Cross-Compilation**: Plugins must be built on the target platform
- **Architecture Matching**: Plugin and Bifrost must use the same architecture (amd64, arm64)
- **Go Version Compatibility**: Plugin must be built with the same Go version as Bifrost
This means if you're running Bifrost on Linux AMD64, you must build your plugin on Linux AMD64 with the same Go version.
### Plugin Lifecycle
1. **Load** - Bifrost loads the `.so` file using Go's `plugin.Open()`
2. **Initialize** - Calls `Init()` with configuration from `config.json`
3. **Hook Execution** - Calls `PreLLMHook()` and `PostLLMHook()` for each request
4. **Cleanup** - Calls `Cleanup()` when Bifrost shuts down
Plugins execute in a specific order:
<Tabs>
<Tab title="v1.4.x+">
1. `HTTPTransportPreHook` - Intercept HTTP requests (HTTP transport only)
2. `PreLLMHook`/`PreMCPHook` - Executes in registration order, can short-circuit requests
3. Provider call (if not short-circuited)
4. `PostLLMHook`/`PostMCPHook` - Executes in reverse order of PreHooks
5. `HTTPTransportPostHook` - Intercept HTTP responses (HTTP transport only, reverse order)
</Tab>
<Tab title="v1.3.x">
1. `TransportInterceptor` - Modifies raw HTTP requests (HTTP transport only)
2. `PreHook` - Executes in registration order, can short-circuit requests
3. Provider call (if not short-circuited)
4. `PostHook` - Executes in reverse order of PreHooks
</Tab>
</Tabs>
## Next Steps
Ready to build your first plugin? Choose your approach:
- **[Writing Go Plugins](./writing-go-plugin)** - Native Go plugins using shared objects (`.so` files). Best for performance and full Go ecosystem access.
- **[Writing WASM Plugins](./writing-wasm-plugin)** - Cross-platform plugins using WebAssembly. Write in TypeScript, Go (TinyGo), or Rust. No version matching required.

View File

@@ -0,0 +1,465 @@
---
title: "Plugin Migration Guide"
description: "How to migrate your Bifrost plugins from v1.3.x to v1.4.x"
icon: "arrow-up-right-dots"
---
## Overview
Bifrost v1.4.x introduces a new plugin interface for HTTP transport layer interception. This guide helps you migrate existing plugins from the v1.3.x `TransportInterceptor` pattern to the v1.4.x `HTTPTransportPreHook` and `HTTPTransportPostHook` pattern.
<Note>
If your plugin doesn't use `TransportInterceptor`, no migration is needed. The `PreLLMHook`, `PostLLMHook`, `Init`, `GetName`, and `Cleanup` functions remain unchanged.
</Note>
## What Changed?
The HTTP transport interception mechanism changed from a simple function that receives and returns headers/body to a dual-hook pattern that works with both native `.so` plugins and WASM plugins.
### Key Differences
| Aspect | v1.3.x (TransportInterceptor) | v1.4.x+ (Pre/Post Hooks) |
|--------|-------------------------------|--------------------------|
| Signature | `TransportInterceptor(ctx, url, headers, body)` | `HTTPTransportPreHook(ctx, req)` + `HTTPTransportPostHook(ctx, req, resp)` |
| Return type | `(headers, body, error)` | Pre: `(*HTTPResponse, error)`, Post: `error` |
| Request type | Separate `headers map`, `body map` | Unified `*HTTPRequest` struct |
| Response access | Not available | Post-hook receives `*HTTPResponse` |
| Modification | Return modified maps | Modify `req`/`resp` in-place |
| Short-circuit | Return error | Return `*HTTPResponse` |
| WASM support | No | Yes |
| Context | Limited `BifrostContext` | Full `*BifrostContext` with `SetValue`/`Value` |
### Why the Change?
The new dual-hook pattern provides:
1. **WASM plugin support** - Serializable types work across WASM boundary
2. **Response interception** - Post-hook can modify responses before returning to client
3. **Simpler API** - No middleware wrapper, direct function call
4. **Better testability** - No fasthttp dependency in plugin tests
5. **Full context access** - BifrostContext available for sharing data between hooks
6. **Custom response short-circuits** - Return a full response to short-circuit
## Migration Steps
### Step 1: Update Imports
Remove the `fasthttp` import if present:
```go
import (
"fmt"
"github.com/maximhq/bifrost/core/schemas"
// Remove: "github.com/valyala/fasthttp"
)
```
### Step 2: Replace the Function
**Before (v1.3.x):**
```go
// TransportInterceptor modifies raw HTTP headers and body
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
// Add custom header
headers["X-Custom-Header"] = "value"
// Modify body
body["custom_field"] = "custom_value"
return headers, body, nil
}
```
**After (v1.4.x+):**
```go
// HTTPTransportPreHook intercepts requests BEFORE they enter Bifrost core
// Modify req in-place. Return (*HTTPResponse, nil) to short-circuit.
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
// Add custom header (in-place modification)
req.Headers["x-custom-header"] = "value"
// Modify body (in-place modification)
var body map[string]any
sonic.Unmarshal(req.Body, &body)
body["custom_field"] = "custom_value"
req.Body, _ = sonic.Marshal(body)
// Store values in context for use in post-hook
ctx.SetValue(schemas.BifrostContextKey("my-plugin-key"), "my-value")
// Return nil to continue, or return &HTTPResponse{} to short-circuit
return nil, nil
}
// HTTPTransportPostHook intercepts responses AFTER they exit Bifrost core
// Modify resp in-place. Called in reverse order of pre-hooks.
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Add response header
resp.Headers["x-processed-by"] = "my-plugin"
// Read values set in pre-hook
if val := ctx.Value(schemas.BifrostContextKey("my-plugin-key")); val != nil {
fmt.Println("Context value:", val)
}
// Return nil to continue, or return error to short-circuit
return nil
}
```
### Step 3: Update Body Modification Logic
In v1.3.x, you received the body as a `map[string]any`. In v1.4.x, you work with `req.Body` bytes:
**Before (v1.3.x):**
```go
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
// Direct map access
body["model"] = "gpt-4"
return headers, body, nil
}
```
**After (v1.4.x+):**
```go
import "github.com/bytedance/sonic"
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
// Parse body
var body map[string]any
if err := sonic.Unmarshal(req.Body, &body); err == nil {
// Modify body
body["model"] = "gpt-4"
// Update req.Body in-place
req.Body, _ = sonic.Marshal(body)
}
return nil, nil
}
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Modify response body if needed
var respBody map[string]any
if err := sonic.Unmarshal(resp.Body, &respBody); err == nil {
respBody["plugin_processed"] = true
resp.Body, _ = sonic.Marshal(respBody)
}
return nil
}
```
## Common Migration Patterns
### Adding Headers
**v1.3.x:**
```go
headers["authorization"] = "Bearer " + token
return headers, body, nil
```
**v1.4.x+:**
```go
// In HTTPTransportPreHook - modify request headers
req.Headers["authorization"] = "Bearer " + token
return nil, nil
// In HTTPTransportPostHook - modify response headers
resp.Headers["x-request-id"] = requestID
return nil
```
### Reading Headers
**v1.3.x:**
```go
apiKey := headers["X-API-Key"]
```
**v1.4.x+:**
```go
// Use case-insensitive helper for reading (recommended)
apiKey := req.CaseInsensitiveHeaderLookup("X-API-Key")
// Or direct map access (case-sensitive)
apiKey := req.Headers["x-api-key"]
```
### Conditional Processing
**v1.3.x:**
```go
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
if headers["x-skip-processing"] == "true" {
return headers, body, nil
}
// Process...
return headers, body, nil
}
```
**v1.4.x+:**
```go
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
if req.CaseInsensitiveHeaderLookup("x-skip-processing") == "true" {
return nil, nil // Continue without modification
}
// Process...
return nil, nil
}
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Post-hook always runs unless pre-hook short-circuited
return nil
}
```
### Error Handling / Short-Circuit
**v1.3.x:**
```go
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
if headers["x-api-key"] == "" {
return nil, nil, fmt.Errorf("missing API key")
}
return headers, body, nil
}
```
**v1.4.x+:**
```go
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
if req.CaseInsensitiveHeaderLookup("x-api-key") == "" {
// Return a custom response to short-circuit
return &schemas.HTTPResponse{
StatusCode: 401,
Headers: map[string]string{"Content-Type": "application/json"},
Body: []byte(`{"error": "missing API key"}`),
}, nil
}
return nil, nil
}
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Not called if pre-hook short-circuited
return nil
}
```
### Accessing Request Method and Path
**v1.3.x:**
```go
// url parameter contained the full URL
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
// Limited access to URL
return headers, body, nil
}
```
**v1.4.x+:**
```go
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
// Full access to request properties
method := req.Method // "GET", "POST", etc.
path := req.Path // "/v1/chat/completions"
query := req.Query // map[string]string of query params
pathParams := req.PathParams // map[string]string of path variables (e.g., {model})
return nil, nil
}
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Access both request and response
statusCode := resp.StatusCode
responseHeaders := resp.Headers
responseBody := resp.Body
_ = statusCode // Use variables...
_ = responseHeaders
_ = responseBody
return nil
}
```
## Testing Your Migration
1. **Build your updated plugin:**
```bash
go build -buildmode=plugin -o my-plugin.so main.go
```
2. **Update Bifrost to v1.4.x:**
```bash
go get github.com/maximhq/bifrost/core@v1.4.0
```
3. **Test with a simple request:**
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'
```
4. **Verify logs show both hooks being called:**
```
HTTPTransportPreHook called
PreLLMHook called
PostLLMHook called
HTTPTransportPostHook called
```
## Troubleshooting
### Plugin fails to load after migration
**Error:** `plugin: symbol TransportInterceptor not found`
This error occurs if Bifrost v1.4.x is looking for the old function. Make sure:
1. You've updated to `HTTPTransportPreHook` and `HTTPTransportPostHook`
2. The function signatures match exactly:
- `func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error)`
- `func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error`
3. You've rebuilt the plugin with the correct core version
### Body modification not working
Make sure you're assigning back to `req.Body` in the pre-hook:
```go
// Wrong - body changes lost
var body map[string]any
sonic.Unmarshal(req.Body, &body)
body["model"] = "gpt-4"
// Missing: req.Body = ...
// Correct - body changes applied
var body map[string]any
sonic.Unmarshal(req.Body, &body)
body["model"] = "gpt-4"
req.Body, _ = sonic.Marshal(body) // Assign back!
```
### Response modification not working
Make sure you're modifying `resp` in the post-hook:
```go
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Modify response headers
resp.Headers["x-custom-header"] = "value"
// Modify response body
var body map[string]any
sonic.Unmarshal(resp.Body, &body)
body["extra_field"] = "value"
resp.Body, _ = sonic.Marshal(body)
return nil
}
```
### Headers not being set
Make sure you're modifying `req.Headers` or `resp.Headers` directly:
```go
// Set request header in pre-hook
req.Headers["x-custom-header"] = "value"
// Set response header in post-hook
resp.Headers["x-custom-header"] = "value"
// Read headers using case-insensitive helper
value := req.CaseInsensitiveHeaderLookup("X-Custom-Header")
```
### Context values not available in post-hook
Make sure you're using the correct context key type:
```go
// In pre-hook - set value
ctx.SetValue(schemas.BifrostContextKey("my-key"), "my-value")
// In post-hook - read value
if val := ctx.Value(schemas.BifrostContextKey("my-key")); val != nil {
// Use val
}
```
## Streaming Chunk Hook (v1.4.x)
Bifrost v1.4.x introduces a new hook for intercepting streaming response chunks:
### HTTPTransportStreamChunkHook
This hook is called for each chunk during streaming responses, allowing plugins to modify or filter chunks before they're sent to the client.
```go
// HTTPTransportStreamChunkHook intercepts streaming chunks BEFORE they're written to the client.
// Modify chunk data or return nil to skip the chunk entirely.
// Only called for streaming responses when using HTTP transport (bifrost-http).
func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) {
// chunk is a typed struct containing one of:
// - BifrostTextCompletionResponse (text completion streaming)
// - BifrostChatResponse (chat completion streaming)
// - BifrostResponsesStreamResponse (responses API streaming)
// - BifrostSpeechStreamResponse (speech synthesis streaming)
// - BifrostTranscriptionStreamResponse (transcription streaming)
// - BifrostImageGenerationStreamResponse (image generation streaming)
// - BifrostError (error during streaming)
// Return chunk unchanged to pass through
return chunk, nil
// Return nil to skip/filter this chunk
// return nil, nil
// Return modified chunk
// modifiedChunk := &schemas.BifrostStreamChunk{BifrostChatResponse: ...}
// return modifiedChunk, nil
}
```
**Key differences from `HTTPTransportPostHook`:**
| Aspect | HTTPTransportPostHook | HTTPTransportStreamChunkHook |
|--------|----------------------|------------------------------|
| When called | After complete response | Per-chunk during streaming |
| Input | Full HTTPResponse | `*BifrostStreamChunk` (typed struct) |
| Can modify | Full response | Individual chunk struct |
| Can skip | N/A | Return nil to skip chunk |
<Note>
`HTTPTransportPostHook` is **not called** for streaming responses. Use `HTTPTransportStreamChunkHook` instead to intercept streaming data.
</Note>
### Migration for Existing Plugins
If your plugin implements `HTTPTransportPostHook` and you want to also handle streaming responses, add the new hook:
```go
// Existing hook for non-streaming responses
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
// Handle complete responses
return nil
}
// NEW: Add this for streaming responses
func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) {
// Handle streaming chunks (typed struct, not raw bytes)
// Return chunk unchanged if no modification needed
return chunk, nil
}
```
## Need Help?
- **Discord Community**: [Join our Discord](https://discord.gg/exN5KAydbU)
- **GitHub Issues**: [Report bugs or request features](https://github.com/maximhq/bifrost/issues)
- **Writing Plugins Guide**: [Full plugin documentation](./writing-plugin)

190
docs/plugins/sequencing.mdx Normal file
View File

@@ -0,0 +1,190 @@
---
title: "Plugin Sequencing"
description: "Control the execution order of custom plugins relative to Bifrost's built-in plugins using placement groups and ordering."
icon: "arrow-down-1-9"
---
## Overview
When you have multiple plugins — both built-in and custom — the order in which they execute matters. A logging plugin should capture the final request, an auth plugin should validate before anything else runs, and a response transformer should run after the provider returns data.
Plugin sequencing lets you control **where** your custom plugins execute relative to Bifrost's built-in plugins (telemetry, logging, governance, etc.) and **in what order** they execute relative to each other.
---
## How it works
Bifrost organizes plugins into three **placement groups** that execute in a fixed order:
```mermaid
graph LR
A["Pre-builtin plugins"] --> B["Built-in plugins"]
B --> C["Post-builtin plugins"]
```
| Placement Group | Pre-hooks (request) | Post-hooks (response) |
|-----------------|--------------------|-----------------------|
| `pre_builtin` | Runs **first** | Runs **last** |
| `builtin` | Runs **second** | Runs **second** |
| `post_builtin` | Runs **third** | Runs **first** |
<Info>
Post-hooks execute in **reverse order** of pre-hooks (LIFO pattern). This means a `pre_builtin` plugin's `PreLLMHook` runs first, but its `PostLLMHook` runs last — ensuring proper cleanup and state unwinding.
</Info>
### Ordering within a group
Within each placement group, plugins are sorted by their `order` value (lower executes earlier). Plugins with the same order preserve their registration order.
**Example:** Three custom plugins configured as:
| Plugin | Placement | Order | Pre-hook runs | Post-hook runs |
|--------|-----------|-------|---------------|----------------|
| auth-validator | `pre_builtin` | 0 | 1st | 5th (last) |
| request-enricher | `pre_builtin` | 1 | 2nd | 4th |
| *Built-in plugins* | — | — | 3rd | 3rd |
| response-logger | `post_builtin` | 0 | 4th | 2nd |
| analytics | `post_builtin` | 1 | 5th (last) | 1st |
---
## Configuration
<Tabs group="config-method">
<Tab title="Web UI">
1. Navigate to the **Plugins** page in the sidebar
2. Click the **Edit Plugin Sequence** button (appears when you have at least one custom plugin installed)
![Plugin Sequence Editor](../media/plugin-ordering.png)
3. **Drag** custom plugins above or below the **Built-in Plugins** block:
- Plugins **above** the block get `pre_builtin` placement
- Plugins **below** the block get `post_builtin` placement
4. The order within each group is determined by position (top = lowest order value)
5. Click **Save Sequence** to apply the changes
<Note>
If your `config.json` file has plugin sequence configured, it will take precedence over the sequence configured in the UI after restarting Bifrost.
</Note>
</Tab>
<Tab title="API">
Update a plugin's placement and order using the update endpoint:
```bash
curl -X PUT http://localhost:8080/api/plugins/my-plugin \
-H "Content-Type: application/json" \
-d '{
"enabled": true,
"path": "/path/to/my-plugin.so",
"placement": "pre_builtin",
"order": 0
}'
```
**Response:**
```json
{
"message": "Plugin updated successfully",
"plugin": {
"name": "my-plugin",
"enabled": true,
"isCustom": true,
"path": "/path/to/my-plugin.so",
"placement": "pre_builtin",
"order": 0,
"status": {
"status": "active"
}
}
}
```
You can also set placement when creating a plugin:
```bash
curl -X POST http://localhost:8080/api/plugins \
-H "Content-Type: application/json" \
-d '{
"name": "my-plugin",
"enabled": true,
"path": "/path/to/my-plugin.so",
"placement": "pre_builtin",
"order": 0
}'
```
</Tab>
<Tab title="config.json">
Set `placement` and `order` on each plugin in the `plugins` array:
```json
{
"plugins": [
{
"name": "auth-validator",
"enabled": true,
"path": "/plugins/auth-validator.so",
"placement": "pre_builtin",
"order": 0
},
{
"name": "request-enricher",
"enabled": true,
"path": "/plugins/request-enricher.so",
"placement": "pre_builtin",
"order": 1
},
{
"name": "response-logger",
"enabled": true,
"path": "/plugins/response-logger.so",
"placement": "post_builtin",
"order": 0
}
]
}
```
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `placement` | string | No | `post_builtin` | `"pre_builtin"` or `"post_builtin"`. Controls whether the plugin runs before or after built-in plugins. |
| `order` | integer | No | `0` | Position within the placement group. Lower values execute earlier. |
</Tab>
</Tabs>
---
## When to use each placement
### `pre_builtin` — run before built-in plugins
Use this when your plugin needs to:
- **Validate or authenticate** requests before any built-in processing
- **Enrich requests** with data that built-in plugins should see (e.g., injecting headers or metadata)
- **Short-circuit** requests before they reach governance checks or telemetry
### `post_builtin` (default) — run after built-in plugins
Use this when your plugin needs to:
- **Transform responses** after all built-in processing is complete
- **Log or analyze** the final request/response (after governance, telemetry, etc.)
- **Add custom headers** or modify the response before it reaches the client
<Tip>
When in doubt, use the default `post_builtin` placement. Most custom plugins — logging, analytics, response transformations — work best after built-in plugins have finished their processing.
</Tip>
---
## Next steps
- **[Writing a Go plugin](./writing-go-plugin)** — Build your first custom plugin with `PreLLMHook` and `PostLLMHook`
- **[Writing a WASM plugin](./writing-wasm-plugin)** — Build a portable WASM plugin
- **[Plugin architecture](../architecture/core/plugins)** — Deep dive into the plugin lifecycle and hook execution model

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff