first commit
This commit is contained in:
705
docs/plugins/building-dynamic-binary.mdx
Normal file
705
docs/plugins/building-dynamic-binary.mdx
Normal file
@@ -0,0 +1,705 @@
|
||||
---
|
||||
title: "Building Dynamically Linked Bifrost Binary"
|
||||
description: "Learn how to build a dynamically linked Bifrost binary required for custom plugin support"
|
||||
icon: "hammer"
|
||||
---
|
||||
|
||||
## Why Dynamic Linking?
|
||||
|
||||
Go's plugin system requires **dynamic linking** to load `.so` files at runtime. By default, Bifrost builds are **statically linked** for maximum portability across Linux distributions - they bundle all dependencies including the C standard library (libc). However, statically linked binaries **cannot load Go plugins**.
|
||||
|
||||
To use custom plugins with Bifrost, you must build a dynamically linked binary that links against the system's libc at runtime.
|
||||
|
||||
<Warning>
|
||||
Dynamic plugins only work on **Linux** and **macOS** (Darwin). Windows is not supported by Go's plugin system.
|
||||
</Warning>
|
||||
|
||||
## Static vs Dynamic Builds
|
||||
|
||||
### Static Builds (Default)
|
||||
|
||||
Bifrost's default build configuration creates statically linked binaries:
|
||||
|
||||
```bash
|
||||
go build \
|
||||
-ldflags="-w -s -extldflags '-static' -X main.Version=v1.3.30" \
|
||||
-tags "sqlite_static" \
|
||||
-o bifrost-http
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- ✅ Portable across all Linux distributions (musl, glibc, etc.)
|
||||
- ✅ No external dependencies required at runtime
|
||||
- ✅ Smaller deployment surface area
|
||||
- ❌ **Cannot load Go plugins**
|
||||
|
||||
**Use static builds when:** You don't need custom plugins and want maximum portability.
|
||||
|
||||
### Dynamic Builds (For Plugins)
|
||||
|
||||
To enable plugin support, build without static linking flags:
|
||||
|
||||
```bash
|
||||
go build \
|
||||
-ldflags="-w -s -X main.Version=v1.3.30" \
|
||||
-o bifrost-http
|
||||
```
|
||||
|
||||
**Characteristics:**
|
||||
- ✅ **Can load Go plugins** (`.so` files)
|
||||
- ✅ Slightly faster compilation
|
||||
- ⚠️ Must match the target system's libc (musl vs glibc)
|
||||
- ⚠️ Less portable across different Linux distributions
|
||||
|
||||
**Use dynamic builds when:** You need custom plugin support.
|
||||
|
||||
## Building with Makefile
|
||||
|
||||
The easiest way to build a dynamic binary is using the `DYNAMIC=1` flag with the Makefile:
|
||||
|
||||
### Local Build
|
||||
|
||||
```bash
|
||||
# Build dynamically linked binary for your current platform
|
||||
make build DYNAMIC=1
|
||||
|
||||
# With version tag
|
||||
make build DYNAMIC=1 VERSION=1.3.30
|
||||
```
|
||||
|
||||
This creates `tmp/bifrost-http` as a dynamically linked binary.
|
||||
|
||||
### Cross-Compilation
|
||||
|
||||
```bash
|
||||
# Build for Linux AMD64 (uses Docker if cross-compiling)
|
||||
make build DYNAMIC=1 GOOS=linux GOARCH=amd64
|
||||
|
||||
# Build for Linux ARM64
|
||||
make build DYNAMIC=1 GOOS=linux GOARCH=arm64
|
||||
```
|
||||
|
||||
### How It Works
|
||||
|
||||
The `DYNAMIC=1` flag automatically:
|
||||
- ✅ Removes `-extldflags "-static"` from ldflags
|
||||
- ✅ Removes `-tags "sqlite_static"` build tag
|
||||
- ✅ Keeps `CGO_ENABLED=1` (required for SQLite and plugins)
|
||||
- ✅ Uses Docker for cross-compilation when needed
|
||||
|
||||
## Building with Docker
|
||||
|
||||
For containerized deployments, you'll need to modify the Dockerfile. Here are two complete examples based on your target environment's libc.
|
||||
|
||||
### Option A: Alpine Linux (musl libc)
|
||||
|
||||
Use this for Alpine-based deployments or when you want minimal image size.
|
||||
|
||||
<Accordion title="Complete Dockerfile for Alpine (musl libc)">
|
||||
|
||||
```dockerfile
|
||||
# --- UI Build Stage: Build the React + Vite frontend ---
|
||||
FROM node:25-alpine3.23 AS ui-builder
|
||||
WORKDIR /app
|
||||
|
||||
# Copy UI package files and install dependencies
|
||||
COPY ui/package*.json ./
|
||||
RUN npm ci
|
||||
|
||||
# Copy UI source code
|
||||
COPY ui/ ./
|
||||
|
||||
# Build UI (skip the copy-build step)
|
||||
RUN npm run build-enterprise
|
||||
|
||||
# --- Go Build Stage: Compile the Go binary ---
|
||||
FROM golang:1.26.1-alpine3.23 AS builder
|
||||
WORKDIR /app
|
||||
|
||||
# Install dependencies including gcc for CGO and sqlite
|
||||
RUN apk add --no-cache gcc musl-dev sqlite-dev
|
||||
|
||||
# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
|
||||
ENV CGO_ENABLED=1 GOOS=linux
|
||||
|
||||
COPY transports/go.mod transports/go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code and dependencies
|
||||
COPY transports/ ./
|
||||
|
||||
COPY --from=ui-builder /app/out ./bifrost-http/ui
|
||||
|
||||
# Build the binary with CGO enabled for DYNAMIC LINKING
|
||||
ENV GOWORK=off
|
||||
ARG VERSION=unknown
|
||||
RUN go build \
|
||||
-ldflags="-w -s -X main.Version=v${VERSION}" \
|
||||
-a -trimpath \
|
||||
-o /app/main \
|
||||
./bifrost-http
|
||||
|
||||
# Verify build succeeded
|
||||
RUN test -f /app/main || (echo "Build failed" && exit 1)
|
||||
|
||||
# --- Runtime Stage: Minimal runtime image ---
|
||||
FROM alpine:3.23
|
||||
WORKDIR /app
|
||||
|
||||
# Install runtime dependencies for CGO-enabled dynamic binary
|
||||
# musl: C standard library (required for CGO binaries)
|
||||
# libgcc: GCC runtime library
|
||||
# ca-certificates: For HTTPS connections
|
||||
# wget: For healthcheck
|
||||
RUN apk add --no-cache musl libgcc ca-certificates wget
|
||||
|
||||
# Create data directory and set up user
|
||||
COPY --from=builder /app/main .
|
||||
COPY --from=builder /app/docker-entrypoint.sh .
|
||||
|
||||
# Getting arguments
|
||||
ARG ARG_APP_PORT=8080
|
||||
ARG ARG_APP_HOST=0.0.0.0
|
||||
ARG ARG_LOG_LEVEL=info
|
||||
ARG ARG_LOG_STYLE=json
|
||||
ARG ARG_APP_DIR=/app/data
|
||||
|
||||
# Environment variables with defaults (can be overridden at runtime)
|
||||
ENV APP_PORT=$ARG_APP_PORT \
|
||||
APP_HOST=$ARG_APP_HOST \
|
||||
LOG_LEVEL=$ARG_LOG_LEVEL \
|
||||
LOG_STYLE=$ARG_LOG_STYLE \
|
||||
APP_DIR=$ARG_APP_DIR
|
||||
|
||||
RUN mkdir -p $APP_DIR/logs && \
|
||||
adduser -D -s /bin/sh appuser && \
|
||||
chown -R appuser:appuser /app && \
|
||||
chmod +x /app/docker-entrypoint.sh
|
||||
USER appuser
|
||||
|
||||
# Declare volume for data persistence
|
||||
VOLUME ["/app/data"]
|
||||
EXPOSE $APP_PORT
|
||||
|
||||
# Health check for container status monitoring
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1
|
||||
|
||||
# Use entrypoint script that handles volume permissions and argument processing
|
||||
ENTRYPOINT ["/app/docker-entrypoint.sh"]
|
||||
CMD ["/app/main"]
|
||||
```
|
||||
|
||||
</Accordion>
|
||||
|
||||
**Key changes from static build:**
|
||||
- Line 40-44: Removed `-extldflags '-static'` and `-tags "sqlite_static"`
|
||||
- Removed UPX compression step (optional, but simpler)
|
||||
- Runtime uses musl libc from Alpine base image
|
||||
|
||||
**Build and run:**
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -f transports/Dockerfile -t bifrost:dynamic-alpine .
|
||||
|
||||
# Run the container
|
||||
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-alpine
|
||||
```
|
||||
|
||||
### Option B: Debian (glibc)
|
||||
|
||||
Use this for Debian/Ubuntu-based deployments or when deploying to glibc-based systems.
|
||||
|
||||
<Accordion title="Complete Dockerfile for Debian (glibc)">
|
||||
|
||||
```dockerfile
|
||||
# --- UI Build Stage: Build the React + Vite frontend ---
|
||||
FROM node:25-bookworm AS ui-builder
|
||||
WORKDIR /app
|
||||
|
||||
# Copy UI package files and install dependencies
|
||||
COPY ui/package*.json ./
|
||||
RUN npm ci
|
||||
|
||||
# Copy UI source code
|
||||
COPY ui/ ./
|
||||
|
||||
# Build UI
|
||||
RUN npm run build-enterprise
|
||||
|
||||
# --- Go Build Stage: Compile the Go binary ---
|
||||
FROM golang:1.26.1-bookworm AS builder
|
||||
WORKDIR /app
|
||||
|
||||
# Install dependencies including gcc for CGO and sqlite
|
||||
RUN apt-get update && apt-get install -y \
|
||||
gcc \
|
||||
libc6-dev \
|
||||
libsqlite3-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Set environment for CGO-enabled build (required for go-sqlite3 and plugins)
|
||||
ENV CGO_ENABLED=1 GOOS=linux
|
||||
|
||||
COPY transports/go.mod transports/go.sum ./
|
||||
RUN go mod download
|
||||
|
||||
# Copy source code and dependencies
|
||||
COPY transports/ ./
|
||||
|
||||
COPY --from=ui-builder /app/out ./bifrost-http/ui
|
||||
|
||||
# Build the binary with CGO enabled for DYNAMIC LINKING
|
||||
ENV GOWORK=off
|
||||
ARG VERSION=unknown
|
||||
RUN go build \
|
||||
-ldflags="-w -s -X main.Version=v${VERSION}" \
|
||||
-a -trimpath \
|
||||
-o /app/main \
|
||||
./bifrost-http
|
||||
|
||||
# Verify build succeeded
|
||||
RUN test -f /app/main || (echo "Build failed" && exit 1)
|
||||
|
||||
# --- Runtime Stage: Minimal runtime image ---
|
||||
FROM debian:bookworm-slim
|
||||
WORKDIR /app
|
||||
|
||||
# Install runtime dependencies for CGO-enabled dynamic binary
|
||||
# libc6: GNU C Library (required for glibc-linked binaries)
|
||||
# ca-certificates: For HTTPS connections
|
||||
RUN apt-get update && apt-get install -y \
|
||||
libc6 \
|
||||
ca-certificates \
|
||||
wget \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Create data directory and set up user
|
||||
COPY --from=builder /app/main .
|
||||
COPY --from=builder /app/docker-entrypoint.sh .
|
||||
|
||||
# Getting arguments
|
||||
ARG ARG_APP_PORT=8080
|
||||
ARG ARG_APP_HOST=0.0.0.0
|
||||
ARG ARG_LOG_LEVEL=info
|
||||
ARG ARG_LOG_STYLE=json
|
||||
ARG ARG_APP_DIR=/app/data
|
||||
|
||||
# Environment variables with defaults (can be overridden at runtime)
|
||||
ENV APP_PORT=$ARG_APP_PORT \
|
||||
APP_HOST=$ARG_APP_HOST \
|
||||
LOG_LEVEL=$ARG_LOG_LEVEL \
|
||||
LOG_STYLE=$ARG_LOG_STYLE \
|
||||
APP_DIR=$ARG_APP_DIR
|
||||
|
||||
RUN mkdir -p $APP_DIR/logs && \
|
||||
useradd -m -s /bin/sh appuser && \
|
||||
chown -R appuser:appuser /app && \
|
||||
chmod +x /app/docker-entrypoint.sh
|
||||
USER appuser
|
||||
|
||||
# Declare volume for data persistence
|
||||
VOLUME ["/app/data"]
|
||||
EXPOSE $APP_PORT
|
||||
|
||||
# Health check for container status monitoring
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD wget --no-verbose --tries=1 --spider http://127.0.0.1:${APP_PORT}/metrics || exit 1
|
||||
|
||||
# Use entrypoint script that handles volume permissions and argument processing
|
||||
ENTRYPOINT ["/app/docker-entrypoint.sh"]
|
||||
CMD ["/app/main"]
|
||||
```
|
||||
|
||||
</Accordion>
|
||||
|
||||
**Key differences from Alpine version:**
|
||||
- Uses `bookworm` (Debian 12) base images instead of Alpine
|
||||
- Installs `apt` packages instead of `apk`
|
||||
- Runtime uses glibc (libc6) instead of musl
|
||||
- Uses `useradd` instead of `adduser` for user creation
|
||||
|
||||
**Build and run:**
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -f transports/Dockerfile.debian -t bifrost:dynamic-debian .
|
||||
|
||||
# Run the container
|
||||
docker run -p 8080:8080 -v ./plugins:/app/data/plugins bifrost:dynamic-debian
|
||||
```
|
||||
|
||||
## libc Compatibility
|
||||
|
||||
Understanding libc (C standard library) compatibility is **critical** when building dynamic binaries and plugins.
|
||||
|
||||
### musl vs glibc
|
||||
|
||||
Linux distributions use one of two main C standard libraries:
|
||||
|
||||
| libc Type | Used By | Characteristics |
|
||||
|-----------|---------|-----------------|
|
||||
| **musl** | Alpine Linux | Lightweight, minimal, security-focused |
|
||||
| **glibc** | Debian, Ubuntu, RHEL, CentOS, Fedora, Amazon Linux | Standard GNU C Library, feature-rich |
|
||||
|
||||
### The Golden Rule
|
||||
|
||||
<Warning>
|
||||
- **A binary built with musl will NOT run on glibc systems.**
|
||||
- **A binary built with glibc will NOT run on musl systems.**
|
||||
- **Plugins and Bifrost MUST use the same libc.**
|
||||
</Warning>
|
||||
|
||||
### Why This Matters
|
||||
|
||||
When you build a dynamic binary:
|
||||
|
||||
```bash
|
||||
# Built on Alpine (musl)
|
||||
$ ldd bifrost-http
|
||||
linux-vdso.so.1
|
||||
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1
|
||||
|
||||
# Built on Debian (glibc)
|
||||
$ ldd bifrost-http
|
||||
linux-vdso.so.1
|
||||
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
|
||||
```
|
||||
|
||||
The binary is linked to a **specific** libc implementation. If you try to run it on a system with a different libc, you'll get errors like:
|
||||
|
||||
```
|
||||
error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file
|
||||
```
|
||||
|
||||
### Choosing Your Build Environment
|
||||
|
||||
**Decision Matrix:**
|
||||
|
||||
| Target Deployment | Build With | Dockerfile Base |
|
||||
|-------------------|------------|-----------------|
|
||||
| Alpine containers | musl | `golang:1.26.1-alpine3.23` |
|
||||
| Debian/Ubuntu containers | glibc | `golang:1.26.1-bookworm` |
|
||||
| Ubuntu/Debian servers | glibc | `golang:1.26.1-bookworm` |
|
||||
| RHEL/CentOS servers | glibc | Native build or glibc container |
|
||||
| Kubernetes (Alpine) | musl | `golang:1.26.1-alpine3.23` |
|
||||
| Kubernetes (Debian) | glibc | `golang:1.26.1-bookworm` |
|
||||
|
||||
**Simple rule:** Build with the same base OS family as your deployment target.
|
||||
|
||||
### Building Plugins
|
||||
|
||||
Plugins **must** be built with the **exact same environment** as your Bifrost binary:
|
||||
|
||||
```bash
|
||||
# If Bifrost was built with Alpine/musl
|
||||
docker run --rm \
|
||||
-v "$PWD:/work" \
|
||||
-w /work \
|
||||
golang:1.26.1-alpine3.23 \
|
||||
sh -c "apk add --no-cache gcc musl-dev && \
|
||||
go build -buildmode=plugin -o myplugin.so main.go"
|
||||
|
||||
# If Bifrost was built with Debian/glibc
|
||||
docker run --rm \
|
||||
-v "$PWD:/work" \
|
||||
-w /work \
|
||||
golang:1.26.1-bookworm \
|
||||
sh -c "apt-get update && apt-get install -y gcc && \
|
||||
go build -buildmode=plugin -o myplugin.so main.go"
|
||||
```
|
||||
|
||||
See the [hello-world plugin Makefile](https://github.com/maximhq/bifrost/blob/main/examples/plugins/hello-world/Makefile) for a complete example.
|
||||
|
||||
## Verification
|
||||
|
||||
### Verify Dynamic Linking
|
||||
|
||||
After building, check that your binary is dynamically linked:
|
||||
|
||||
```bash
|
||||
# Check binary dependencies
|
||||
ldd tmp/bifrost-http
|
||||
|
||||
# Expected output (musl):
|
||||
linux-vdso.so.1
|
||||
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1
|
||||
|
||||
# Expected output (glibc):
|
||||
linux-vdso.so.1
|
||||
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
|
||||
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
|
||||
```
|
||||
|
||||
If you see `statically linked`, the binary **will not load plugins**.
|
||||
|
||||
### Verify Plugin Compatibility
|
||||
|
||||
Test that your plugin loads successfully:
|
||||
|
||||
```bash
|
||||
# Start Bifrost with your plugin configured
|
||||
./tmp/bifrost-http -config config.json
|
||||
|
||||
# Check logs for plugin initialization
|
||||
# Should see: "Plugin loaded successfully: your-plugin-name"
|
||||
```
|
||||
|
||||
## Go Version and Package Compatibility
|
||||
|
||||
### Go Version Requirement
|
||||
|
||||
Bifrost is built with **Go 1.26.1**. Your plugin **must** be compiled with the exact same Go version to ensure compatibility.
|
||||
|
||||
```bash
|
||||
# Check your Go version
|
||||
go version
|
||||
# Should output: go version go1.26.1 ...
|
||||
|
||||
# If you need to install Go 1.26.1
|
||||
# Visit: https://go.dev/dl/
|
||||
```
|
||||
|
||||
### Key Package Versions
|
||||
|
||||
Bifrost uses the following key packages across its three main modules that may affect plugin development:
|
||||
|
||||
#### Transport Layer (`transports/go.mod`)
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/bytedance/sonic` | v1.14.1 | High-performance JSON serialization |
|
||||
| `github.com/valyala/fasthttp` | v1.67.0 | Fast HTTP server/client |
|
||||
| `github.com/fasthttp/router` | v1.5.4 | HTTP router for fasthttp |
|
||||
| `github.com/fasthttp/websocket` | v1.5.12 | WebSocket support |
|
||||
| `github.com/prometheus/client_golang` | v1.23.0 | Prometheus metrics |
|
||||
| `gorm.io/gorm` | v1.31.1 | Database ORM |
|
||||
|
||||
#### Core Layer (`core/go.mod`)
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/bytedance/sonic` | v1.14.1 | High-performance JSON serialization |
|
||||
| `github.com/valyala/fasthttp` | v1.67.0 | Fast HTTP client for providers |
|
||||
| `github.com/google/uuid` | v1.6.0 | UUID generation |
|
||||
| `github.com/rs/zerolog` | v1.34.0 | Zero-allocation JSON logger |
|
||||
| `github.com/mark3labs/mcp-go` | v0.41.1 | Model Context Protocol support |
|
||||
| `golang.org/x/oauth2` | v0.32.0 | OAuth2 client |
|
||||
|
||||
#### Framework Layer (`framework/go.mod`)
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `github.com/redis/go-redis/v9` | v9.14.0 | Redis client for caching |
|
||||
| `github.com/weaviate/weaviate-go-client/v5` | v5.5.0 | Weaviate vector store client |
|
||||
| `github.com/mattn/go-sqlite3` | v1.14.32 | SQLite3 driver (requires CGO) |
|
||||
| `gorm.io/gorm` | v1.31.1 | Database ORM |
|
||||
| `gorm.io/driver/sqlite` | v1.6.0 | GORM SQLite driver |
|
||||
| `gorm.io/driver/postgres` | v1.6.0 | GORM PostgreSQL driver |
|
||||
| `golang.org/x/crypto` | v0.43.0 | Cryptographic functions |
|
||||
|
||||
<Note>
|
||||
If your plugin imports any of these packages, use compatible versions to avoid runtime issues. Check `transports/go.mod`, `core/go.mod`, and `framework/go.mod` for complete dependency lists.
|
||||
</Note>
|
||||
|
||||
### Checking Bifrost's Dependencies
|
||||
|
||||
To see all dependencies used by Bifrost across its three main modules:
|
||||
|
||||
```bash
|
||||
# View transport layer dependencies
|
||||
cat transports/go.mod
|
||||
|
||||
# View core dependencies
|
||||
cat core/go.mod
|
||||
|
||||
# View framework dependencies
|
||||
cat framework/go.mod
|
||||
|
||||
# Or list all dependencies for a specific module
|
||||
cd transports && go list -m all
|
||||
cd ../core && go list -m all
|
||||
cd ../framework && go list -m all
|
||||
```
|
||||
|
||||
### Plugin go.mod Example
|
||||
|
||||
When creating a plugin, your `go.mod` should match Bifrost's Go version:
|
||||
|
||||
```go
|
||||
module github.com/example/my-plugin
|
||||
|
||||
go 1.26.1
|
||||
|
||||
require (
|
||||
github.com/maximhq/bifrost/core v1.2.38
|
||||
// Optional: Add framework for advanced features
|
||||
// github.com/maximhq/bifrost/framework v1.1.48
|
||||
|
||||
// Add other dependencies as needed, matching versions from Bifrost's go.mod files
|
||||
// github.com/bytedance/sonic v1.14.1
|
||||
// github.com/rs/zerolog v1.34.0
|
||||
)
|
||||
```
|
||||
|
||||
<Tip>
|
||||
Import only the Bifrost modules you need. Most plugins only require `core`. Use `framework` if you need access to config stores, vector stores, or other framework features.
|
||||
</Tip>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Errors
|
||||
|
||||
#### 1. Cannot load plugin - Go version mismatch
|
||||
|
||||
```
|
||||
cannot load plugin: plugin was built with a different version of package runtime/internal/sys
|
||||
```
|
||||
|
||||
**Cause:** Plugin and Bifrost were built with different Go versions.
|
||||
|
||||
**Solution:** Use the exact same Go version (Go 1.26.1) for both:
|
||||
|
||||
```bash
|
||||
# Check Go version used for Bifrost
|
||||
./tmp/bifrost-http -version
|
||||
|
||||
# Verify your Go version matches
|
||||
go version # Should output: go version go1.26.1
|
||||
|
||||
# See full compatibility requirements
|
||||
```
|
||||
|
||||
Refer to [Go Version and Package Compatibility](#go-version-and-package-compatibility) for details.
|
||||
|
||||
#### 2. Shared library not found
|
||||
|
||||
```
|
||||
error while loading shared libraries: libc.musl-x86_64.so.1: cannot open shared object file
|
||||
```
|
||||
|
||||
**Cause:** Binary built with musl trying to run on glibc system (or vice versa).
|
||||
|
||||
**Solution:** Rebuild with the correct libc for your target system.
|
||||
|
||||
#### 3. Plugin architecture mismatch
|
||||
|
||||
```
|
||||
plugin was built with a different version of package internal/cpu
|
||||
```
|
||||
|
||||
**Cause:** Plugin and Bifrost built for different architectures (amd64 vs arm64).
|
||||
|
||||
**Solution:** Ensure `GOARCH` matches for both builds:
|
||||
|
||||
```bash
|
||||
# Check architecture
|
||||
uname -m # x86_64 = amd64, aarch64 = arm64
|
||||
|
||||
# Build with explicit architecture
|
||||
GOARCH=amd64 go build ...
|
||||
```
|
||||
|
||||
#### 4. Plugin file not found
|
||||
|
||||
```
|
||||
plugin.Open("myplugin.so"): realpath failed: no such file or directory
|
||||
```
|
||||
|
||||
**Cause:** Plugin file path is incorrect in config.
|
||||
|
||||
**Solution:** Use absolute paths or verify relative paths:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"path": "/app/data/plugins/myplugin.so",
|
||||
"config": {}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Document Your Build Environment
|
||||
|
||||
Create a `BUILD.md` file documenting:
|
||||
- Go version used
|
||||
- Base image (Alpine vs Debian)
|
||||
- Build commands
|
||||
- Target deployment platform
|
||||
|
||||
### 2. Use Consistent Tooling
|
||||
|
||||
Match Bifrost's exact Go version and key dependencies (see [Go Version and Package Compatibility](#go-version-and-package-compatibility)):
|
||||
|
||||
```bash
|
||||
# Pin Go version in Dockerfile
|
||||
FROM golang:1.26.1-alpine3.23 AS builder
|
||||
|
||||
# Pin Go version in Makefile/CI
|
||||
GO_VERSION=1.26.1
|
||||
```
|
||||
|
||||
### 3. Test Plugin Loading Locally
|
||||
|
||||
Before deploying, test plugin loading:
|
||||
|
||||
```bash
|
||||
# Build both Bifrost and plugin
|
||||
make build DYNAMIC=1
|
||||
cd examples/plugins/hello-world && make build
|
||||
|
||||
# Test loading
|
||||
./tmp/bifrost-http -config examples/plugins/hello-world/config.json
|
||||
```
|
||||
|
||||
### 4. Version Your Plugins
|
||||
|
||||
Tag plugin builds with version and build info:
|
||||
|
||||
```bash
|
||||
go build -buildmode=plugin \
|
||||
-ldflags="-X main.Version=v1.0.0 -X main.BuildTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
-o myplugin-v1.0.0.so
|
||||
```
|
||||
|
||||
### 5. Multi-Stage Dockerfiles for Plugins
|
||||
|
||||
Build plugins in the same Dockerfile as Bifrost:
|
||||
|
||||
```dockerfile
|
||||
# Build plugin
|
||||
FROM golang:1.26.1-alpine3.23 AS plugin-builder
|
||||
WORKDIR /plugin
|
||||
COPY plugins/myplugin/ .
|
||||
RUN apk add --no-cache gcc musl-dev && \
|
||||
go build -buildmode=plugin -o myplugin.so main.go
|
||||
|
||||
# Build Bifrost
|
||||
FROM golang:1.26.1-alpine3.23 AS bifrost-builder
|
||||
# ... (bifrost build steps)
|
||||
|
||||
# Runtime
|
||||
FROM alpine:3.23
|
||||
COPY --from=bifrost-builder /app/main .
|
||||
COPY --from=plugin-builder /plugin/myplugin.so /app/plugins/
|
||||
```
|
||||
|
||||
This ensures plugins and Bifrost use identical build environments.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that you have a dynamically linked Bifrost binary:
|
||||
|
||||
1. **[Write your first plugin](./writing-plugin)** - Learn the plugin API and create custom functionality
|
||||
2. **[Deploy with plugins](../deployment-guides)** - Best practices for production deployments
|
||||
3. **[Example plugins](https://github.com/maximhq/bifrost/tree/main/examples/plugins)** - Study working examples
|
||||
|
||||
<Note>
|
||||
For questions or issues with dynamic builds and plugins, visit our [GitHub Discussions](https://github.com/maximhq/bifrost/discussions) or [Discord community](https://discord.gg/exN5KAydbU).
|
||||
</Note>
|
||||
|
||||
113
docs/plugins/getting-started.mdx
Normal file
113
docs/plugins/getting-started.mdx
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
title: "Getting Started"
|
||||
description: "Learn how to extend Bifrost's functionality by creating custom plugins that intercept and modify requests and responses."
|
||||
icon: "book"
|
||||
---
|
||||
|
||||
<Note>
|
||||
Dynamic plugins require dynamic builds of Bifrost which are not enabled by default to keep Bifrost setup easier. If you want to build and try custom plugins on OSS read [building dynamically linked Bifrost binary](./building-dynamic-binary)
|
||||
</Note>
|
||||
|
||||
|
||||
## What are Bifrost Plugins?
|
||||
|
||||
Bifrost plugins allow you to extend the gateway's functionality by intercepting requests and responses. Plugins can modify, log, validate, or enrich data as it flows through the system, giving you powerful hooks into Bifrost's request lifecycle.
|
||||
|
||||
## Use Cases
|
||||
|
||||
Custom plugins enable you to:
|
||||
|
||||
- **Transform requests and responses** - Modify data before it reaches providers or after it returns
|
||||
- **Add custom validation** - Enforce business rules on incoming requests
|
||||
- **Implement custom caching** - Cache responses based on custom logic
|
||||
- **Integrate with external systems** - Send data to logging, monitoring, or analytics platforms
|
||||
- **Apply custom transformations** - Parse, filter, or enrich LLM responses
|
||||
|
||||
## Plugin Architecture
|
||||
|
||||

|
||||
|
||||
Bifrost leverages **Go's native plugin system** to enable dynamic extensibility. Plugins are built as **shared object files** (`.so` files) that are loaded at runtime by the Bifrost gateway.
|
||||
|
||||
### How Go Plugins Work
|
||||
|
||||
Go plugins use the `plugin` package from the standard library, which allows Go programs to dynamically load code at runtime. Here's what makes this approach powerful:
|
||||
|
||||
- **Native Go Integration** - Plugins are written in Go and have full access to Bifrost's type system and interfaces
|
||||
- **Dynamic Loading** - Plugins can be loaded, unloaded, and reloaded without restarting Bifrost
|
||||
- **Type Safety** - Go's type system ensures plugin methods match expected signatures
|
||||
- **Performance** - No IPC overhead; plugins run in the same process as Bifrost
|
||||
|
||||
### Building Shared Objects
|
||||
|
||||
Plugins must be compiled as shared objects using Go's `-buildmode=plugin` flag:
|
||||
|
||||
```bash
|
||||
go build -buildmode=plugin -o myplugin.so main.go
|
||||
```
|
||||
|
||||
This generates a `.so` file that exports specific functions matching Bifrost's plugin interface:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="v1.4.x+">
|
||||
- `Init(config any) error` - Initialize the plugin with configuration
|
||||
- `GetName() string` - Return the plugin name
|
||||
- `HTTPTransportPreHook()` - Intercept HTTP requests before they enter Bifrost core (HTTP transport only)
|
||||
- `HTTPTransportPostHook()` - Intercept HTTP responses after they exit Bifrost core (HTTP transport only)
|
||||
- `PreLLMHook()` - Intercept requests before they reach providers
|
||||
- `PostLLMHook()` - Process responses after provider calls
|
||||
- `Cleanup() error` - Clean up resources on shutdown
|
||||
</Tab>
|
||||
<Tab title="v1.3.x">
|
||||
- `Init(config any) error` - Initialize the plugin with configuration
|
||||
- `GetName() string` - Return the plugin name
|
||||
- `TransportInterceptor()` - Modify raw HTTP headers/body (HTTP transport only)
|
||||
- `PreLLMHook()` - Intercept requests before they reach providers
|
||||
- `PostLLMHook()` - Process responses after provider calls
|
||||
- `Cleanup() error` - Clean up resources on shutdown
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### Platform Requirements
|
||||
|
||||
**Important Limitations:**
|
||||
|
||||
- **Supported Platforms**: Linux and macOS (Darwin) only
|
||||
- **No Cross-Compilation**: Plugins must be built on the target platform
|
||||
- **Architecture Matching**: Plugin and Bifrost must use the same architecture (amd64, arm64)
|
||||
- **Go Version Compatibility**: Plugin must be built with the same Go version as Bifrost
|
||||
|
||||
This means if you're running Bifrost on Linux AMD64, you must build your plugin on Linux AMD64 with the same Go version.
|
||||
|
||||
### Plugin Lifecycle
|
||||
|
||||
1. **Load** - Bifrost loads the `.so` file using Go's `plugin.Open()`
|
||||
2. **Initialize** - Calls `Init()` with configuration from `config.json`
|
||||
3. **Hook Execution** - Calls `PreLLMHook()` and `PostLLMHook()` for each request
|
||||
4. **Cleanup** - Calls `Cleanup()` when Bifrost shuts down
|
||||
|
||||
Plugins execute in a specific order:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="v1.4.x+">
|
||||
1. `HTTPTransportPreHook` - Intercept HTTP requests (HTTP transport only)
|
||||
2. `PreLLMHook`/`PreMCPHook` - Executes in registration order, can short-circuit requests
|
||||
3. Provider call (if not short-circuited)
|
||||
4. `PostLLMHook`/`PostMCPHook` - Executes in reverse order of PreHooks
|
||||
5. `HTTPTransportPostHook` - Intercept HTTP responses (HTTP transport only, reverse order)
|
||||
</Tab>
|
||||
<Tab title="v1.3.x">
|
||||
1. `TransportInterceptor` - Modifies raw HTTP requests (HTTP transport only)
|
||||
2. `PreHook` - Executes in registration order, can short-circuit requests
|
||||
3. Provider call (if not short-circuited)
|
||||
4. `PostHook` - Executes in reverse order of PreHooks
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Next Steps
|
||||
|
||||
Ready to build your first plugin? Choose your approach:
|
||||
|
||||
- **[Writing Go Plugins](./writing-go-plugin)** - Native Go plugins using shared objects (`.so` files). Best for performance and full Go ecosystem access.
|
||||
- **[Writing WASM Plugins](./writing-wasm-plugin)** - Cross-platform plugins using WebAssembly. Write in TypeScript, Go (TinyGo), or Rust. No version matching required.
|
||||
|
||||
465
docs/plugins/migration-guide.mdx
Normal file
465
docs/plugins/migration-guide.mdx
Normal file
@@ -0,0 +1,465 @@
|
||||
---
|
||||
title: "Plugin Migration Guide"
|
||||
description: "How to migrate your Bifrost plugins from v1.3.x to v1.4.x"
|
||||
icon: "arrow-up-right-dots"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Bifrost v1.4.x introduces a new plugin interface for HTTP transport layer interception. This guide helps you migrate existing plugins from the v1.3.x `TransportInterceptor` pattern to the v1.4.x `HTTPTransportPreHook` and `HTTPTransportPostHook` pattern.
|
||||
|
||||
<Note>
|
||||
If your plugin doesn't use `TransportInterceptor`, no migration is needed. The `PreLLMHook`, `PostLLMHook`, `Init`, `GetName`, and `Cleanup` functions remain unchanged.
|
||||
</Note>
|
||||
|
||||
## What Changed?
|
||||
|
||||
The HTTP transport interception mechanism changed from a simple function that receives and returns headers/body to a dual-hook pattern that works with both native `.so` plugins and WASM plugins.
|
||||
|
||||
### Key Differences
|
||||
|
||||
| Aspect | v1.3.x (TransportInterceptor) | v1.4.x+ (Pre/Post Hooks) |
|
||||
|--------|-------------------------------|--------------------------|
|
||||
| Signature | `TransportInterceptor(ctx, url, headers, body)` | `HTTPTransportPreHook(ctx, req)` + `HTTPTransportPostHook(ctx, req, resp)` |
|
||||
| Return type | `(headers, body, error)` | Pre: `(*HTTPResponse, error)`, Post: `error` |
|
||||
| Request type | Separate `headers map`, `body map` | Unified `*HTTPRequest` struct |
|
||||
| Response access | Not available | Post-hook receives `*HTTPResponse` |
|
||||
| Modification | Return modified maps | Modify `req`/`resp` in-place |
|
||||
| Short-circuit | Return error | Return `*HTTPResponse` |
|
||||
| WASM support | No | Yes |
|
||||
| Context | Limited `BifrostContext` | Full `*BifrostContext` with `SetValue`/`Value` |
|
||||
|
||||
### Why the Change?
|
||||
|
||||
The new dual-hook pattern provides:
|
||||
|
||||
1. **WASM plugin support** - Serializable types work across WASM boundary
|
||||
2. **Response interception** - Post-hook can modify responses before returning to client
|
||||
3. **Simpler API** - No middleware wrapper, direct function call
|
||||
4. **Better testability** - No fasthttp dependency in plugin tests
|
||||
5. **Full context access** - BifrostContext available for sharing data between hooks
|
||||
6. **Custom response short-circuits** - Return a full response to short-circuit
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### Step 1: Update Imports
|
||||
|
||||
Remove the `fasthttp` import if present:
|
||||
|
||||
```go
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/maximhq/bifrost/core/schemas"
|
||||
// Remove: "github.com/valyala/fasthttp"
|
||||
)
|
||||
```
|
||||
|
||||
### Step 2: Replace the Function
|
||||
|
||||
**Before (v1.3.x):**
|
||||
|
||||
```go
|
||||
// TransportInterceptor modifies raw HTTP headers and body
|
||||
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
|
||||
// Add custom header
|
||||
headers["X-Custom-Header"] = "value"
|
||||
|
||||
// Modify body
|
||||
body["custom_field"] = "custom_value"
|
||||
|
||||
return headers, body, nil
|
||||
}
|
||||
```
|
||||
|
||||
**After (v1.4.x+):**
|
||||
|
||||
```go
|
||||
// HTTPTransportPreHook intercepts requests BEFORE they enter Bifrost core
|
||||
// Modify req in-place. Return (*HTTPResponse, nil) to short-circuit.
|
||||
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
|
||||
// Add custom header (in-place modification)
|
||||
req.Headers["x-custom-header"] = "value"
|
||||
|
||||
// Modify body (in-place modification)
|
||||
var body map[string]any
|
||||
sonic.Unmarshal(req.Body, &body)
|
||||
body["custom_field"] = "custom_value"
|
||||
req.Body, _ = sonic.Marshal(body)
|
||||
|
||||
// Store values in context for use in post-hook
|
||||
ctx.SetValue(schemas.BifrostContextKey("my-plugin-key"), "my-value")
|
||||
|
||||
// Return nil to continue, or return &HTTPResponse{} to short-circuit
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// HTTPTransportPostHook intercepts responses AFTER they exit Bifrost core
|
||||
// Modify resp in-place. Called in reverse order of pre-hooks.
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Add response header
|
||||
resp.Headers["x-processed-by"] = "my-plugin"
|
||||
|
||||
// Read values set in pre-hook
|
||||
if val := ctx.Value(schemas.BifrostContextKey("my-plugin-key")); val != nil {
|
||||
fmt.Println("Context value:", val)
|
||||
}
|
||||
|
||||
// Return nil to continue, or return error to short-circuit
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Update Body Modification Logic
|
||||
|
||||
In v1.3.x, you received the body as a `map[string]any`. In v1.4.x, you work with `req.Body` bytes:
|
||||
|
||||
**Before (v1.3.x):**
|
||||
|
||||
```go
|
||||
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
|
||||
// Direct map access
|
||||
body["model"] = "gpt-4"
|
||||
return headers, body, nil
|
||||
}
|
||||
```
|
||||
|
||||
**After (v1.4.x+):**
|
||||
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
|
||||
// Parse body
|
||||
var body map[string]any
|
||||
if err := sonic.Unmarshal(req.Body, &body); err == nil {
|
||||
// Modify body
|
||||
body["model"] = "gpt-4"
|
||||
// Update req.Body in-place
|
||||
req.Body, _ = sonic.Marshal(body)
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Modify response body if needed
|
||||
var respBody map[string]any
|
||||
if err := sonic.Unmarshal(resp.Body, &respBody); err == nil {
|
||||
respBody["plugin_processed"] = true
|
||||
resp.Body, _ = sonic.Marshal(respBody)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
## Common Migration Patterns
|
||||
|
||||
### Adding Headers
|
||||
|
||||
**v1.3.x:**
|
||||
```go
|
||||
headers["authorization"] = "Bearer " + token
|
||||
return headers, body, nil
|
||||
```
|
||||
|
||||
**v1.4.x+:**
|
||||
```go
|
||||
// In HTTPTransportPreHook - modify request headers
|
||||
req.Headers["authorization"] = "Bearer " + token
|
||||
return nil, nil
|
||||
|
||||
// In HTTPTransportPostHook - modify response headers
|
||||
resp.Headers["x-request-id"] = requestID
|
||||
return nil
|
||||
```
|
||||
|
||||
### Reading Headers
|
||||
|
||||
**v1.3.x:**
|
||||
```go
|
||||
apiKey := headers["X-API-Key"]
|
||||
```
|
||||
|
||||
**v1.4.x+:**
|
||||
```go
|
||||
// Use case-insensitive helper for reading (recommended)
|
||||
apiKey := req.CaseInsensitiveHeaderLookup("X-API-Key")
|
||||
|
||||
// Or direct map access (case-sensitive)
|
||||
apiKey := req.Headers["x-api-key"]
|
||||
```
|
||||
|
||||
### Conditional Processing
|
||||
|
||||
**v1.3.x:**
|
||||
```go
|
||||
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
|
||||
if headers["x-skip-processing"] == "true" {
|
||||
return headers, body, nil
|
||||
}
|
||||
// Process...
|
||||
return headers, body, nil
|
||||
}
|
||||
```
|
||||
|
||||
**v1.4.x+:**
|
||||
```go
|
||||
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
|
||||
if req.CaseInsensitiveHeaderLookup("x-skip-processing") == "true" {
|
||||
return nil, nil // Continue without modification
|
||||
}
|
||||
// Process...
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Post-hook always runs unless pre-hook short-circuited
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Error Handling / Short-Circuit
|
||||
|
||||
**v1.3.x:**
|
||||
```go
|
||||
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
|
||||
if headers["x-api-key"] == "" {
|
||||
return nil, nil, fmt.Errorf("missing API key")
|
||||
}
|
||||
return headers, body, nil
|
||||
}
|
||||
```
|
||||
|
||||
**v1.4.x+:**
|
||||
```go
|
||||
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
|
||||
if req.CaseInsensitiveHeaderLookup("x-api-key") == "" {
|
||||
// Return a custom response to short-circuit
|
||||
return &schemas.HTTPResponse{
|
||||
StatusCode: 401,
|
||||
Headers: map[string]string{"Content-Type": "application/json"},
|
||||
Body: []byte(`{"error": "missing API key"}`),
|
||||
}, nil
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Not called if pre-hook short-circuited
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Accessing Request Method and Path
|
||||
|
||||
**v1.3.x:**
|
||||
```go
|
||||
// url parameter contained the full URL
|
||||
func TransportInterceptor(ctx *schemas.BifrostContext, url string, headers map[string]string, body map[string]any) (map[string]string, map[string]any, error) {
|
||||
// Limited access to URL
|
||||
return headers, body, nil
|
||||
}
|
||||
```
|
||||
|
||||
**v1.4.x+:**
|
||||
```go
|
||||
func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error) {
|
||||
// Full access to request properties
|
||||
method := req.Method // "GET", "POST", etc.
|
||||
path := req.Path // "/v1/chat/completions"
|
||||
query := req.Query // map[string]string of query params
|
||||
pathParams := req.PathParams // map[string]string of path variables (e.g., {model})
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Access both request and response
|
||||
statusCode := resp.StatusCode
|
||||
responseHeaders := resp.Headers
|
||||
responseBody := resp.Body
|
||||
_ = statusCode // Use variables...
|
||||
_ = responseHeaders
|
||||
_ = responseBody
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
## Testing Your Migration
|
||||
|
||||
1. **Build your updated plugin:**
|
||||
```bash
|
||||
go build -buildmode=plugin -o my-plugin.so main.go
|
||||
```
|
||||
|
||||
2. **Update Bifrost to v1.4.x:**
|
||||
```bash
|
||||
go get github.com/maximhq/bifrost/core@v1.4.0
|
||||
```
|
||||
|
||||
3. **Test with a simple request:**
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
|
||||
4. **Verify logs show both hooks being called:**
|
||||
```
|
||||
HTTPTransportPreHook called
|
||||
PreLLMHook called
|
||||
PostLLMHook called
|
||||
HTTPTransportPostHook called
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Plugin fails to load after migration
|
||||
|
||||
**Error:** `plugin: symbol TransportInterceptor not found`
|
||||
|
||||
This error occurs if Bifrost v1.4.x is looking for the old function. Make sure:
|
||||
1. You've updated to `HTTPTransportPreHook` and `HTTPTransportPostHook`
|
||||
2. The function signatures match exactly:
|
||||
- `func HTTPTransportPreHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest) (*schemas.HTTPResponse, error)`
|
||||
- `func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error`
|
||||
3. You've rebuilt the plugin with the correct core version
|
||||
|
||||
### Body modification not working
|
||||
|
||||
Make sure you're assigning back to `req.Body` in the pre-hook:
|
||||
|
||||
```go
|
||||
// Wrong - body changes lost
|
||||
var body map[string]any
|
||||
sonic.Unmarshal(req.Body, &body)
|
||||
body["model"] = "gpt-4"
|
||||
// Missing: req.Body = ...
|
||||
|
||||
// Correct - body changes applied
|
||||
var body map[string]any
|
||||
sonic.Unmarshal(req.Body, &body)
|
||||
body["model"] = "gpt-4"
|
||||
req.Body, _ = sonic.Marshal(body) // Assign back!
|
||||
```
|
||||
|
||||
### Response modification not working
|
||||
|
||||
Make sure you're modifying `resp` in the post-hook:
|
||||
|
||||
```go
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Modify response headers
|
||||
resp.Headers["x-custom-header"] = "value"
|
||||
|
||||
// Modify response body
|
||||
var body map[string]any
|
||||
sonic.Unmarshal(resp.Body, &body)
|
||||
body["extra_field"] = "value"
|
||||
resp.Body, _ = sonic.Marshal(body)
|
||||
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
### Headers not being set
|
||||
|
||||
Make sure you're modifying `req.Headers` or `resp.Headers` directly:
|
||||
|
||||
```go
|
||||
// Set request header in pre-hook
|
||||
req.Headers["x-custom-header"] = "value"
|
||||
|
||||
// Set response header in post-hook
|
||||
resp.Headers["x-custom-header"] = "value"
|
||||
|
||||
// Read headers using case-insensitive helper
|
||||
value := req.CaseInsensitiveHeaderLookup("X-Custom-Header")
|
||||
```
|
||||
|
||||
### Context values not available in post-hook
|
||||
|
||||
Make sure you're using the correct context key type:
|
||||
|
||||
```go
|
||||
// In pre-hook - set value
|
||||
ctx.SetValue(schemas.BifrostContextKey("my-key"), "my-value")
|
||||
|
||||
// In post-hook - read value
|
||||
if val := ctx.Value(schemas.BifrostContextKey("my-key")); val != nil {
|
||||
// Use val
|
||||
}
|
||||
```
|
||||
|
||||
## Streaming Chunk Hook (v1.4.x)
|
||||
|
||||
Bifrost v1.4.x introduces a new hook for intercepting streaming response chunks:
|
||||
|
||||
### HTTPTransportStreamChunkHook
|
||||
|
||||
This hook is called for each chunk during streaming responses, allowing plugins to modify or filter chunks before they're sent to the client.
|
||||
|
||||
```go
|
||||
// HTTPTransportStreamChunkHook intercepts streaming chunks BEFORE they're written to the client.
|
||||
// Modify chunk data or return nil to skip the chunk entirely.
|
||||
// Only called for streaming responses when using HTTP transport (bifrost-http).
|
||||
func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) {
|
||||
// chunk is a typed struct containing one of:
|
||||
// - BifrostTextCompletionResponse (text completion streaming)
|
||||
// - BifrostChatResponse (chat completion streaming)
|
||||
// - BifrostResponsesStreamResponse (responses API streaming)
|
||||
// - BifrostSpeechStreamResponse (speech synthesis streaming)
|
||||
// - BifrostTranscriptionStreamResponse (transcription streaming)
|
||||
// - BifrostImageGenerationStreamResponse (image generation streaming)
|
||||
// - BifrostError (error during streaming)
|
||||
|
||||
// Return chunk unchanged to pass through
|
||||
return chunk, nil
|
||||
|
||||
// Return nil to skip/filter this chunk
|
||||
// return nil, nil
|
||||
|
||||
// Return modified chunk
|
||||
// modifiedChunk := &schemas.BifrostStreamChunk{BifrostChatResponse: ...}
|
||||
// return modifiedChunk, nil
|
||||
}
|
||||
```
|
||||
|
||||
**Key differences from `HTTPTransportPostHook`:**
|
||||
|
||||
| Aspect | HTTPTransportPostHook | HTTPTransportStreamChunkHook |
|
||||
|--------|----------------------|------------------------------|
|
||||
| When called | After complete response | Per-chunk during streaming |
|
||||
| Input | Full HTTPResponse | `*BifrostStreamChunk` (typed struct) |
|
||||
| Can modify | Full response | Individual chunk struct |
|
||||
| Can skip | N/A | Return nil to skip chunk |
|
||||
|
||||
<Note>
|
||||
`HTTPTransportPostHook` is **not called** for streaming responses. Use `HTTPTransportStreamChunkHook` instead to intercept streaming data.
|
||||
</Note>
|
||||
|
||||
### Migration for Existing Plugins
|
||||
|
||||
If your plugin implements `HTTPTransportPostHook` and you want to also handle streaming responses, add the new hook:
|
||||
|
||||
```go
|
||||
// Existing hook for non-streaming responses
|
||||
func HTTPTransportPostHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, resp *schemas.HTTPResponse) error {
|
||||
// Handle complete responses
|
||||
return nil
|
||||
}
|
||||
|
||||
// NEW: Add this for streaming responses
|
||||
func HTTPTransportStreamChunkHook(ctx *schemas.BifrostContext, req *schemas.HTTPRequest, chunk *schemas.BifrostStreamChunk) (*schemas.BifrostStreamChunk, error) {
|
||||
// Handle streaming chunks (typed struct, not raw bytes)
|
||||
// Return chunk unchanged if no modification needed
|
||||
return chunk, nil
|
||||
}
|
||||
```
|
||||
|
||||
## Need Help?
|
||||
|
||||
- **Discord Community**: [Join our Discord](https://discord.gg/exN5KAydbU)
|
||||
- **GitHub Issues**: [Report bugs or request features](https://github.com/maximhq/bifrost/issues)
|
||||
- **Writing Plugins Guide**: [Full plugin documentation](./writing-plugin)
|
||||
|
||||
|
||||
190
docs/plugins/sequencing.mdx
Normal file
190
docs/plugins/sequencing.mdx
Normal file
@@ -0,0 +1,190 @@
|
||||
---
|
||||
title: "Plugin Sequencing"
|
||||
description: "Control the execution order of custom plugins relative to Bifrost's built-in plugins using placement groups and ordering."
|
||||
icon: "arrow-down-1-9"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
When you have multiple plugins — both built-in and custom — the order in which they execute matters. A logging plugin should capture the final request, an auth plugin should validate before anything else runs, and a response transformer should run after the provider returns data.
|
||||
|
||||
Plugin sequencing lets you control **where** your custom plugins execute relative to Bifrost's built-in plugins (telemetry, logging, governance, etc.) and **in what order** they execute relative to each other.
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
||||
Bifrost organizes plugins into three **placement groups** that execute in a fixed order:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A["Pre-builtin plugins"] --> B["Built-in plugins"]
|
||||
B --> C["Post-builtin plugins"]
|
||||
```
|
||||
|
||||
| Placement Group | Pre-hooks (request) | Post-hooks (response) |
|
||||
|-----------------|--------------------|-----------------------|
|
||||
| `pre_builtin` | Runs **first** | Runs **last** |
|
||||
| `builtin` | Runs **second** | Runs **second** |
|
||||
| `post_builtin` | Runs **third** | Runs **first** |
|
||||
|
||||
<Info>
|
||||
Post-hooks execute in **reverse order** of pre-hooks (LIFO pattern). This means a `pre_builtin` plugin's `PreLLMHook` runs first, but its `PostLLMHook` runs last — ensuring proper cleanup and state unwinding.
|
||||
</Info>
|
||||
|
||||
### Ordering within a group
|
||||
|
||||
Within each placement group, plugins are sorted by their `order` value (lower executes earlier). Plugins with the same order preserve their registration order.
|
||||
|
||||
**Example:** Three custom plugins configured as:
|
||||
|
||||
| Plugin | Placement | Order | Pre-hook runs | Post-hook runs |
|
||||
|--------|-----------|-------|---------------|----------------|
|
||||
| auth-validator | `pre_builtin` | 0 | 1st | 5th (last) |
|
||||
| request-enricher | `pre_builtin` | 1 | 2nd | 4th |
|
||||
| *Built-in plugins* | — | — | 3rd | 3rd |
|
||||
| response-logger | `post_builtin` | 0 | 4th | 2nd |
|
||||
| analytics | `post_builtin` | 1 | 5th (last) | 1st |
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
<Tabs group="config-method">
|
||||
<Tab title="Web UI">
|
||||
|
||||
1. Navigate to the **Plugins** page in the sidebar
|
||||
2. Click the **Edit Plugin Sequence** button (appears when you have at least one custom plugin installed)
|
||||
|
||||

|
||||
|
||||
3. **Drag** custom plugins above or below the **Built-in Plugins** block:
|
||||
- Plugins **above** the block get `pre_builtin` placement
|
||||
- Plugins **below** the block get `post_builtin` placement
|
||||
4. The order within each group is determined by position (top = lowest order value)
|
||||
5. Click **Save Sequence** to apply the changes
|
||||
|
||||
<Note>
|
||||
If your `config.json` file has plugin sequence configured, it will take precedence over the sequence configured in the UI after restarting Bifrost.
|
||||
</Note>
|
||||
|
||||
</Tab>
|
||||
<Tab title="API">
|
||||
|
||||
Update a plugin's placement and order using the update endpoint:
|
||||
|
||||
```bash
|
||||
curl -X PUT http://localhost:8080/api/plugins/my-plugin \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"enabled": true,
|
||||
"path": "/path/to/my-plugin.so",
|
||||
"placement": "pre_builtin",
|
||||
"order": 0
|
||||
}'
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"message": "Plugin updated successfully",
|
||||
"plugin": {
|
||||
"name": "my-plugin",
|
||||
"enabled": true,
|
||||
"isCustom": true,
|
||||
"path": "/path/to/my-plugin.so",
|
||||
"placement": "pre_builtin",
|
||||
"order": 0,
|
||||
"status": {
|
||||
"status": "active"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You can also set placement when creating a plugin:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/api/plugins \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"name": "my-plugin",
|
||||
"enabled": true,
|
||||
"path": "/path/to/my-plugin.so",
|
||||
"placement": "pre_builtin",
|
||||
"order": 0
|
||||
}'
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="config.json">
|
||||
|
||||
Set `placement` and `order` on each plugin in the `plugins` array:
|
||||
|
||||
```json
|
||||
{
|
||||
"plugins": [
|
||||
{
|
||||
"name": "auth-validator",
|
||||
"enabled": true,
|
||||
"path": "/plugins/auth-validator.so",
|
||||
"placement": "pre_builtin",
|
||||
"order": 0
|
||||
},
|
||||
{
|
||||
"name": "request-enricher",
|
||||
"enabled": true,
|
||||
"path": "/plugins/request-enricher.so",
|
||||
"placement": "pre_builtin",
|
||||
"order": 1
|
||||
},
|
||||
{
|
||||
"name": "response-logger",
|
||||
"enabled": true,
|
||||
"path": "/plugins/response-logger.so",
|
||||
"placement": "post_builtin",
|
||||
"order": 0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `placement` | string | No | `post_builtin` | `"pre_builtin"` or `"post_builtin"`. Controls whether the plugin runs before or after built-in plugins. |
|
||||
| `order` | integer | No | `0` | Position within the placement group. Lower values execute earlier. |
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
---
|
||||
|
||||
## When to use each placement
|
||||
|
||||
### `pre_builtin` — run before built-in plugins
|
||||
|
||||
Use this when your plugin needs to:
|
||||
|
||||
- **Validate or authenticate** requests before any built-in processing
|
||||
- **Enrich requests** with data that built-in plugins should see (e.g., injecting headers or metadata)
|
||||
- **Short-circuit** requests before they reach governance checks or telemetry
|
||||
|
||||
### `post_builtin` (default) — run after built-in plugins
|
||||
|
||||
Use this when your plugin needs to:
|
||||
|
||||
- **Transform responses** after all built-in processing is complete
|
||||
- **Log or analyze** the final request/response (after governance, telemetry, etc.)
|
||||
- **Add custom headers** or modify the response before it reaches the client
|
||||
|
||||
<Tip>
|
||||
When in doubt, use the default `post_builtin` placement. Most custom plugins — logging, analytics, response transformations — work best after built-in plugins have finished their processing.
|
||||
</Tip>
|
||||
|
||||
---
|
||||
|
||||
## Next steps
|
||||
|
||||
- **[Writing a Go plugin](./writing-go-plugin)** — Build your first custom plugin with `PreLLMHook` and `PostLLMHook`
|
||||
- **[Writing a WASM plugin](./writing-wasm-plugin)** — Build a portable WASM plugin
|
||||
- **[Plugin architecture](../architecture/core/plugins)** — Deep dive into the plugin lifecycle and hook execution model
|
||||
1183
docs/plugins/writing-go-plugin.mdx
Normal file
1183
docs/plugins/writing-go-plugin.mdx
Normal file
File diff suppressed because it is too large
Load Diff
1331
docs/plugins/writing-wasm-plugin.mdx
Normal file
1331
docs/plugins/writing-wasm-plugin.mdx
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user