beyhan/bifrost
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
Beyhan Oğur
2026-04-26 21:52:23 +03:00
commit 880f412e2c
2662 changed files with 866266 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

242
docs/features/prompt-repository/playground.mdx Normal file
View File

@@ -0,0 +1,242 @@
---
title: "Playground"
description: "Create, test, and version prompts in an interactive playground."
icon: "square-terminal"
---
## Overview
The **Playground** in Bifrost is an interactive workspace for building, testing, and managing prompts. It allows you to experiment with messages, switch models, adjust parameters, and iterate until the output looks right. Once you're satisfied, you can **publish a version** and use it directly in your codebase. Over time, the prompt repository becomes a centralized **CMS for all your prompts**, making it easier to manage versions, collaborate with teammates, and maintain production-ready prompts.
![Prompt Repository Overview](../../media/prompt-repo-overview.png)
## How it Works
The playground is built around four core concepts: **Prompts, Sessions, and Versions**.
### Folders
Folders help organize prompts into logical groups. Teams often structure them by product area, feature, or use case.
- Each folder has a **name** and optional **description**
- Prompts can live inside folders or at the root level
- Deleting a folder removes **all prompts, sessions, and versions inside it**
### Prompts
A **Prompt** is the main unit in the repository.
Think of it as a container that holds the full lifecycle of a prompt, from early experiments to production-ready versions.
Each prompt can have:
- Multiple **sessions** for experimentation
- Multiple **versions** for stable releases
### Sessions (Working Copies)
Sessions are **editable working copies** where you experiment with a prompt.
You can freely:
- Modify messages
- Switch providers or models
- Adjust parameters
- Run the prompt repeatedly
Sessions don't affect committed versions, so you can iterate safely.
If your session has unsaved changes, a **red asterisk appears next to the prompt name** in the top bar.
You can save your progress using:
- **Save Session** button
- `Cmd + S` / `Ctrl + S`
Saved sessions can be **renamed and restored** from the dropdown next to the Save button.
### Versions (Immutable Snapshots)
When you're happy with a prompt, you can **commit it as a version**.
Versions are **immutable snapshots**; once created, they cannot be edited. When the config differs from the last saved version, the **Unpublished Changes** badge appears, and it can be committed to create a new version.
Each version stores:
- The selected **message history** (system, user, assistant)
- **Provider and model configuration**
- **Model parameters** (temperature, max tokens, etc.)
- A **commit message** describing the change
Versions are automatically numbered:
```
v1 → v2 → v3 → ...
```
You can also **restore a previous version** from the dropdown next to the **Commit Version** button.
---
## Workspace Layout
The playground uses a simple **three-panel layout**:
| Panel | Purpose |
|------|---------|
| **Sidebar (left)** | Browse prompts, manage folders, and organize items |
| **Playground (center)** | Build and test your prompt messages |
| **Settings (right)** | Configure provider, model, API key, variables, parameters, and deployments |
The settings panel is organized into collapsible sections:
- **Configuration** — Provider, model, API key, variables, and model parameters
- **Deployments** — Prompt deployment strategies and traffic routing (enterprise)
![Workspace Layout](../../media/prompt-repo-layout.png)
---
## Getting Started
<Steps>
<Step title="Create a folder (optional)">
Click the **"+"** button in the sidebar and select **New Folder**.
Folders help organize prompts by team, feature, or use case.
![Create Folder](../../media/prompt-repo-create-folder.png)
</Step>
<Step title="Create a prompt">
Click **"+"** again and choose **New Prompt**.
Give it a name and optionally assign it to a folder.
![Create Prompt](../../media/prompt-repo-create-prompt.png)
</Step>
<Step title="Build your prompt">
Add messages to your prompt in the Playground:
- **System messages** for instructions
- **User messages** for input
- **Assistant messages** for examples or few-shot responses
Configure the provider, model, and parameters from the settings panel on the right.
![Playground](../../media/prompt-repo-playground.png)
</Step>
<Step title="Run the prompt">
Click **Run** or press `Cmd + S` / `Ctrl + S`.
Optionally, if you do not want to execute the prompt and only want to add a message to history, use the **+ Add** button.
</Step>
<Step title="Save and publish a version">
Once you're satisfied with the results:
1. **Save Session** to preserve your work
2. **Commit Version** to create an immutable snapshot
![Commit Version](../../media/prompt-repo-commit.png)
</Step>
</Steps>
## Key Capabilities
### Version Control
Each committed version creates a permanent record of your prompt.
This allows teams to track changes and safely iterate without breaking production prompts.
Key characteristics:
- **Sequential versioning** — v1, v2, v3, ...
- **Commit messages** explaining what changed
- **Immutable history**
### Multi-Provider Testing
You can switch between providers and models directly in the Playground.
Supported providers may include:
- OpenAI
- Anthropic
- AWS Bedrock
- Others configured in your Bifrost instance
You can also choose which API key to use:
- **Auto**: Uses the first available key.
- **Specific key**: Select a particular key.
- **Virtual key**: Uses governance-managed keys.
This makes it easy to compare how different models respond to the same prompt.
### Message Types
The Playground supports several message roles:
- **System**: Defines behavior or instructions.
- **User**: Input to the model.
- **Assistant**: The model's response to the user's input.
- **Tool Calls**: Function calls made by the model.
- **Tool Results**: Mock or real responses from called tools.
These allow you to simulate complex conversations and agent workflows.
### Attachments
For models that support multimodal input, you can attach files directly to user messages.
Supported attachments may include:
- Images
- PDFs
- Other supported file types
Attachments are only enabled when the selected model supports them.
### Drag-and-Drop Organization
Prompts can be reorganized easily using drag and drop in the sidebar.
You can move prompts:
- Between folders
- Back to the root level
## Session Management
Sessions store the state of your prompt experiments.
Each prompt maintains its **own session history**, allowing you to explore different approaches without losing previous work.
With sessions you can:
- Save specific conversation states
- Rename sessions for clarity
- Switch between past experiments
![Sessions](../../media/prompt-repo-sessions.png)
---
## Using prompts in production
To attach committed versions to **Chat Completions** or **Responses** requests through the gateway (HTTP headers, merging, and caching behavior), see the [Prompts plugin](/features/prompt-repository/prompts-plugin).

148
docs/features/prompt-repository/prompts-plugin.mdx Normal file
View File

@@ -0,0 +1,148 @@
---
title: "Prompts plugin"
description: "Use committed prompt templates from the Prompt Repository on inference requests via HTTP headers or custom resolvers."
icon: "puzzle-piece"
---
## Overview
The **Prompts** plugin connects the [Prompt Repository](/features/prompt-repository/playground) to inference. It loads committed prompt versions from the config store and **prepends** their messages to **Chat Completions** and **Responses** requests. It also **merges model parameters** from the stored version with the incoming request (request values take precedence).
**What it does:**
- Resolves which prompt and version to apply per request (default: HTTP headers).
- Injects the version’s message history **before** the client’s messages.
- Applies the version’s `model` parameters as defaults, then overrides with whatever the client sent for the same parameters.
---
## Prerequisites
- **Config store** with Prompt Repository tables (typically **PostgreSQL**). File-backed config alone does not store prompts.
- Prompts authored and **committed as versions** in the UI or via the `/api/prompt-repo/...` HTTP API (see `docs/openapi/openapi.yaml` in the repository).
- A **prompt ID** (UUID) for each prompt you reference at runtime. You can read it from the repository API or the playground.
---
## How it works
```mermaid
flowchart TB
Client([Client]) --> Gateway[Bifrost HTTP]
Gateway --> PreHook["HTTP transport pre-hook:<br/>copy x-bf-prompt-id / x-bf-prompt-version to context"]
PreHook --> PreLLM["PreLLM hook:<br/>resolve version, merge params,<br/>prepend template messages"]
PreLLM --> Provider[Provider]
```
1. **Transport (HTTP):** Incoming headers `x-bf-prompt-id` and `x-bf-prompt-version` are copied onto the Bifrost context (header name matching is case-insensitive).
2. **Resolve:** The plugin looks up the prompt and the requested version. If **`x-bf-prompt-version` is omitted**, the prompt’s **latest committed version** is used.
3. **Parameters:** Version `model` parameters are merged into the request; any field already set on the request wins.
4. **Messages:** Messages from the committed version are **prepended** to `messages` (chat) or `input` (responses). Your request body adds the user turn(s) after the template.
If the prompt ID is missing, the plugin does nothing and the request passes through unchanged.
---
## HTTP headers (gateway)
| Header | Required | Description |
|--------|----------|-------------|
| `x-bf-prompt-id` | Yes, to enable injection | UUID of the prompt in the repository. |
| `x-bf-prompt-version` | No | **Integer version number** (e.g. `3` for v3). If omitted, the **latest** committed version for that prompt is used. |
Invalid or unknown IDs / versions are logged as warnings; the request is **not** failed by the plugin (it proceeds without template injection).
---
## Example: Chat Completions
Use the same JSON body as a normal chat request. Only the headers select the template.
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-H "x-bf-prompt-id: YOUR-PROMPT-UUID" \
-H "x-bf-vk: sk-bf-your-virtual-key" \
-d '{
"model": "openai/gpt-5.4",
"messages": [
{
"role": "user",
"content": "Tell me about Bifrost Gateway?"
}
]
}'
```
![Commit Version with Stream enabled in the playground](../../media/prompt-plugin-version-commit.png)
When you commit a version from the playground, the model parameters (temperature, max tokens, etc.) are saved with it. These parameters are merged into the outgoing request, with client-supplied values taking precedence.
![LLM log for the same request showing Type: Chat Stream](../../media/prompt-plugin-llm-log.png)
In **Logs**, that run shows the full conversation: the committed **system** template, your **user** message from the request body, and the assistant reply. The log also displays the **Selected Prompt** name and version number for easy traceability.
The provider receives the merged model parameters from both the prompt version and the client request, with the messages from the committed version prepended before the client’s messages.
---
## Example: Responses API
```bash
curl -X POST http://localhost:8080/v1/responses \
-H "Content-Type: application/json" \
-H "x-bf-prompt-id: YOUR-PROMPT-UUID" \
-H "x-bf-prompt-version: 4" \
-H "x-bf-vk: sk-bf-your-virtual-key" \
-d '{
"model": "openai/gpt-5-nano-2025-08-07",
"input": "What is Pale Blue Dot?"
}'
```
---
## Streaming
Streaming is controlled entirely by the client request. If you want streaming, set `"stream": true` in the request body. The plugin merges model parameters from the committed version (request values take precedence), but does **not** override the transport-level streaming mode.
---
## Cache and updates
The plugin keeps an in-memory cache of prompts and versions (loaded with a small number of store queries at startup). When you create, update, or delete prompts or versions through the **gateway APIs**, the server **reloads** that cache so new commits are visible without a full process restart.
---
## Go SDK and custom resolution
For embedded Bifrost (Go SDK), register the plugin with `prompts.Init` and a **config store** that implements the prompt tables API. The default resolver reads the same logical keys from `BifrostContext`:
- `prompts.PromptIDKey` (`x-bf-prompt-id`)
- `prompts.PromptVersionKey` (`x-bf-prompt-version`)
Set them on the context you pass to `ChatCompletion` / `Responses` if you are not going through the HTTP transport hooks.
For advanced routing (for example, choosing a prompt from governance metadata), implement `prompts.PromptResolver` and use **`prompts.InitWithResolver`**. The interface is:
```go
type PromptResolver interface {
Resolve(ctx *schemas.BifrostContext, req *schemas.BifrostRequest) (promptID string, versionNumber int, err error)
}
```
Return an empty `promptID` to skip injection for a request. Return `versionNumber == 0` to use the prompt's **latest** committed version; any positive integer selects that specific version.
After injection, the plugin sets the following context keys (read by the logging plugin to populate log fields):
- `schemas.BifrostContextKeySelectedPromptID` — UUID of the applied prompt
- `schemas.BifrostContextKeySelectedPromptName` — Display name of the prompt
- `schemas.BifrostContextKeySelectedPromptVersion` — Version number as a string (e.g. `"3"`)
---
## Related
- [Playground](/features/prompt-repository/playground) — create folders, prompts, sessions, and committed versions.
- [Writing Go plugins](/plugins/writing-go-plugin) — plugin interfaces and lifecycle.
- Built-in plugin name in code: `prompts` (`github.com/maximhq/bifrost/plugins/prompts`).