567 lines
15 KiB
Plaintext
567 lines
15 KiB
Plaintext
---
|
|
title: "Mocker"
|
|
description: "Mock AI provider responses for testing, development, and simulation purposes."
|
|
icon: "mask"
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
### Minimal Configuration
|
|
|
|
The simplest way to use the Mocker plugin is with no configuration - it will create a default catch-all rule:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
bifrost "github.com/maximhq/bifrost/core"
|
|
"github.com/maximhq/bifrost/core/schemas"
|
|
mocker "github.com/maximhq/bifrost/plugins/mocker"
|
|
)
|
|
|
|
func main() {
|
|
// Create plugin with minimal config
|
|
plugin, err := mocker.NewMockerPlugin(mocker.MockerConfig{
|
|
Enabled: true, // Default rule will be created automatically
|
|
})
|
|
if err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
// Initialize Bifrost with the plugin
|
|
client, initErr := bifrost.Init(context.Background(), schemas.BifrostConfig{
|
|
Account: &yourAccount,
|
|
LLMPlugins: []schemas.LLMPlugin{plugin},
|
|
})
|
|
if err != nil {
|
|
panic(err)
|
|
}
|
|
defer client.Shutdown()
|
|
|
|
// All chat and responses requests will now return: "This is a mock response from the Mocker plugin"
|
|
|
|
// Chat completion request
|
|
chatResponse, _ := client.ChatCompletionRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), &schemas.BifrostChatRequest{
|
|
Provider: schemas.OpenAI,
|
|
Model: "gpt-4",
|
|
Input: []schemas.ChatMessage{
|
|
{
|
|
Role: schemas.ChatMessageRoleUser,
|
|
Content: schemas.ChatMessageContent{
|
|
ContentStr: bifrost.Ptr("Hello!"),
|
|
},
|
|
},
|
|
},
|
|
})
|
|
|
|
// Responses request
|
|
responsesResponse, _ := client.ResponsesRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), &schemas.BifrostResponsesRequest{
|
|
Provider: schemas.OpenAI,
|
|
Model: "gpt-4o",
|
|
Input: []schemas.ResponsesMessage{
|
|
{
|
|
Role: bifrost.Ptr(schemas.ResponsesInputMessageRoleUser),
|
|
Content: &schemas.ResponsesMessageContent{
|
|
ContentStr: bifrost.Ptr("Hello!"),
|
|
},
|
|
},
|
|
},
|
|
})
|
|
}
|
|
```
|
|
|
|
### Custom Response
|
|
|
|
```go
|
|
plugin, err := mocker.NewMockerPlugin(mocker.MockerConfig{
|
|
Enabled: true,
|
|
Rules: []mocker.MockRule{
|
|
{
|
|
Name: "openai-mock",
|
|
Enabled: true,
|
|
Probability: 1.0, // Always trigger
|
|
Conditions: mocker.Conditions{
|
|
Providers: []string{"openai"},
|
|
},
|
|
Responses: []mocker.Response{
|
|
{
|
|
Type: mocker.ResponseTypeSuccess,
|
|
Content: &mocker.SuccessResponse{
|
|
Message: "Hello! This is a custom mock response for OpenAI.",
|
|
Usage: &mocker.Usage{
|
|
PromptTokens: 15,
|
|
CompletionTokens: 25,
|
|
TotalTokens: 40,
|
|
},
|
|
},
|
|
},
|
|
},
|
|
},
|
|
},
|
|
})
|
|
```
|
|
|
|
### Responses Request Example
|
|
|
|
The mocker plugin automatically handles both chat completion and responses requests with the same configuration:
|
|
|
|
```go
|
|
// This rule will work for both ChatCompletionRequest and ResponsesRequest
|
|
{
|
|
Name: "universal-mock",
|
|
Enabled: true,
|
|
Probability: 1.0,
|
|
Conditions: mocker.Conditions{
|
|
MessageRegex: stringPtr("(?i).*hello.*"),
|
|
},
|
|
Responses: []mocker.Response{
|
|
{
|
|
Type: mocker.ResponseTypeSuccess,
|
|
Content: &mocker.SuccessResponse{
|
|
Message: "Hello! I'm a mock response that works for both request types.",
|
|
},
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
## Installation
|
|
|
|
Add the plugin to your project:
|
|
|
|
```bash
|
|
go get github.com/maximhq/bifrost/plugins/mocker
|
|
```
|
|
|
|
Import in your code:
|
|
|
|
```go
|
|
import mocker "github.com/maximhq/bifrost/plugins/mocker"
|
|
```
|
|
|
|
## Basic Usage
|
|
|
|
### Creating the Plugin
|
|
|
|
```go
|
|
config := mocker.MockerConfig{
|
|
Enabled: true,
|
|
DefaultBehavior: mocker.DefaultBehaviorPassthrough, // "passthrough", "success", "error"
|
|
Rules: []mocker.MockRule{
|
|
// Your rules here
|
|
},
|
|
}
|
|
|
|
plugin, err := mocker.NewMockerPlugin(config)
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
```
|
|
|
|
### Adding to Bifrost
|
|
|
|
```go
|
|
client, initErr := bifrost.Init(context.Background(), schemas.BifrostConfig{
|
|
Account: &yourAccount,
|
|
LLMPlugins: []schemas.LLMPlugin{plugin},
|
|
Logger: bifrost.NewDefaultLogger(schemas.LogLevelInfo),
|
|
})
|
|
```
|
|
|
|
### Disabling the Plugin
|
|
|
|
```go
|
|
config := mocker.MockerConfig{
|
|
Enabled: false, // All requests pass through to real providers
|
|
}
|
|
```
|
|
|
|
## Supported Request Types
|
|
|
|
The Mocker plugin supports the following Bifrost request types:
|
|
|
|
- **Chat Completion Requests** (`ChatCompletionRequest`) - Standard chat-based interactions
|
|
- **Responses Requests** (`ResponsesRequest`) - OpenAI-compatible responses API format
|
|
- **Skip Context Key** - Use `"skip-mocker"` context key to bypass mocking per request
|
|
|
|
### Skip Mocker for Specific Requests
|
|
|
|
You can skip the mocker plugin for specific requests by adding a context key:
|
|
|
|
```go
|
|
import "github.com/maximhq/bifrost/core/schemas"
|
|
|
|
// Create context that skips mocker
|
|
ctx := context.WithValue(context.Background(),
|
|
schemas.BifrostContextKey("skip-mocker"), true)
|
|
|
|
// This request will bypass the mocker and go to the real provider
|
|
response, err := client.ChatCompletionRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), request)
|
|
```
|
|
|
|
## Key Features
|
|
|
|
### Template Variables
|
|
|
|
Create dynamic responses using templates:
|
|
|
|
```go
|
|
Response{
|
|
Type: mocker.ResponseTypeSuccess,
|
|
Content: &mocker.SuccessResponse{
|
|
MessageTemplate: stringPtr("Hello from {{provider}} using model {{model}}!"),
|
|
},
|
|
}
|
|
```
|
|
|
|
**Available Variables:**
|
|
- `{{provider}}` - Provider name (e.g., "openai", "anthropic")
|
|
- `{{model}}` - Model name (e.g., "gpt-4", "claude-3")
|
|
- `{{faker.*}}` - Fake data generation (see Configuration Reference)
|
|
|
|
### Weighted Response Selection
|
|
|
|
Configure multiple responses with different probabilities:
|
|
|
|
```go
|
|
Responses: []mocker.Response{
|
|
{
|
|
Type: mocker.ResponseTypeSuccess,
|
|
Weight: 0.8, // 80% chance
|
|
Content: &mocker.SuccessResponse{
|
|
Message: "Success response",
|
|
},
|
|
},
|
|
{
|
|
Type: mocker.ResponseTypeError,
|
|
Weight: 0.2, // 20% chance
|
|
Error: &mocker.ErrorResponse{
|
|
Message: "Rate limit exceeded",
|
|
Type: stringPtr("rate_limit"),
|
|
Code: stringPtr("429"),
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
### Latency Simulation
|
|
|
|
Add realistic delays to responses:
|
|
|
|
```go
|
|
// Fixed latency
|
|
Latency: &mocker.Latency{
|
|
Type: mocker.LatencyTypeFixed,
|
|
Min: 250 * time.Millisecond,
|
|
}
|
|
|
|
// Variable latency
|
|
Latency: &mocker.Latency{
|
|
Type: mocker.LatencyTypeUniform,
|
|
Min: 100 * time.Millisecond,
|
|
Max: 500 * time.Millisecond,
|
|
}
|
|
```
|
|
|
|
### Advanced Matching
|
|
|
|
#### Regex Message Matching
|
|
```go
|
|
Conditions: mocker.Conditions{
|
|
MessageRegex: stringPtr(`(?i).*support.*|.*help.*`),
|
|
}
|
|
```
|
|
|
|
#### Request Size Filtering
|
|
```go
|
|
Conditions: mocker.Conditions{
|
|
RequestSize: &mocker.SizeRange{
|
|
Min: 100, // bytes
|
|
Max: 1000, // bytes
|
|
},
|
|
}
|
|
```
|
|
|
|
### Faker Data Generation
|
|
|
|
Create realistic test data using faker variables:
|
|
|
|
```go
|
|
{
|
|
Name: "user-profile-example",
|
|
Responses: []mocker.Response{
|
|
{
|
|
Type: mocker.ResponseTypeSuccess,
|
|
Content: &mocker.SuccessResponse{
|
|
MessageTemplate: stringPtr(`User Profile:
|
|
- Name: {{faker.name}}
|
|
- Email: {{faker.email}}
|
|
- Company: {{faker.company}}
|
|
- Address: {{faker.address}}, {{faker.city}}
|
|
- Phone: {{faker.phone}}
|
|
- User ID: {{faker.uuid}}
|
|
- Join Date: {{faker.date}}
|
|
- Premium Account: {{faker.boolean}}`),
|
|
},
|
|
},
|
|
},
|
|
}
|
|
```
|
|
|
|
### Statistics and Monitoring
|
|
|
|
Get runtime statistics for monitoring:
|
|
|
|
```go
|
|
stats := plugin.GetStatistics()
|
|
fmt.Printf("Plugin enabled: %v\n", stats.Enabled)
|
|
fmt.Printf("Total requests: %d\n", stats.TotalRequests)
|
|
fmt.Printf("Mocked requests: %d\n", stats.MockedRequests)
|
|
|
|
// Rule-specific stats
|
|
for ruleName, ruleStats := range stats.Rules {
|
|
fmt.Printf("Rule %s: %d triggers\n", ruleName, ruleStats.Triggers)
|
|
}
|
|
```
|
|
|
|
## Configuration Reference
|
|
|
|
### MockerConfig
|
|
|
|
| Field | Type | Default | Description |
|
|
|-------|------|---------|-------------|
|
|
| `Enabled` | `bool` | `false` | Enable/disable the entire plugin |
|
|
| `DefaultBehavior` | `string` | `"passthrough"` | Action when no rules match: `"passthrough"`, `"success"`, `"error"` |
|
|
| `GlobalLatency` | `*Latency` | `nil` | Global latency applied to all rules |
|
|
| `Rules` | `[]MockRule` | `[]` | List of mock rules evaluated in priority order |
|
|
|
|
### MockRule
|
|
|
|
| Field | Type | Default | Description |
|
|
|-------|------|---------|-------------|
|
|
| `Name` | `string` | - | Unique rule name for identification |
|
|
| `Enabled` | `bool` | `true` | Enable/disable this specific rule |
|
|
| `Priority` | `int` | `0` | Higher numbers = higher priority |
|
|
| `Probability` | `float64` | `1.0` | Activation probability (0.0=never, 1.0=always) |
|
|
| `Conditions` | `Conditions` | `{}` | Matching conditions (empty = match all) |
|
|
| `Responses` | `[]Response` | - | Possible responses (weighted random selection) |
|
|
| `Latency` | `*Latency` | `nil` | Rule-specific latency override |
|
|
|
|
### Conditions
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `Providers` | `[]string` | Match specific providers: `["openai", "anthropic"]` |
|
|
| `Models` | `[]string` | Match specific models: `["gpt-4", "claude-3"]` |
|
|
| `MessageRegex` | `*string` | Regex pattern to match message content |
|
|
| `RequestSize` | `*SizeRange` | Request size constraints in bytes |
|
|
|
|
### Response
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `Type` | `string` | Response type: `"success"` or `"error"` |
|
|
| `Weight` | `float64` | Weight for random selection (default: 1.0) |
|
|
| `Content` | `*SuccessResponse` | Required if `Type="success"` |
|
|
| `Error` | `*ErrorResponse` | Required if `Type="error"` |
|
|
| `AllowFallbacks` | `*bool` | Control fallback behavior (`nil`=allow, `false`=block) |
|
|
|
|
### SuccessResponse
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `Message` | `string` | Static response message |
|
|
| `MessageTemplate` | `*string` | Template with variables: `{{provider}}`, `{{model}}`, `{{faker.*}}` |
|
|
| `Model` | `*string` | Override model name in response |
|
|
| `Usage` | `*Usage` | Token usage information |
|
|
| `FinishReason` | `*string` | Completion reason (default: `"stop"`) |
|
|
| `CustomFields` | `map[string]interface{}` | Additional metadata fields |
|
|
|
|
### ErrorResponse
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `Message` | `string` | Error message to return |
|
|
| `Type` | `*string` | Error type (e.g., `"rate_limit"`, `"auth_error"`) |
|
|
| `Code` | `*string` | Error code (e.g., `"429"`, `"401"`) |
|
|
| `StatusCode` | `*int` | HTTP status code |
|
|
|
|
### Latency
|
|
|
|
| Field | Type | Description |
|
|
|-------|------|-------------|
|
|
| `Type` | `string` | Latency type: `"fixed"` or `"uniform"` |
|
|
| `Min` | `time.Duration` | Minimum/exact latency (use `time.Millisecond`) |
|
|
| `Max` | `time.Duration` | Maximum latency (required for `"uniform"`) |
|
|
|
|
**Important**: Use Go's `time.Duration` constants:
|
|
- ✅ Correct: `100 * time.Millisecond`
|
|
- ❌ Wrong: `100` (nanoseconds, barely noticeable)
|
|
|
|
### Faker Variables
|
|
|
|
#### Personal Information
|
|
- `{{faker.name}}` - Full name
|
|
- `{{faker.first_name}}` - First name only
|
|
- `{{faker.last_name}}` - Last name only
|
|
- `{{faker.email}}` - Email address
|
|
- `{{faker.phone}}` - Phone number
|
|
|
|
#### Location
|
|
- `{{faker.address}}` - Street address
|
|
- `{{faker.city}}` - City name
|
|
- `{{faker.state}}` - State/province
|
|
- `{{faker.zip_code}}` - Postal code
|
|
|
|
#### Business
|
|
- `{{faker.company}}` - Company name
|
|
- `{{faker.job_title}}` - Job title
|
|
|
|
#### Text and Data
|
|
- `{{faker.lorem_ipsum}}` - Lorem ipsum text
|
|
- `{{faker.lorem_ipsum:10}}` - Lorem ipsum with 10 words
|
|
- `{{faker.uuid}}` - UUID v4
|
|
- `{{faker.hex_color}}` - Hex color code
|
|
|
|
#### Numbers and Dates
|
|
- `{{faker.integer}}` - Random integer (1-100)
|
|
- `{{faker.integer:10,50}}` - Random integer between 10-50
|
|
- `{{faker.float}}` - Random float (0-100, 2 decimals)
|
|
- `{{faker.float:1,10}}` - Random float between 1-10
|
|
- `{{faker.boolean}}` - Random boolean
|
|
- `{{faker.date}}` - Date (YYYY-MM-DD format)
|
|
- `{{faker.datetime}}` - Datetime (YYYY-MM-DD HH:MM:SS format)
|
|
|
|
## Best Practices
|
|
|
|
### Rule Organization
|
|
|
|
```go
|
|
// Use priority to control rule evaluation order
|
|
rules := []mocker.MockRule{
|
|
{Name: "specific-error", Priority: 100, Conditions: /* specific */},
|
|
{Name: "general-success", Priority: 50, Conditions: /* general */},
|
|
{Name: "catch-all", Priority: 0, Conditions: /* empty */},
|
|
}
|
|
```
|
|
|
|
### Development vs Production
|
|
|
|
```go
|
|
// Development: High mock rate
|
|
config := mocker.MockerConfig{
|
|
Enabled: true,
|
|
Rules: []mocker.MockRule{
|
|
{Probability: 1.0}, // Always mock
|
|
},
|
|
}
|
|
|
|
// Production: Occasional testing
|
|
config := mocker.MockerConfig{
|
|
Enabled: true,
|
|
Rules: []mocker.MockRule{
|
|
{Probability: 0.1}, // 10% mock rate
|
|
},
|
|
}
|
|
```
|
|
|
|
### Performance Considerations
|
|
|
|
- Place specific conditions before general ones (higher priority)
|
|
- Use simple string matching over complex regex when possible
|
|
- Keep response templates reasonably sized
|
|
- Consider disabling debug logging in production
|
|
|
|
### Testing Your Configuration
|
|
|
|
```go
|
|
func validateMockerConfig(config mocker.MockerConfig) error {
|
|
_, err := mocker.NewMockerPlugin(config)
|
|
return err
|
|
}
|
|
|
|
// Test before deployment
|
|
if err := validateMockerConfig(yourConfig); err != nil {
|
|
log.Fatalf("Invalid mocker configuration: %v", err)
|
|
}
|
|
```
|
|
|
|
## Common Issues
|
|
|
|
### Plugin Not Triggering
|
|
|
|
1. Check if plugin is enabled: `Enabled: true`
|
|
2. Verify rule is enabled: `rule.Enabled: true`
|
|
3. Check probability: `Probability: 1.0` for testing
|
|
4. Verify conditions match your request
|
|
|
|
### Latency Not Working
|
|
|
|
Use `time.Duration` constants, not raw integers:
|
|
|
|
```go
|
|
// ❌ Wrong: 100 nanoseconds (barely noticeable)
|
|
Min: 100
|
|
|
|
// ✅ Correct: 100 milliseconds
|
|
Min: 100 * time.Millisecond
|
|
```
|
|
|
|
### Regex Not Matching
|
|
|
|
Test your regex pattern and ensure proper escaping:
|
|
|
|
```go
|
|
// Case-insensitive matching
|
|
MessageRegex: stringPtr(`(?i).*help.*`)
|
|
|
|
// Escape special characters
|
|
MessageRegex: stringPtr(`\$\d+\.\d+`) // Match $12.34
|
|
```
|
|
|
|
### Controlling Fallbacks
|
|
|
|
```go
|
|
Response{
|
|
Type: mocker.ResponseTypeError,
|
|
AllowFallbacks: boolPtr(false), // Block fallbacks
|
|
Error: &mocker.ErrorResponse{
|
|
Message: "Authentication failed",
|
|
},
|
|
}
|
|
```
|
|
|
|
### Skip Mocker Not Working
|
|
|
|
Ensure you're using the correct context key format:
|
|
|
|
```go
|
|
// ✅ Correct
|
|
ctx := context.WithValue(context.Background(),
|
|
schemas.BifrostContextKey("skip-mocker"), true)
|
|
|
|
// ❌ Wrong
|
|
ctx := context.WithValue(context.Background(), "skip-mocker", true)
|
|
```
|
|
|
|
### Responses Request Issues
|
|
|
|
If responses requests aren't being mocked:
|
|
|
|
1. Verify the plugin supports `ResponsesRequest` (version 1.2.13+)
|
|
2. Check that your regex patterns match the message content
|
|
3. Ensure the request type is `schemas.ResponsesRequest`
|
|
|
|
### Debug Mode
|
|
|
|
Enable debug logging to troubleshoot:
|
|
|
|
```go
|
|
client, initErr := bifrost.Init(context.Background(), schemas.BifrostConfig{
|
|
Account: &account,
|
|
LLMPlugins: []schemas.LLMPlugin{plugin},
|
|
Logger: bifrost.NewDefaultLogger(schemas.LogLevelDebug),
|
|
})
|
|
```
|