first commit
This commit is contained in:
Beyhan Oğur
414
docs/mcp/tool-execution.mdx
Normal file
414
docs/mcp/tool-execution.mdx
Normal file
@@ -0,0 +1,414 @@
|
||||
---
|
||||
title: "Tool Execution"
|
||||
sidebarTitle: "Tool Execution"
|
||||
description: "Execute MCP tools with full control over approval and conversation flow."
|
||||
icon: "play"
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
When an LLM returns tool calls in its response, Bifrost does **not** automatically execute them. Instead, your application explicitly calls the tool execution API, giving you full control over:
|
||||
|
||||
- Which tool calls to execute
|
||||
- User approval workflows
|
||||
- Security validation
|
||||
- Audit logging
|
||||
|
||||
The basic flow is: **Chat Request → Review Tool Calls → Execute Tools → Continue Conversation**. For detailed architecture diagrams, see the [MCP Architecture](/architecture/core/mcp#tool-execution-engine) documentation.
|
||||
|
||||
---
|
||||
|
||||
## Authentication
|
||||
|
||||
The `/v1/mcp/tool/execute` endpoint uses the same authentication as other inference endpoints like `/v1/chat/completions`:
|
||||
|
||||
| Auth Configuration | Behavior |
|
||||
|--------------------|----------|
|
||||
| `disable_auth_on_inference: true` | No auth required |
|
||||
| `disable_auth_on_inference: false` | Auth required |
|
||||
|
||||
Virtual keys and authentication are independent layers that work together. For details on how to use virtual keys with authentication, see [Authentication and Virtual Keys](/features/governance/virtual-keys#authentication-and-virtual-keys).
|
||||
|
||||
---
|
||||
|
||||
## End-to-End Example
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Gateway">
|
||||
|
||||
### Step 1: Send Chat Request
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "List files in the current directory"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
**Response with tool calls:**
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-abc123",
|
||||
"choices": [{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": null,
|
||||
"tool_calls": [{
|
||||
"id": "call_xyz789",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "filesystem_list_directory",
|
||||
"arguments": "{\"path\": \".\"}"
|
||||
}
|
||||
}]
|
||||
},
|
||||
"finish_reason": "tool_calls"
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
Tool names are prefixed with the MCP client name (e.g., `filesystem_list_directory`). This ensures uniqueness across multiple MCP clients.
|
||||
</Note>
|
||||
|
||||
### Step 2: Execute the Tool
|
||||
|
||||
The request body matches the tool call object from the response:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/v1/mcp/tool/execute \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"id": "call_xyz789",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "filesystem_list_directory",
|
||||
"arguments": "{\"path\": \".\"}"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
**Tool result response:**
|
||||
```json
|
||||
{
|
||||
"role": "tool",
|
||||
"content": "[\"config.json\", \"main.go\", \"README.md\"]",
|
||||
"tool_call_id": "call_xyz789"
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Continue the Conversation
|
||||
|
||||
Assemble the full conversation history and continue:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-4o",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "List files in the current directory"
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": null,
|
||||
"tool_calls": [{
|
||||
"id": "call_xyz789",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "filesystem_list_directory",
|
||||
"arguments": "{\"path\": \".\"}"
|
||||
}
|
||||
}]
|
||||
},
|
||||
{
|
||||
"role": "tool",
|
||||
"content": "[\"config.json\", \"main.go\", \"README.md\"]",
|
||||
"tool_call_id": "call_xyz789"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
**Final response:**
|
||||
```json
|
||||
{
|
||||
"choices": [{
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": "The current directory contains 3 files:\n\n1. **config.json** - Configuration file\n2. **main.go** - Go source file\n3. **README.md** - Documentation"
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="Go SDK">
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
bifrost "github.com/maximhq/bifrost/core"
|
||||
"github.com/maximhq/bifrost/core/schemas"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// Initialize Bifrost with MCP (see Connecting to Servers)
|
||||
client, _ := bifrost.Init(context.Background(), config)
|
||||
|
||||
// Step 1: Send initial request
|
||||
firstMessage := schemas.ChatMessage{
|
||||
Role: schemas.ChatMessageRoleUser,
|
||||
Content: schemas.ChatMessageContent{
|
||||
ContentStr: bifrost.Ptr("List files in the current directory"),
|
||||
},
|
||||
}
|
||||
|
||||
request := &schemas.BifrostChatRequest{
|
||||
Provider: schemas.OpenAI,
|
||||
Model: "gpt-4o",
|
||||
Input: []schemas.ChatMessage{firstMessage},
|
||||
}
|
||||
|
||||
response, err := client.ChatCompletionRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), request)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Build conversation history
|
||||
history := []schemas.ChatMessage{firstMessage}
|
||||
|
||||
// Step 2: Process tool calls
|
||||
if response.Choices[0].Message.ToolCalls != nil {
|
||||
assistantMessage := response.Choices[0].Message
|
||||
history = append(history, assistantMessage)
|
||||
|
||||
for _, toolCall := range *assistantMessage.ToolCalls {
|
||||
fmt.Printf("Tool requested: %s\n", *toolCall.Function.Name)
|
||||
|
||||
// YOUR APPROVAL LOGIC HERE
|
||||
// - Validate arguments
|
||||
// - Check permissions
|
||||
// - Get user confirmation if needed
|
||||
|
||||
// Step 3: Execute the tool
|
||||
toolResult, err := client.ExecuteChatMCPTool(context.Background(), toolCall)
|
||||
if err != nil {
|
||||
fmt.Printf("Tool execution failed: %v\n", err)
|
||||
continue
|
||||
}
|
||||
|
||||
fmt.Printf("Tool result: %s\n", *toolResult.Content.ContentStr)
|
||||
history = append(history, *toolResult)
|
||||
}
|
||||
}
|
||||
|
||||
// Step 4: Continue conversation with results
|
||||
finalRequest := &schemas.BifrostChatRequest{
|
||||
Provider: schemas.OpenAI,
|
||||
Model: "gpt-4o",
|
||||
Input: history,
|
||||
}
|
||||
|
||||
finalResponse, err := client.ChatCompletionRequest(schemas.NewBifrostContext(context.Background(), schemas.NoDeadline), finalRequest)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
fmt.Printf("Final response: %s\n", *finalResponse.Choices[0].Message.Content.ContentStr)
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
---
|
||||
|
||||
## Response Formats
|
||||
|
||||
Bifrost supports two API formats for tool execution:
|
||||
|
||||
### Chat Format (Default)
|
||||
|
||||
Use `?format=chat` or omit the parameter:
|
||||
|
||||
```bash
|
||||
POST /v1/mcp/tool/execute?format=chat
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"id": "call_xyz789",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "filesystem_read_file",
|
||||
"arguments": "{\"path\": \"config.json\"}"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"role": "tool",
|
||||
"content": "{\"key\": \"value\"}",
|
||||
"tool_call_id": "call_xyz789"
|
||||
}
|
||||
```
|
||||
|
||||
### Responses Format
|
||||
|
||||
Use `?format=responses` for the Responses API format:
|
||||
|
||||
```bash
|
||||
POST /v1/mcp/tool/execute?format=responses
|
||||
```
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{
|
||||
"type": "function_call_output",
|
||||
"call_id": "call_xyz789",
|
||||
"name": "filesystem_read_file",
|
||||
"arguments": "{\"path\": \"config.json\"}"
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"type": "function_call_output",
|
||||
"call_id": "call_xyz789",
|
||||
"output": "{\"key\": \"value\"}"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Multiple Tool Calls
|
||||
|
||||
LLMs often request multiple tools in a single response. Execute them in sequence or parallel:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Sequential">
|
||||
|
||||
```go
|
||||
for _, toolCall := range *response.Choices[0].Message.ToolCalls {
|
||||
result, err := client.ExecuteChatMCPTool(ctx, toolCall)
|
||||
if err != nil {
|
||||
// Handle error
|
||||
continue
|
||||
}
|
||||
history = append(history, *result)
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="Parallel">
|
||||
|
||||
```go
|
||||
toolCalls := *response.Choices[0].Message.ToolCalls
|
||||
results := make([]*schemas.ChatMessage, len(toolCalls))
|
||||
var wg sync.WaitGroup
|
||||
|
||||
for i, toolCall := range toolCalls {
|
||||
wg.Add(1)
|
||||
go func(idx int, tc schemas.ChatAssistantMessageToolCall) {
|
||||
defer wg.Done()
|
||||
result, err := client.ExecuteChatMCPTool(ctx, tc)
|
||||
if err == nil {
|
||||
results[idx] = result
|
||||
}
|
||||
}(i, toolCall)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
for _, result := range results {
|
||||
if result != nil {
|
||||
history = append(history, *result)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
Tool execution can fail for various reasons:
|
||||
|
||||
```go
|
||||
result, err := client.ExecuteChatMCPTool(ctx, toolCall)
|
||||
if err != nil {
|
||||
switch {
|
||||
case errors.Is(err, context.DeadlineExceeded):
|
||||
// Tool execution timed out
|
||||
case strings.Contains(err.Error(), "tool not found"):
|
||||
// Tool doesn't exist or client disconnected
|
||||
case strings.Contains(err.Error(), "not allowed"):
|
||||
// Tool filtered out by configuration
|
||||
default:
|
||||
// Other execution error
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Gateway error responses:**
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"type": "tool_execution_error",
|
||||
"message": "Tool 'filesystem_delete_file' is not allowed for this request"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Copy-Pastable Responses
|
||||
|
||||
Tool execution responses are designed to be directly appended to your conversation history:
|
||||
|
||||
```go
|
||||
// Tool result is already in the correct format
|
||||
toolResult, _ := client.ExecuteChatMCPTool(ctx, toolCall)
|
||||
|
||||
// Just append it directly
|
||||
history = append(history, *toolResult)
|
||||
```
|
||||
|
||||
The response includes:
|
||||
- Correct `role` field (`"tool"`)
|
||||
- Matching `tool_call_id` for correlation
|
||||
- Properly formatted `content`
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Agent Mode" icon="robot" href="./agent-mode">
|
||||
Enable autonomous tool execution with auto-approval
|
||||
</Card>
|
||||
<Card title="Tool Filtering" icon="filter" href="./filtering">
|
||||
Control which tools are available per request
|
||||
</Card>
|
||||
</CardGroup>
|
||||
Reference in New Issue
Block a user