bifrost/docs/architecture/core/mcp.mdx

---
title: "Model Context Protocol (MCP)"
description: "Deep dive into Bifrost's Model Context Protocol (MCP) integration - how external tool discovery, execution, and integration work internally."
icon: "toolbox"
---

## MCP Architecture Overview

### **What is MCP in Bifrost?**

The Model Context Protocol (MCP) system in Bifrost enables AI models to seamlessly discover and execute external tools, transforming static chat models into dynamic, action-capable agents. This architecture bridges the gap between AI reasoning and real-world tool execution.

**Core MCP Principles:**

- **Dynamic Discovery** - Tools are discovered at runtime, not hardcoded
- **Client-Side Execution** - Bifrost controls all tool execution for security
- **Multi-Protocol Support** - STDIO, HTTP, and SSE connection types
- **Request-Level Filtering** - Granular control over tool availability
- **Async Execution** - Non-blocking tool invocation and response handling

### **MCP System Components**

```mermaid
graph TB
    subgraph "MCP Management Layer"
        MCPMgr[MCP Manager<br/>Central Controller]
        ClientRegistry[Client Registry<br/>Connection Management]
        ToolDiscovery[Tool Discovery<br/>Runtime Registration]
    end

    subgraph "MCP Execution Layer"
        ToolFilter[Tool Filter<br/>Access Control]
        ToolExecutor[Tool Executor<br/>Invocation Engine]
        ResultProcessor[Result Processor<br/>Response Handling]
    end

    subgraph "Connection Types"
        STDIOConn[STDIO Connections<br/>Command-line Tools]
        HTTPConn[HTTP Connections<br/>Web Services]
        SSEConn[SSE Connections<br/>Real-time Streams]
    end

    subgraph "External MCP Servers"
        FileSystem[Filesystem Tools<br/>File Operations]
        WebSearch[Web Search<br/>Information Retrieval]
        Database[Database Tools<br/>Data Access]
        Custom[Custom Tools<br/>Business Logic]
    end

    MCPMgr --> ClientRegistry
    ClientRegistry --> ToolDiscovery
    ToolDiscovery --> ToolFilter
    ToolFilter --> ToolExecutor
    ToolExecutor --> ResultProcessor

    ClientRegistry --> STDIOConn
    ClientRegistry --> HTTPConn
    ClientRegistry --> SSEConn

    STDIOConn --> FileSystem
    HTTPConn --> WebSearch
    HTTPConn --> Database
    STDIOConn --> Custom
```

---

## MCP Connection Architecture

### **Multi-Protocol Connection System**

Bifrost supports four MCP connection types, each optimized for different tool deployment patterns:

```mermaid
graph TB
    subgraph "InProcess Connections"
        InProcess[In-Memory Tools<br/>Same Process]
        InProcessEx[Examples:<br/>• Embedded tools<br/>• High-perf operations<br/>• Testing tools]
    end

    subgraph "STDIO Connections"
        STDIO[Command Line Tools<br/>Local Execution]
        STDIOEx[Examples:<br/>• Filesystem tools<br/>• Local scripts<br/>• CLI utilities]
    end

    subgraph "HTTP Connections"
        HTTP[Web Service Tools<br/>Remote APIs]
        HTTPEx[Examples:<br/>• Web search APIs<br/>• Database services<br/>• External integrations]
    end

    subgraph "SSE Connections"
        SSE[Real-time Tools<br/>Streaming Data]
        SSEEx[Examples:<br/>• Live data feeds<br/>• Real-time monitoring<br/>• Event streams]
    end

    subgraph "Connection Characteristics"
        Latency[Latency:<br/>InProcess < STDIO < HTTP < SSE]
        Security[Security:<br/>InProcess/Local > HTTP > SSE]
        Scalability[Scalability:<br/>HTTP > SSE > STDIO > InProcess]
        Complexity[Complexity:<br/>InProcess < STDIO < HTTP < SSE]
    end

    InProcess --> Latency
    STDIO --> Latency
    HTTP --> Security
    SSE --> Scalability
    HTTP --> Complexity
```

### **Connection Type Details**

**InProcess Connections (In-Memory Tools):**

- **Use Case:** Embedded tools, high-performance operations, testing
- **Performance:** Lowest possible latency (~0.1ms) with no IPC overhead
- **Security:** Highest security as tools run in the same process
- **Limitations:** Go package only, cannot be configured via JSON

**STDIO Connections (Local Tools):**

- **Use Case:** Command-line tools, local scripts, filesystem operations
- **Performance:** Low latency (~1-10ms) due to local execution
- **Security:** High security with full local control
- **Limitations:** Single-server deployment, resource sharing

**HTTP Connections (Remote Services):**

- **Use Case:** Web APIs, microservices, cloud functions
- **Performance:** Network-dependent latency (~10-500ms)
- **Security:** Configurable with authentication and encryption
- **Advantages:** Scalable, multi-server deployment, service isolation

**SSE Connections (Streaming Tools):**

- **Use Case:** Real-time data feeds, live monitoring, event streams
- **Performance:** Variable latency depending on stream frequency
- **Security:** Similar to HTTP with streaming capabilities
- **Benefits:** Real-time updates, persistent connections, event-driven

> **MCP Configuration:** [MCP Setup Guide →](../../mcp/overview)

---

## Tool Discovery & Registration

### **Dynamic Tool Discovery Process**

The MCP system discovers tools at runtime rather than requiring static configuration, enabling flexible and adaptive tool availability:

```mermaid
sequenceDiagram
    participant Bifrost
    participant MCPManager
    participant MCPServer
    participant ToolRegistry
    participant AIModel

    Note over Bifrost: System Startup
    Bifrost->>MCPManager: Initialize MCP System
    MCPManager->>MCPServer: Establish Connection
    MCPServer-->>MCPManager: Connection Ready

    MCPManager->>MCPServer: List Available Tools
    MCPServer-->>MCPManager: Tool Definitions
    MCPManager->>ToolRegistry: Register Tools

    Note over Bifrost: Runtime Request Processing
    AIModel->>MCPManager: Request Available Tools
    MCPManager->>ToolRegistry: Query Tools
    ToolRegistry-->>MCPManager: Filtered Tool List
    MCPManager-->>AIModel: Available Tools

    AIModel->>MCPManager: Execute Tool Call
    MCPManager->>MCPServer: Tool Invocation
    MCPServer->>MCPServer: Execute Tool Logic
    MCPServer-->>MCPManager: Tool Result
    MCPManager-->>AIModel: Enhanced Response
```

### **Tool Registry Management**

**Registration Process:**

1. **Connection Establishment** - MCP client connects to configured servers
2. **Capability Exchange** - Server announces available tools and schemas
3. **Tool Validation** - Bifrost validates tool definitions and security
4. **Registry Update** - Tools are registered in the internal tool registry
5. **Availability Notification** - Tools become available for AI model use

**Registry Features:**

- **Dynamic Updates** - Tools can be added/removed during runtime
- **Version Management** - Support for tool versioning and compatibility
- **Access Control** - Request-level tool filtering and permissions
- **Health Monitoring** - Continuous tool availability checking

**Tool Metadata Structure:**

- **Name & Description** - Human-readable tool identification
- **Parameters Schema** - JSON schema for tool input validation
- **Return Schema** - Expected response format definition
- **Capabilities** - Tool feature flags and limitations
- **Authentication** - Required credentials and permissions

---

## Tool Filtering & Access Control

### **Multi-Level Filtering System**

Bifrost provides granular control over tool availability through a sophisticated filtering system:

```mermaid
flowchart TD
    Request[Incoming Request] --> GlobalFilter{Global MCP Filter}
    GlobalFilter -->|Enabled| ClientFilter[MCP Client Filtering]
    GlobalFilter -->|Disabled| NoMCP[No MCP Tools]

    ClientFilter --> IncludeClients{Include Clients?}
    IncludeClients -->|Yes| IncludeList[Include Specified<br/>MCP Clients]
    IncludeClients -->|No| AllClients[All MCP Clients]

    IncludeList --> ExcludeClients{Exclude Clients?}
    AllClients --> ExcludeClients
    ExcludeClients -->|Yes| RemoveClients[Remove Excluded<br/>MCP Clients]
    ExcludeClients -->|No| ClientsFiltered[Filtered Clients]

    RemoveClients --> ToolFilter[Tool-Level Filtering]
    ClientsFiltered --> ToolFilter

    ToolFilter --> IncludeTools{Include Tools?}
    IncludeTools -->|Yes| IncludeSpecific[Include Specified<br/>Tools Only]
    IncludeTools -->|No| AllTools[All Available Tools]

    IncludeSpecific --> ExcludeTools{Exclude Tools?}
    AllTools --> ExcludeTools
    ExcludeTools -->|Yes| RemoveTools[Remove Excluded<br/>Tools]
    ExcludeTools -->|No| FinalTools[Final Tool Set]

    RemoveTools --> FinalTools
    FinalTools --> AIModel[Available to AI Model]
    NoMCP --> AIModel
```

### **Filtering Configuration Levels**

**Request-Level Filtering:**

```bash
# Include only specific MCP clients
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "x-bf-mcp-include-clients: filesystem,websearch" \
  -d '{"model": "gpt-4o-mini", "messages": [...]}'

# Include only specific tools
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "x-bf-mcp-include-tools: filesystem-read_file,websearch-search" \
  -d '{"model": "gpt-4o-mini", "messages": [...]}'
```

**Configuration-Level Filtering:**

- **Client Selection** - Choose which MCP servers to connect to
- **Tool Blacklisting** - Permanently disable dangerous or unwanted tools
- **Permission Mapping** - Map user roles to available tool sets
- **Environment-Based** - Different tool sets for development vs production

**Security Benefits:**

- **Principle of Least Privilege** - Only necessary tools are exposed
- **Dynamic Access Control** - Per-request tool availability
- **Audit Trail** - Track which tools are used by which requests
- **Risk Mitigation** - Prevent access to dangerous operations

> **📖 Tool Filtering:** [MCP Tool Control →](../../mcp/filtering)

---

## Tool Execution Engine

### **Async Tool Execution Architecture**

The MCP execution engine handles tool invocation asynchronously to maintain system responsiveness and enable complex multi-tool workflows:

```mermaid
sequenceDiagram
    participant AIModel
    participant ExecutionEngine
    participant ToolInvoker
    participant MCPServer
    participant ResultProcessor

    AIModel->>ExecutionEngine: Tool Call Request
    ExecutionEngine->>ExecutionEngine: Validate Tool Call
    ExecutionEngine->>ToolInvoker: Queue Tool Execution

    Note over ToolInvoker: Async Tool Execution
    ToolInvoker->>MCPServer: Invoke Tool
    MCPServer->>MCPServer: Execute Tool Logic
    MCPServer-->>ToolInvoker: Raw Tool Result

    ToolInvoker->>ResultProcessor: Process Result
    ResultProcessor->>ResultProcessor: Format & Validate
    ResultProcessor-->>ExecutionEngine: Processed Result

    ExecutionEngine-->>AIModel: Tool Execution Complete

    Note over AIModel: Multi-turn Conversation
    AIModel->>ExecutionEngine: Continue with Tool Results
    ExecutionEngine->>ExecutionEngine: Merge Results into Context
    ExecutionEngine-->>AIModel: Enhanced Response
```

### **Execution Flow Characteristics**

**Validation Phase:**

- **Parameter Validation** - Ensure tool arguments match expected schema
- **Permission Checking** - Verify tool access permissions for the request
- **Rate Limiting** - Apply per-tool and per-user rate limits
- **Security Scanning** - Check for potentially dangerous operations

**Execution Phase:**

- **Timeout Management** - Bounded execution time to prevent hanging
- **Error Handling** - Graceful handling of tool failures and timeouts
- **Result Streaming** - Support for tools that return streaming responses
- **Resource Monitoring** - Track tool resource usage and performance

**Response Phase:**

- **Result Formatting** - Convert tool outputs to consistent format
- **Error Enrichment** - Add context and suggestions for tool failures
- **Multi-Result Aggregation** - Combine multiple tool outputs coherently
- **Context Integration** - Merge tool results into conversation context

### **Multi-Turn Conversation Support**

The MCP system enables sophisticated multi-turn conversations where AI models can:

1. **Initial Tool Discovery** - Request available tools for a given context
2. **Tool Execution** - Execute one or more tools based on user request
3. **Result Analysis** - Analyze tool outputs and determine next steps
4. **Follow-up Actions** - Execute additional tools based on previous results
5. **Response Synthesis** - Combine tool results into coherent user response

**Example Multi-Turn Flow:**

```
User: "Find recent news about AI and save interesting articles"
AI: → Execute web_search("AI news recent")
AI: → Analyze search results
AI: → Execute save_article() for each interesting result
AI: → Respond with summary of saved articles
```

### **Complete User-Controlled Tool Execution Flow**

The following diagram shows the end-to-end user experience with MCP tool execution, highlighting the critical user control points and decision-making process:

```mermaid
flowchart TD
    A["👤 User Message<br/>\"List files in current directory\""] --> B["🤖 Bifrost Core"]

    B --> C["🔧 MCP Manager<br/>Auto-discovers and adds<br/>available tools to request"]

    C --> D["🌐 LLM Provider<br/>(OpenAI, Anthropic, etc.)"]

    D --> E{"🔍 Response contains<br/>tool_calls?"}

    E -->|No| F["✅ Final Response<br/>Display to user"]

    E -->|Yes| G["📝 Add assistant message<br/>with tool_calls to history"]

    G --> H["🛡️ YOUR EXECUTION LOGIC<br/>(Security, Approval, Logging)"]

    H --> I{"🤔 User Decision Point<br/>Execute this tool?"}

    I -->|Deny| J["❌ Create denial result<br/>Add to conversation history"]

    I -->|Approve| K["⚙️ client.ExecuteMCPTool()<br/>Bifrost executes via MCP"]

    K --> L["📊 Tool Result<br/>Add to conversation history"]

    J --> M["🔄 Continue conversation loop<br/>Send updated history back to LLM"]
    L --> M

    M --> D

    style A fill:#e1f5fe
    style F fill:#e8f5e8
    style H fill:#fff3e0
    style I fill:#fce4ec
    style K fill:#f3e5f5
```

**Key Flow Characteristics:**

**User Control Points:**

- **Security Layer** - Your application controls all tool execution decisions
- **Approval Gate** - Users can approve or deny each tool execution
- **Transparency** - Full visibility into what tools will be executed and why
- **Conversation Continuity** - Tool results seamlessly integrate into conversation flow

**Security Benefits:**

- **No Automatic Execution** - Tools never execute without explicit approval
- **Audit Trail** - Complete logging of all tool execution decisions
- **Contextual Security** - Approval decisions can consider full conversation context
- **Graceful Denials** - Denied tools result in informative responses, not errors

**Implementation Patterns:**

```go
// Example tool execution control in your application
func handleToolExecution(toolCall schemas.ChatToolCall, userContext UserContext) error {
    // YOUR SECURITY AND APPROVAL LOGIC HERE
    if !userContext.HasPermission(toolCall.Function.Name) {
        return createDenialResponse("Tool not permitted for user role")
    }

    if requiresApproval(toolCall) {
        approved := promptUserForApproval(toolCall)
        if !approved {
            return createDenialResponse("User denied tool execution")
        }
    }

    // Execute the tool via Bifrost
    result, err := client.ExecuteMCPTool(ctx, toolCall)
    if err != nil {
        return handleToolError(err)
    }

    return addToolResultToHistory(result)
}
```

This flow ensures that while AI models can discover and request tool usage, all actual execution remains under user control, providing the perfect balance of AI capability and human oversight.

---

## Agent Mode Architecture

Agent Mode transforms Bifrost into an autonomous agent runtime by automatically executing pre-approved tools. This section details the internal architecture of the agent execution loop.

### **Agent Execution Loop**

The agent mode operates as an iterative loop that continues until one of the termination conditions is met:

```mermaid
flowchart TD
    subgraph "Agent Mode Entry"
        A["📥 Incoming Chat Request"] --> B{"🔍 Check MCP Config<br/>Any tools_to_auto_execute?"}
        B -->|No| C["📤 Standard Flow<br/>Return tool_calls for manual execution"]
        B -->|Yes| D["🤖 Enter Agent Loop"]
    end

    subgraph "Agent Execution Loop"
        D --> E["🌐 Send to LLM Provider<br/>With available tools"]
        E --> F{"🔧 Response has<br/>tool_calls?"}
        F -->|No| G["✅ Return Final Response<br/>No more tools needed"]
        F -->|Yes| H["📋 Classify Tool Calls"]

        H --> I{"🔐 Separate by<br/>auto-execute status"}
        I --> J["⚡ Auto-Executable Tools"]
        I --> K["🛡️ Non-Auto-Executable Tools"]

        J --> L["🔄 Execute in Parallel<br/>Via ToolsManager"]
        L --> M["📊 Collect Results"]

        K --> N{"Any non-auto<br/>tools found?"}
        N -->|Yes| O["🛑 Exit Loop Early<br/>Return mixed response"]
        N -->|No| P{"⏱️ Max depth<br/>reached?"}

        M --> P
        P -->|Yes| Q["⚠️ Return Current State<br/>May have pending tools"]
        P -->|No| R["📝 Add results to history"]
        R --> E
    end

    subgraph "Response Handling"
        O --> S["📦 Create Mixed Response<br/>• Content: executed results JSON<br/>• tool_calls: pending tools<br/>• finish_reason: stop"]
        G --> T["📦 Standard Response<br/>Final answer from LLM"]
        Q --> U["📦 Depth Limit Response<br/>Current state with any pending"]
    end

    style D fill:#e3f2fd
    style L fill:#e8f5e9
    style O fill:#fff3e0
    style S fill:#fce4ec
```

### **Tool Classification System**

When the LLM returns tool calls, Bifrost classifies each tool based on the client configuration:

```mermaid
flowchart LR
    subgraph "Tool Call Classification"
        TC["🔧 Tool Call<br/>from LLM Response"] --> CHECK{"Tool in<br/>tools_to_execute?"}
        CHECK -->|No| SKIP["❌ Skip<br/>Not allowed"]
        CHECK -->|Yes| AUTO{"Tool in<br/>tools_to_auto_execute?"}
        AUTO -->|Yes| EXEC["⚡ Auto-Execute<br/>Run immediately"]
        AUTO -->|No| MANUAL["🛡️ Manual<br/>Return to caller"]
    end

    subgraph "Configuration Example"
        CONFIG["MCPClientConfig"]
        CONFIG --> TE["tools_to_execute: [*]<br/>All tools available"]
        CONFIG --> TAE["tools_to_auto_execute:<br/>[read_file, list_dir]"]
    end

    style EXEC fill:#c8e6c9
    style MANUAL fill:#fff9c4
    style SKIP fill:#ffcdd2
```

### **Mixed Tool Response Format**

When a response contains both auto-executable and non-auto-executable tools, the agent creates a special response format:

<AccordionGroup>
  <Accordion title="Chat API Response Format" icon="message" defaultOpen>

```json
{
  "id": "chatcmpl-abc123",
  "choices": [{
    "index": 0,
    "finish_reason": "stop",
    "message": {
      "role": "assistant",
      "content": "The Output from allowed tools calls is - {\"filesystem_read_file\":\"file contents here\",\"filesystem_list_directory\":\"[\\\"file1.txt\\\",\\\"file2.txt\\\"]\"}\n\nNow I shall call these tools next...",
      "tool_calls": [
        {
          "id": "call_write_123",
          "type": "function",
          "function": {
            "name": "filesystem_write_file",
            "arguments": "{\"path\":\"output.txt\",\"content\":\"...\"}"
          }
        }
      ]
    }
  }]
}
```

<Note>
The `content` field contains JSON-formatted results from auto-executed tools. The `tool_calls` array contains only non-auto-executable tools awaiting approval. Setting `finish_reason` to `"stop"` ensures the agent loop exits.
</Note>

  </Accordion>

  <Accordion title="Responses API Format" icon="code">

```json
{
  "id": "resp-abc123",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [{
        "type": "text",
        "text": "The Output from allowed tools calls is - {...}\n\nNow I shall call these tools next..."
      }]
    },
    {
      "type": "function_call",
      "role": "assistant",
      "call_id": "call_write_123",
      "name": "filesystem_write_file",
      "arguments": "{\"path\":\"output.txt\",\"content\":\"...\"}"
    }
  ]
}
```

  </Accordion>
</AccordionGroup>

### **Agent Depth Control**

The `max_agent_depth` setting prevents infinite loops and controls resource usage:

```mermaid
graph LR
    subgraph "Depth Tracking"
        D0["Depth 0<br/>Initial Request"] --> D1["Depth 1<br/>First tool execution"]
        D1 --> D2["Depth 2<br/>Second iteration"]
        D2 --> D3["Depth 3<br/>..."]
        D3 --> DN["Depth N<br/>Max reached"]
    end

    DN --> EXIT["🛑 Force Exit<br/>Return current state"]

    subgraph "Configuration"
        CFG["MCPToolManagerConfig"]
        CFG --> MAX["max_agent_depth: 10<br/>(default)"]
        CFG --> TIMEOUT["tool_execution_timeout:<br/>30s per tool"]
    end
```

<Warning>
When max depth is reached, the response may contain pending tool calls that weren't executed. Your application should handle this gracefully.
</Warning>

---

## Code Mode Architecture

Code Mode enables AI models to write and execute Python code (Starlark) that orchestrates multiple MCP tools in a single request. This provides a powerful meta-layer for complex multi-tool workflows.

### **Code Mode System Overview**

```mermaid
graph TB
    subgraph "Code Mode Components"
        VM["🖥️ Starlark Interpreter<br/>Python-like Runtime"]
        VFS["📁 Virtual File System<br/>Tool Definitions as .pyi"]
        EXEC["⚙️ Code Executor<br/>Sandboxed Execution"]
    end

    subgraph "Meta Tools"
        LIST["listToolFiles()<br/>Discover available servers"]
        READ["readToolFile(fileName)<br/>Get tool signatures"]
        DOCS["getToolDocs(server, tool)<br/>Get detailed docs"]
        CODE["executeToolCode(code)<br/>Run Python code"]
    end

    subgraph "MCP Integration"
        TOOLS["🔧 Connected MCP Tools"]
        RESULTS["📊 Tool Results"]
    end

    LLM["🤖 LLM"] --> LIST
    LIST --> VFS
    VFS --> LLM
    LLM --> READ
    READ --> VFS
    VFS --> LLM
    LLM --> DOCS
    DOCS --> VFS
    VFS --> LLM
    LLM --> CODE
    CODE --> VM
    VM --> EXEC
    EXEC --> TOOLS
    TOOLS --> RESULTS
    RESULTS --> LLM

    style VM fill:#e8eaf6
    style VFS fill:#e3f2fd
    style CODE fill:#e8f5e9
```

### **Virtual File System (VFS)**

Code Mode generates Python stub files (`.pyi`) for all connected MCP tools, providing compact function signatures:

<Tabs>
  <Tab title="Server-Level Binding">

When `code_mode_binding_level: "server"` (default), tools are grouped by MCP client:

```
servers/
├── filesystem.pyi      → All filesystem tools
├── web_search.pyi      → All web search tools
└── database.pyi        → All database tools
```

**Generated Stub Example:**
```python
# servers/filesystem.pyi
# Usage: filesystem.tool_name(param=value)
# For detailed docs: use getToolDocs(server="filesystem", tool="tool_name")

def read_file(path: str) -> dict:  # Read contents of a file
def write_file(path: str, content: str) -> dict:  # Write content to a file
def list_directory(path: str) -> dict:  # List directory contents
```

**Usage in Code:**
```python
files = filesystem.list_directory(path=".")
content = filesystem.read_file(path=files["entries"][0])
result = content
```

  </Tab>
  <Tab title="Tool-Level Binding">

When `code_mode_binding_level: "tool"`, each tool gets its own file:

```
servers/
├── filesystem/
│   ├── read_file.pyi
│   ├── write_file.pyi
│   └── list_directory.pyi
├── web_search/
│   └── search.pyi
└── database/
    └── query.pyi
```

**Generated Stub Example:**
```python
# servers/filesystem/read_file.pyi
# Usage: filesystem.read_file(param=value)

def read_file(path: str) -> dict:  # Read contents of a file
```

**Usage in Code:**
```python
content = filesystem.read_file(path="config.json")
result = content
```

  </Tab>
</Tabs>

### **Code Execution Flow**

```mermaid
sequenceDiagram
    participant LLM as 🤖 LLM
    participant CM as 📝 Code Mode Handler
    participant VM as 🖥️ Starlark Interpreter
    participant TM as 🔧 Tools Manager
    participant MCP as 🌐 MCP Servers

    LLM->>CM: executeToolCode({ code: "..." })
    CM->>VM: Initialize sandbox
    CM->>VM: Inject tool bindings
    CM->>VM: Execute Python code

    loop For each tool call in code
        VM->>TM: server.tool(param=value)
        TM->>MCP: Execute tool
        MCP-->>TM: Tool result
        TM-->>VM: Return result
    end

    VM-->>CM: Execution result
    CM-->>LLM: { result, logs }
```

### **Starlark Sandbox**

The code execution environment is carefully sandboxed using Starlark, a Python-like language designed for configuration and embedded scripting:

<AccordionGroup>
  <Accordion title="Available Features" icon="check" defaultOpen>

  - ✅ **Python-like syntax** - Familiar Python syntax and semantics
  - ✅ **Synchronous calls** - No async/await needed, direct function calls
  - ✅ **List comprehensions** - `[x for x in items if condition]`
  - ✅ **print()** - Output captured and returned in logs
  - ✅ **Dict/List operations** - Standard Python data structures
  - ✅ **Tool bindings** - All connected MCP tools as globals
  </Accordion>

  <Accordion title="Restricted Features" icon="ban">

  - ❌ **Imports** - No `import` statements (tools are pre-bound)
  - ❌ **Classes** - Use dicts and functions instead
  - ❌ **File I/O** - No direct filesystem access (use MCP tools)
  - ❌ **Network** - No direct network access (use MCP tools)
  - ❌ **Randomness/Time** - Deterministic execution only

  </Accordion>
</AccordionGroup>

### **Code Mode Security Model**

```mermaid
graph TB
    subgraph "Security Layers"
        L1["🔒 Code Validation<br/>Syntax checking before execution"]
        L2["🛡️ Sandboxed Runtime<br/>No external module access"]
        L3["⏱️ Execution Timeout<br/>Bounded runtime"]
        L4["🔐 Tool ACL<br/>Only allowed tools accessible"]
    end

    subgraph "Execution Boundaries"
        B1["No filesystem access<br/>(except via MCP tools)"]
        B2["No network access<br/>(except via MCP tools)"]
        B3["No process spawning"]
        B4["Memory isolation enforced"]
    end

    L1 --> L2 --> L3 --> L4
    L4 --> B1
    L4 --> B2
    L4 --> B3
    L4 --> B4
```

### **Code Mode Configuration**

<Tabs>
  <Tab title="Gateway (config.json)">

```json
{
  "mcp": {
    "client_configs": [
      {
        "name": "filesystem",
        "is_code_mode_client": true,
        "connection_type": "stdio",
        "stdio_config": {
          "command": "npx",
          "args": ["-y", "@anthropic/mcp-filesystem"]
        },
        "tools_to_execute": ["*"]
      }
    ],
    "tool_manager_config": {
      "code_mode_binding_level": "server",
      "tool_execution_timeout": "30s"
    }
  }
}
```

  </Tab>
  <Tab title="Go SDK">

```go
mcpConfig := &schemas.MCPConfig{
    ClientConfigs: []schemas.MCPClientConfig{
        {
            Name:             "filesystem",
            IsCodeModeClient: true,
            ConnectionType:   schemas.MCPConnectionTypeSTDIO,
            StdioConfig: &schemas.MCPStdioConfig{
                Command: "npx",
                Args:    []string{"-y", "@anthropic/mcp-filesystem"},
            },
            ToolsToExecute: []string{"*"},
        },
    },
    ToolManagerConfig: &schemas.MCPToolManagerConfig{
        CodeModeBindingLevel: schemas.CodeModeBindingLevelServer,
        ToolExecutionTimeout: 30 * time.Second,
    },
}
```

  </Tab>
</Tabs>

### **Code Mode vs Agent Mode**

| Aspect | Agent Mode | Code Mode |
|--------|------------|-----------|
| **Execution Model** | LLM decides one tool at a time | LLM writes code orchestrating multiple tools |
| **Iterations** | Multiple LLM round-trips | Single LLM call, code handles orchestration |
| **Complexity** | Simple tool chains | Complex workflows with conditionals/loops |
| **Latency** | Higher (multiple LLM calls) | Lower (single LLM call + code execution) |
| **Control** | Per-tool approval possible | Code runs atomically |
| **Best For** | Interactive agents | Batch operations, complex data processing |

---

## MCP Integration Patterns

### **Common Integration Scenarios**

**1. Filesystem Operations**

- **Tools:** `list_files`, `read_file`, `write_file`, `create_directory`
- **Use Cases:** Code analysis, document processing, file management
- **Security:** Sandboxed file access, path validation, permission checks
- **Performance:** Local execution for fast file operations

**2. Web Search & Information Retrieval**

- **Tools:** `web_search`, `fetch_url`, `extract_content`, `summarize`
- **Use Cases:** Research assistance, fact-checking, content gathering
- **Integration:** External search APIs, content parsing services
- **Caching:** Response caching for repeated queries

**3. Database Operations**

- **Tools:** `query_database`, `insert_record`, `update_record`, `schema_info`
- **Use Cases:** Data analysis, report generation, database administration
- **Security:** Read-only access by default, query validation, injection prevention
- **Performance:** Connection pooling, query optimization

**4. API Integrations**

- **Tools:** Custom business logic tools, third-party service integration
- **Use Cases:** CRM operations, payment processing, notification sending
- **Authentication:** API key management, OAuth token handling
- **Error Handling:** Retry logic, fallback mechanisms

### **MCP Server Development Patterns**

**Simple STDIO Server:**

- **Language:** Any language that can read/write JSON to stdin/stdout
- **Deployment:** Single executable, minimal dependencies
- **Use Case:** Local tools, development utilities, simple scripts

**HTTP Service Server:**

- **Architecture:** RESTful API with MCP protocol endpoints
- **Scalability:** Horizontal scaling, load balancing
- **Use Case:** Shared tools, enterprise integrations, cloud services

**Hybrid Approach:**

- **Local + Remote:** Combine STDIO tools for local operations with HTTP for remote services
- **Failover:** Use local fallbacks when remote services are unavailable
- **Optimization:** Route tool calls to most appropriate execution environment

> **📖 MCP Development:** [Tool Development Guide →](../../mcp/overview)

---

## Security & Safety Considerations

### **MCP Security Architecture**

```mermaid
graph TB
    subgraph "Security Layers"
        L1[Connection Security<br/>Authentication & Encryption]
        L2[Tool Validation<br/>Schema & Permission Checks]
        L3[Execution Security<br/>Sandboxing & Limits]
        L4[Result Security<br/>Output Validation & Filtering]
    end

    subgraph "Threat Mitigation"
        T1[Malicious Tools<br/>Code Injection Prevention]
        T2[Resource Abuse<br/>Rate Limiting & Quotas]
        T3[Data Exposure<br/>Output Sanitization]
        T4[System Access<br/>Privilege Isolation]
    end

    L1 --> T1
    L2 --> T2
    L3 --> T4
    L4 --> T3
```

**Security Measures:**

**Connection Security:**

- **Authentication** - API keys, certificates, or token-based auth for HTTP/SSE
- **Encryption** - TLS for HTTP connections, secure pipes for STDIO
- **Network Isolation** - Firewall rules and network segmentation

**Execution Security:**

- **Sandboxing** - Isolated execution environments for tools
- **Resource Limits** - CPU, memory, and time constraints
- **Permission Model** - Principle of least privilege for tool access

**Operational Security:**

- **Regular Updates** - Keep MCP servers and tools updated
- **Monitoring** - Continuous security monitoring and alerting
- **Incident Response** - Procedures for security incidents involving tools

---

## Related Architecture Documentation

- **[Request Flow](./request-flow)** - MCP integration in request processing
- **[Concurrency Model](./concurrency)** - MCP concurrency and worker integration
- **[Plugin System](./plugins)** - Integration between MCP and plugin systems
- **[Benchmarks](../../benchmarking/getting-started)** - MCP performance impact and optimization