import { Button } from "@/components/ui/button"; import { Card, CardContent } from "@/components/ui/card"; import { Input } from "@/components/ui/input"; import { Label } from "@/components/ui/label"; import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select"; import { Separator } from "@/components/ui/separator"; import { Switch } from "@/components/ui/switch"; import { getProviderLabel } from "@/lib/constants/logs"; import { getErrorMessage, useCreatePluginMutation, useGetPluginsQuery, useGetProvidersQuery, useUpdatePluginMutation } from "@/lib/store"; import { CacheConfig, EditorCacheConfig, ModelProviderName } from "@/lib/types/config"; import { SEMANTIC_CACHE_PLUGIN } from "@/lib/types/plugins"; import { cacheConfigSchema } from "@/lib/types/schemas"; import { Loader2 } from "lucide-react"; import { useEffect, useMemo, useState } from "react"; import { toast } from "sonner"; const defaultCacheConfig: EditorCacheConfig = { ttl_seconds: 300, threshold: 0.8, conversation_history_threshold: 3, exclude_system_prompt: false, cache_by_model: true, cache_by_provider: true, }; const toEditorCacheConfig = (config?: Partial): EditorCacheConfig => ({ ...defaultCacheConfig, ...config, }); const normalizeCacheConfigForSave = (config: EditorCacheConfig) => { const normalized: Record = { ttl_seconds: config.ttl_seconds, threshold: config.threshold, cache_by_model: config.cache_by_model, cache_by_provider: config.cache_by_provider, }; if (config.conversation_history_threshold !== undefined) { normalized.conversation_history_threshold = config.conversation_history_threshold; } if (config.exclude_system_prompt !== undefined) { normalized.exclude_system_prompt = config.exclude_system_prompt; } if (config.created_at !== undefined) { normalized.created_at = config.created_at; } if (config.updated_at !== undefined) { normalized.updated_at = config.updated_at; } if (config.keys !== undefined) { normalized.keys = config.keys; } const provider = config.provider?.trim(); const embeddingModel = config.embedding_model?.trim(); if (provider) { normalized.provider = provider; } if (embeddingModel) { normalized.embedding_model = embeddingModel; } if (config.dimension !== undefined) { normalized.dimension = config.dimension; } return normalized; }; interface PluginsFormProps { isVectorStoreEnabled: boolean; } export default function PluginsForm({ isVectorStoreEnabled }: PluginsFormProps) { const [cacheConfig, setCacheConfig] = useState(defaultCacheConfig); const [originalCacheEnabled, setOriginalCacheEnabled] = useState(false); const [serverCacheConfig, setServerCacheConfig] = useState(defaultCacheConfig); const [serverCacheEnabled, setServerCacheEnabled] = useState(false); const { data: providersData, error: providersError, isLoading: providersLoading } = useGetProvidersQuery(); const providers = useMemo(() => providersData || [], [providersData]); useEffect(() => { if (providersError) { toast.error(`Failed to load providers: ${getErrorMessage(providersError as any)}`); } }, [providersError]); // RTK Query hooks const { data: plugins, isLoading: loading } = useGetPluginsQuery(); const [updatePlugin, { isLoading: isUpdating }] = useUpdatePluginMutation(); const [createPlugin, { isLoading: isCreating }] = useCreatePluginMutation(); // Get semantic cache plugin and its config const semanticCachePlugin = useMemo(() => plugins?.find((plugin) => plugin.name === SEMANTIC_CACHE_PLUGIN), [plugins]); const isSemanticCacheEnabled = Boolean(semanticCachePlugin?.enabled); const loadedDirectOnlyConfig = serverCacheConfig.dimension === 1 && !serverCacheConfig.provider; const hasInvalidProviderBackedDimension = cacheConfig.dimension === 1 && Boolean(cacheConfig.provider?.trim()); // Initialize cache config from plugin data useEffect(() => { if (semanticCachePlugin?.config) { const config = toEditorCacheConfig(semanticCachePlugin.config as Partial); setCacheConfig(config); setServerCacheConfig(config); setOriginalCacheEnabled(semanticCachePlugin.enabled); setServerCacheEnabled(semanticCachePlugin.enabled); } }, [semanticCachePlugin]); // Update default provider when providers are loaded (only for new configs) useEffect(() => { if (providers.length > 0 && !semanticCachePlugin?.config) { setCacheConfig((prev) => ({ ...prev, provider: providers[0].name as ModelProviderName, embedding_model: prev.embedding_model ?? "text-embedding-3-small", dimension: prev.dimension ?? 1536, })); } }, [providers, semanticCachePlugin?.config]); const hasChanges = useMemo(() => { if (originalCacheEnabled !== serverCacheEnabled) return true; return ( cacheConfig.provider !== serverCacheConfig.provider || cacheConfig.embedding_model !== serverCacheConfig.embedding_model || cacheConfig.dimension !== serverCacheConfig.dimension || cacheConfig.ttl_seconds !== serverCacheConfig.ttl_seconds || cacheConfig.threshold !== serverCacheConfig.threshold || cacheConfig.conversation_history_threshold !== serverCacheConfig.conversation_history_threshold || cacheConfig.exclude_system_prompt !== serverCacheConfig.exclude_system_prompt || cacheConfig.cache_by_model !== serverCacheConfig.cache_by_model || cacheConfig.cache_by_provider !== serverCacheConfig.cache_by_provider ); }, [cacheConfig, serverCacheConfig, originalCacheEnabled, serverCacheEnabled]); // Handle semantic cache toggle (create or update) const handleSemanticCacheToggle = (enabled: boolean) => { setOriginalCacheEnabled(enabled); }; // Update cache config locally const updateCacheConfigLocal = (updates: Partial) => { setCacheConfig((prev) => ({ ...prev, ...updates })); }; // Save all changes const handleSave = async () => { if (hasInvalidProviderBackedDimension) { toast.error( "Provider-backed semantic cache requires the embedding model's real dimension. Use a value greater than 1, or remove the provider to keep direct-only mode.", ); return; } const parseResult = cacheConfigSchema.safeParse(normalizeCacheConfigForSave(cacheConfig)); if (!parseResult.success) { const firstIssue = parseResult.error.issues[0]?.message ?? "Semantic cache configuration is invalid."; toast.error(firstIssue); return; } const savedConfig = parseResult.data as CacheConfig; try { if (semanticCachePlugin) { // Update existing plugin await updatePlugin({ name: SEMANTIC_CACHE_PLUGIN, data: { enabled: originalCacheEnabled, config: savedConfig }, }).unwrap(); } else { // Create new plugin await createPlugin({ name: SEMANTIC_CACHE_PLUGIN, enabled: originalCacheEnabled, config: savedConfig, path: "", }).unwrap(); } toast.success("Plugin configuration updated successfully"); // Update server state to match current state const normalizedConfig = toEditorCacheConfig(savedConfig); setCacheConfig(normalizedConfig); setServerCacheConfig(normalizedConfig); setServerCacheEnabled(originalCacheEnabled); } catch (error) { const errorMessage = getErrorMessage(error); toast.error(`Failed to update plugin configuration: ${errorMessage}`); } }; if (loading) { return (
Loading plugins configuration...
); } return (
{/* Semantic Cache Toggle */}

Enable semantic caching for requests. Send x-bf-cache-key header with requests to use semantic caching.{" "} {!isVectorStoreEnabled && ( Requires vector store to be configured and enabled in config.json. )} {!providersLoading && providers?.length === 0 && ( Requires at least one provider to be configured. )}

{ if (isVectorStoreEnabled) { handleSemanticCacheToggle(checked); } }} /> {(isSemanticCacheEnabled || originalCacheEnabled) && ( )}
{/* Cache Configuration (only show when enabled) */} {originalCacheEnabled && isVectorStoreEnabled && (providersLoading ? (
) : (
{loadedDirectOnlyConfig && (
This plugin was loaded in direct-only mode via config.json. The Web UI currently edits provider-backed semantic cache settings; keep using config.json if you want to stay in direct-only mode.
)} {hasInvalidProviderBackedDimension && (
You selected a provider while keeping dimension: 1. That is only valid for direct-only mode. Set the embedding model's real dimension before saving, or remove the provider to stay in direct-only mode.
)} {/* Provider and Model Settings */}

Provider and Model Settings

updateCacheConfigLocal({ embedding_model: e.target.value })} />
{/* Cache Settings */}

Cache Settings

{ const value = e.target.value; if (value === "") { updateCacheConfigLocal({ ttl_seconds: undefined }); return; } const parsed = parseInt(value); if (!Number.isNaN(parsed)) { updateCacheConfigLocal({ ttl_seconds: parsed }); } }} />
{ const value = e.target.value; if (value === "") { updateCacheConfigLocal({ threshold: undefined }); return; } const parsed = parseFloat(value); if (!Number.isNaN(parsed)) { updateCacheConfigLocal({ threshold: parsed }); } }} />
{ const value = e.target.value; if (value === "") { updateCacheConfigLocal({ dimension: undefined }); return; } const parsed = parseInt(value); if (!Number.isNaN(parsed)) { updateCacheConfigLocal({ dimension: parsed }); } }} />

API keys for the embedding provider will be inherited from the main provider configuration. The semantic cache will use the configured provider's keys automatically. Updates in keys will be reflected on Bifrost restart.

{/* Conversation Settings */}

Conversation Settings

updateCacheConfigLocal({ conversation_history_threshold: parseInt(e.target.value) || 3 })} />

Skip caching for conversations with more than this number of messages (prevents false positives)

Exclude system messages from cache key generation

updateCacheConfigLocal({ exclude_system_prompt: checked })} size="md" />
{/* Cache Behavior */}

Cache Behavior

Include model name in cache key

updateCacheConfigLocal({ cache_by_model: checked })} size="md" />

Include provider name in cache key

updateCacheConfigLocal({ cache_by_provider: checked })} size="md" />
  • You can pass x-bf-cache-ttl header with requests to use request-specific TTL.
  • You can pass x-bf-cache-threshold header with requests to use request-specific similarity threshold.
  • You can pass x-bf-cache-type header with "direct" or "semantic" to control cache behavior.
  • You can pass x-bf-cache-no-store header with "true" to disable response caching.
))}
); }