186 lines
5.8 KiB
Plaintext
186 lines
5.8 KiB
Plaintext
---
|
|
title: "Vector Store"
|
|
description: "Vector database implementations for semantic search, embeddings storage, and AI-powered features in Bifrost."
|
|
icon: "diagram-project"
|
|
---
|
|
|
|
## Overview
|
|
|
|
The VectorStore is a core component of Bifrost's framework package that provides a unified interface for vector database operations. It enables plugins to store embeddings, perform similarity searches, and build AI-powered features like semantic caching, content recommendations, and knowledge retrieval.
|
|
|
|
**Key Capabilities:**
|
|
- **Vector Similarity Search**: Find semantically similar content using embeddings
|
|
- **Namespace Management**: Organize data into separate collections with custom schemas
|
|
- **Flexible Filtering**: Query data with complex filters and pagination
|
|
- **Multiple Backends**: Support for Weaviate, Redis/Valkey-compatible, Qdrant, and Pinecone vector stores
|
|
- **High Performance**: Optimized for production workloads
|
|
- **Scalable Storage**: Handle millions of vectors with efficient indexing
|
|
|
|
## VectorStore Interface Usage
|
|
|
|
### Creating Namespaces
|
|
Create collections (namespaces) with custom schemas:
|
|
|
|
```go
|
|
// Define properties for your data
|
|
properties := map[string]vectorstore.VectorStoreProperties{
|
|
"content": {
|
|
DataType: vectorstore.VectorStorePropertyTypeString,
|
|
Description: "The main content text",
|
|
},
|
|
"category": {
|
|
DataType: vectorstore.VectorStorePropertyTypeString,
|
|
Description: "Content category",
|
|
},
|
|
"tags": {
|
|
DataType: vectorstore.VectorStorePropertyTypeStringArray,
|
|
Description: "Content tags",
|
|
},
|
|
}
|
|
|
|
// Create namespace
|
|
err := store.CreateNamespace(ctx, "my_content", 1536, properties)
|
|
if err != nil {
|
|
log.Fatal("Failed to create namespace:", err)
|
|
}
|
|
```
|
|
|
|
### Storing Data with Embeddings
|
|
Add data with vector embeddings for similarity search:
|
|
|
|
```go
|
|
// Your embedding data (typically from an embedding model)
|
|
embedding := []float32{0.1, 0.2, 0.3 } // example 3-dimensional vector
|
|
|
|
// Metadata associated with this vector
|
|
metadata := map[string]interface{}{
|
|
"content": "This is my content text",
|
|
"category": "documentation",
|
|
"tags": []string{"guide", "tutorial"},
|
|
}
|
|
|
|
// Store in vector database
|
|
err := store.Add(ctx, "my_content", "unique-id-123", embedding, metadata)
|
|
if err != nil {
|
|
log.Fatal("Failed to add data:", err)
|
|
}
|
|
```
|
|
|
|
### Similarity Search
|
|
Find similar content using vector similarity:
|
|
|
|
```go
|
|
// Query embedding (from user query)
|
|
queryEmbedding := []float32{0.15, 0.25, 0.35, ...}
|
|
|
|
// Optional filters
|
|
filters := []vectorstore.Query{
|
|
{
|
|
Field: "category",
|
|
Operator: vectorstore.QueryOperatorEqual,
|
|
Value: "documentation",
|
|
},
|
|
}
|
|
|
|
// Perform similarity search
|
|
results, err := store.GetNearest(
|
|
ctx,
|
|
"my_content", // namespace
|
|
queryEmbedding, // query vector
|
|
filters, // optional filters
|
|
[]string{"content", "category"}, // fields to return
|
|
0.7, // similarity threshold (0-1)
|
|
10, // limit
|
|
)
|
|
|
|
for _, result := range results {
|
|
fmt.Printf("Score: %.3f, Content: %s\n", *result.Score, result.Properties["content"])
|
|
}
|
|
```
|
|
|
|
### Data Retrieval and Management
|
|
Query and manage stored data:
|
|
|
|
```go
|
|
// Get specific item by ID
|
|
item, err := store.GetChunk(ctx, "my_content", "unique-id-123")
|
|
if err != nil {
|
|
log.Fatal("Failed to get item:", err)
|
|
}
|
|
|
|
// Get all items with filtering and pagination
|
|
allResults, cursor, err := store.GetAll(
|
|
ctx,
|
|
"my_content",
|
|
[]vectorstore.Query{
|
|
{Field: "category", Operator: vectorstore.QueryOperatorEqual, Value: "documentation"},
|
|
},
|
|
[]string{"content", "tags"}, // select fields
|
|
nil, // cursor for pagination
|
|
50, // limit
|
|
)
|
|
|
|
// Delete items
|
|
err = store.Delete(ctx, "my_content", "unique-id-123")
|
|
```
|
|
|
|
## Supported Vector Stores
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Weaviate" icon="database" href="/integrations/vector-databases/weaviate">
|
|
Production-ready vector database with gRPC support.
|
|
</Card>
|
|
<Card title="Redis / Valkey" icon="database" href="/integrations/vector-databases/redis">
|
|
High-performance in-memory vector store.
|
|
</Card>
|
|
<Card title="Qdrant" icon="database" href="/integrations/vector-databases/qdrant">
|
|
Rust-based vector search engine with advanced filtering.
|
|
</Card>
|
|
<Card title="Pinecone" icon="database" href="/integrations/vector-databases/pinecone">
|
|
Managed vector database with serverless options.
|
|
</Card>
|
|
</CardGroup>
|
|
|
|
---
|
|
|
|
## Use Cases
|
|
|
|
### [Semantic Caching](../../features/semantic-caching)
|
|
Build intelligent caching systems that understand query intent rather than just exact matches.
|
|
|
|
**Applications:**
|
|
- Customer support systems with FAQ matching
|
|
- Code completion and documentation search
|
|
- Content management with semantic deduplication
|
|
|
|
### Knowledge Base & Search
|
|
Create intelligent search systems that understand user queries contextually.
|
|
|
|
**Applications:**
|
|
- Document search and retrieval systems
|
|
- Product recommendation engines
|
|
- Research paper and knowledge discovery platforms
|
|
|
|
### Content Classification
|
|
Automatically categorize and tag content based on semantic similarity.
|
|
|
|
**Applications:**
|
|
- Email classification and routing
|
|
- Content moderation and filtering
|
|
- News article categorization and clustering
|
|
|
|
### Recommendation Systems
|
|
Build personalized recommendation engines using vector similarity.
|
|
|
|
**Applications:**
|
|
- Product recommendations based on user preferences
|
|
- Content suggestions for media platforms
|
|
- Similar document or article recommendations
|
|
|
|
## Related Documentation
|
|
|
|
| Topic | Documentation | Description |
|
|
|-------|---------------|-------------|
|
|
| **Framework Overview** | [What is Framework](./what-is-framework) | Understanding the framework package and VectorStore interface |
|
|
| **Semantic Caching** | [Semantic Caching](../../features/semantic-caching) | Using VectorStore for AI response caching |
|