Skip to main content

Knowledge Bases

A Knowledge Base (KB) is a per-tenant collection of documents that are chunked, embedded, and made searchable for Retrieval-Augmented Generation (RAG) — used by the query-kb node and the kb_search agent tool.

Creating a Knowledge Base​

POST /api/v1/knowledge-bases
{
  "name": "Product Docs",
  "description": "Internal product documentation",
  "embed_provider": "openai",
  "embed_model": "text-embedding-3-small",
  "embed_api_key": "{{ secret.OPENAI_API_KEY }}",
  "chunk_size": 1000,
  "chunk_overlap": 200
}
FieldDescription
embed_provideropenai, nvidia, or custom (any OpenAI-compatible embeddings endpoint via embed_base_url).
embed_modelEmbedding model ID (e.g. text-embedding-3-small, an NVIDIA NIM embedding model).
chunk_size / chunk_overlapControls the recursive character text splitter (ChunkText) — splits on paragraph/sentence/word boundaries.

Adding documents​

EndpointUse for
POST /api/v1/knowledge-bases/{id}/documentsAdd a document by URL or raw text — the URL is fetched and HTML-stripped automatically.
POST /api/v1/knowledge-bases/{id}/documents/uploadUpload a file (PDF, text, etc.) directly.
GET /api/v1/knowledge-bases/{id}/documentsList documents with status (pending, processing, completed, failed) and chunk counts.
DELETE /api/v1/knowledge-bases/{id}/documents/{docId}Remove a document and its chunks.

Processing pipeline​

For each document, OrcFlows:

  1. Fetches the content (for URL sources) and strips HTML.
  2. Splits it into overlapping chunks using chunk_size/chunk_overlap.
  3. Embeds every chunk with the KB's configured embedding provider.
  4. Stores chunk text + embedding vector + metadata, and updates doc_count/chunk_count on the KB.

If processing fails at any step, the document's status becomes failed with an error message.

Querying​

POST /api/v1/knowledge-bases/{id}/query
{ "query": "How do I reset my password?", "top_k": 5 }

Returns the top-k chunks ranked by similarity, each with content, score, doc_name, and metadata — the same results a query-kb node or kb_search tool call would receive.

Storage backends​

By default, embeddings are stored as float4[] columns in PostgreSQL, with a custom cosine_similarity SQL function (added in migration 000002_knowledge_base.up.sql) — no extra infrastructure required, and works well for small-to-medium KBs.

For larger-scale RAG, set WEAVIATE_URL to point at the bundled Weaviate service (port 8082). Each KB is mapped to an isolated Weaviate tenant shard for true multi-tenant ANN (approximate nearest neighbor) search at scale. OrcFlows brings its own embeddings — Weaviate's vectorizer modules are disabled (DEFAULT_VECTORIZER_MODULE: none).

Using a KB in a workflow​

{
  "id": "search_docs",
  "type": "query-kb",
  "config": {
    "kb_id": "{{vars.product_docs_kb}}",
    "query": "{{trigger.body.question}}",
    "top_k": 5
  }
}

Or give an agent access:

"builtin_tools": ["kb_search"],
"config": { "kb_id": "{{vars.product_docs_kb}}" }

Next​