Knowledge Base (RAG)

Per-org document upload + chunking + embedding + cosine retrieval. Owners and admins upload PDFs, DOCX, Markdown, or text files; all roles can ask the AI questions and get answers cited from those documents via the searchKnowledgeBase tool.

---

What It Is

Documents are split into ~500-token chunks, embedded with OpenAI text-embedding-3-small (1536 dims), and stored in Postgres with the pgvector extension. The AI assistant calls searchKnowledgeBase at chat time, runs a cosine top-K lookup scoped by organizationId, and weaves snippets into its reply.

---

Local Dev Quick Start

The docker-compose Postgres image (pgvector/pgvector:pg15) already ships with pgvector. Two ways to run embeddings:

Offline (no API key):

# .env.local
EMBEDDING_PROVIDER="mock"

mock returns deterministic vectors so the upload → ingest → search pipeline runs end-to-end without a remote API call. Same input always produces the same vector. Good for tests, demos, and offline dev.

Real embeddings:

See LLM Providers for OpenAI key setup including production recovery steps.

Then start the stack:

make dev           # Next.js + Postgres
make inngest-dev   # Worker for document ingestion (separate terminal)

Without Inngest, uploads still work — the service falls back to inline ingestion. Rows just stay pending longer. To run Inngest properly in production see Inngest.

---

How Users Interact

Owners and admins navigate to /dashboard/knowledge-base, drop a file (up to 10 MB), and watch the row transition pending → processing → ready. Members do not see this page; the sidebar entry is filtered out for them.

Once a document is ready, anyone in the org can open the AI panel and ask a question. The assistant decides when to call searchKnowledgeBase, gets back the top-5 chunks, and cites them in its answer. The tool call surfaces in chat as a Knowledge base searched card with the source document title.

---

Production Setup

The make setup-deployment wizard prompts for a knowledge-base GCS bucket name. The wizard sets the GitHub repository variable and the Terraform terraform.tfvars value; both are read by their respective consumers (GitHub Actions for the Cloud Run env var, Terraform for bucket provisioning). Defaults to <gcp-project-id>-knowledge-base. Skip the prompt and uploads will fail in prod.

Cloud SQL needs the pgvector extension — the enable_pgvector migration runs CREATE EXTENSION IF NOT EXISTS vector; as part of prisma migrate deploy. Cloud SQL for PostgreSQL supports the vector extension directly, so no database flag is required.

First-time deploy: make setup-deployment runs terraform apply to create the Cloud SQL instance, then the deploy workflow runs prisma migrate deploy — the enable_pgvector migration enables the extension before the vector(1536) column is added.

EMBEDDING_PROVIDER is hardcoded to openai in the deploy workflow — mock mode is local-dev-only. OPENAI_API_KEY is set by the deployment wizard. Skipped it, or rotating the key? See LLM Providers — both gh secret set and terraform apply are required.

Inngest is required in production for RAG ingestion. See Inngest.

---

Extending

Concern	File
Chunker	src/services/rag-service.ts (chunkText)
Embedding provider	src/lib/ai/embedding-provider.ts
Retrieval RPC	src/lib/repositories/document-chunk-repository.ts (searchByCosine)
Tool definition	src/lib/ai/tools.ts (searchKnowledgeBase)
Ingestion pipeline	src/inngest/ingest-document.ts
Chunking constants	src/types/knowledge-base.ts

Swap embedding model: edit getEmbeddingModel() in embedding-provider.ts. The model dim is locked to 1536 to match the vector(1536) column. Changing dim requires a Prisma migration, dropping and recreating the ivfflat index, and re-embedding every existing chunk.

Add a provider: append to SUPPORTED_EMBEDDING_PROVIDERS and add a case to the switch. The mock branch is the reference for what a new entry needs (configured-check, model resolver).

Tune chunk size / overlap: edit CHUNK_TOKENS and CHUNK_OVERLAP_TOKENS in src/types/knowledge-base.ts. Re-embedding existing documents is delete + re-upload — there is no edit-and-re-embed path.

---

What It Doesn’t Do

• No OCR — image-only PDFs fail ingestion with No extractable text found.
• No hybrid (BM25 + vector) search — pure vector top-K only.
• No re-ranking pass over the top-K result.
• No per-document ACLs inside an org — every member can search every document the org uploaded.
• No web URL ingestion — file upload only (PDF, DOCX, MD, TXT).
• No multi-file batch upload — one file per dialog.
• No edit-in-place — replace = delete + re-upload.

---

Related Documentation

• LLM Providers — OpenAI key setup and rotation
• Inngest — Background worker for document ingestion
• AI Memory & Context — Cross-session memory on the same pgvector retrieval stack
• Conversation Memory — Persistent chat threads with full resume
• AI Development Framework — How the assistant and its tools fit together