AI Memory & Context

The AI assistant ships with two features that make long-running conversations work: cross-session memory (durable facts about each user, recalled into every chat) and conversation compaction (long threads fold older turns into a running summary instead of hitting the provider’s context limit). Both landed in v0.7.0 and run on the pgvector stack that powers RAG. This page is for developers who want to understand, configure, or extend them.

---

What it does

Cross-session memory is a per-(organization, user) semantic store. After each substantive turn, durable facts (“prefers TypeScript”, “ships on Fridays”) are extracted and saved to the ai_memories table (pgvector + an ivfflat cosine index). On every subsequent turn the assistant embeds the latest user message, runs a cosine search, and injects the top 5 matches into the system prompt as delimited, non-instruction reference context — so a stored note can’t hijack the model. Memory is scoped to both org and user: facts are personal even to org owners, and content never appears in logs.

Consolidation keeps the store clean, mem0-style. A brand-new fact with no similar neighbor is ADDed deterministically. A fact that resembles existing ones goes through one cheap-model pass that returns ADD, UPDATE, DELETE, or NOOP — merging paraphrases and superseding contradictions (“my favorite color is blue now” replaces the old fact). The model may only reference memory ids it was shown, and there’s a hard 500-memories-per-user cap.

Conversation compaction changes only what the model sees, never the visible transcript. When estimated input tokens cross the trigger ratio of the model’s context window, older turns fold into a running summary on Conversation.summary (cheapest configured model, text-only prompt) while the most recent turns — roughly the last 20 messages, snapped to a user boundary — stay verbatim. Nothing is deleted. A context-usage meter in the assistant header and an “earlier messages summarized” divider in the transcript surface what happened.

---

How to use it

In chat, two tools cover memory writes:

Tool	Trigger	Approval
saveMemory	”remember that I deploy on Fridays”	None — saves immediately
forgetMemory	”forget my deployment preference”	HITL — Approve/Deny card

To change a fact, just state the new one — consolidation supersedes the old one automatically. There’s no recall tool; recall is automatic injection on every turn.

Compaction needs no action. As a thread grows, watch the header gauge drop after each fold and the divider appear where earlier messages were summarized.

---

How to configure or enable

Memory and recall require embeddings configured — the same key RAG uses. Set OPENAI_API_KEY, or use an OpenAI LLM_API_KEY. The Get Started checklist has a “RAG & AI Memory” item backed by the same isEmbeddingConfigured() gate the feature runs behind. Without a key, memory shows a clean disabled state (recall and extraction simply no-op — never an error). Compaction needs no embeddings; it works with any configured chat LLM.

Per user, Preferences > AI Memory is a self-serve manager: search, pagination, inline edit (re-embeds the fact on save), per-row delete, clear-all, and an on/off toggle (User.preferences.aiMemoryEnabled, default on). Turning it off retains stored facts but stops recall and extraction. Every user-initiated mutation writes an audit row.

Tuning constants (recall K, the trigger ratio, the per-user cap, similarity thresholds) live in src/types/memory.ts and src/types/conversation.ts.

---

Key files

File	Purpose
src/services/memory-service.ts	Recall, extraction dispatch, consolidation, CRUD
src/services/conversation-service.ts	assembleContextForModel — the recall + compaction seam
src/lib/context-usage.ts	Header meter vs. transcript-divider state
src/types/memory.ts, src/types/conversation.ts	Tuning constants
src/components/preferences/	AI Memory manager UI

assembleContextForModel() is called by the chat route before streaming — it’s the single place to look when memory or compaction behaves unexpectedly.

---

How to extend

• Tune recall or window size: adjust MEMORY_RECALL_TOP_K, CONTEXT_WINDOW_TURNS, or the trigger ratio in the type files above — no service changes needed.
• Change extraction triggers: MemoryService.requestExtraction owns the dispatch (a trivial-turn gate, then the Inngest extractMemoriesFn job with an inline fallback). Edit the gate or the extraction prompt there.
• Add a memory-aware tool: follow saveMemory / forgetMemory in the tools file. Gate destructive tools with needsApproval — the approval card is the confirmation, so never prompt the model to confirm in chat first.

For the retrieval mechanics shared with documents, see RAG & Knowledge Base. For embedding-provider setup, see LLM Providers.