04 / 10 · SYSTEM DESIGN

Memory: design what to retain and what to forget

Separate conversation storage, context, summaries, retrieval indexes, and caches.

Content reviewed:

Design question

How can past information help without ignoring corrections and deletion?

First read the whole diagram. Expand “Read at full size” to inspect the labels and scroll horizontally, then trace the failure path.

Memory is more than one store

AI memory can mean stored conversations, the current model context, extracted preferences, a retrieval index, or a cache. Mixing these responsibilities makes stale answers and resurrected deletions difficult to explain. This chapter presents an educational reference architecture, not Kumyu production behavior.

Choose an authority first. Here, conversation records and explicit user settings are authoritative; summaries and embeddings are rebuildable derivatives. A model context is one inference input, not the durable database. A vector index finds semantically similar candidates; it does not establish ownership or truth.

Flow: assemble context from authoritative records

Conversations and settings feed summaries and indexes; authorization and version checks precede inclusion in model context
Read at full size (scroll horizontally)
Conversations and settings feed summaries and indexes; authorization and version checks precede inclusion in model context
Figure 1: Authority, derivatives, and context Open full-size diagram ↗

Figure 1 is an original conceptual diagram. Arrows show how authoritative records feed derivatives and model input, and how deletion propagates. Resolve user, conversation, and purpose before retrieving. Recheck source ownership and deletion status after retrieval. A missing search filter must not expose another user's data.

Context is finite. Prioritize recent messages, applicable explicit settings, and summaries with provenance. Semantic relevance cannot override a correction. RAG retrieves selected material instead of placing everything in context. Record selected source versions with the run so an answer can be investigated later.

State and schema: summaries need provenance

A job fixes the source version, generates a summary, and publishes only if the current version still matches
Read at full size (scroll horizontally)
A job fixes the source version, generates a summary, and publishes only if the current version still matches
Figure 2: Publishing a summary Open full-size diagram ↗

An edit can occur between reading source text and publishing its summary. Do not hold a database transaction open during a long model call. Capture a version, generate, then conditionally publish against the current version. Discard stale output and rebuild if necessary.

{
  "memory_id": "memory-example",
  "owner_id": "user-example",
  "source_version": 12,
  "source_range": [1, 80],
  "summary_version": 3,
  "expires_at": "2026-11-23T00:00:00Z",
  "state": "active"
}

This unexecuted example records which messages and source revision produced a summary. Summarization loses information; a summary is not interchangeable with original evidence. Distinguish explicit preferences from inferred candidates and define how conflicts resolve. This design gives applicable explicit instructions precedence.

Concurrency behavior depends on isolation. Consult PostgreSQL transaction isolation and select update rules that preserve the application invariant. Here, UPDATE ... WHERE version = expected_version detects stale writes through its affected row count. Do not overwrite using an old read without checking the version.

Failure: deletion races regeneration

Deleted source facts remain in summaries and prompt caches until derived copies are invalidated
Read at full size (scroll horizontally)
Deleted source facts remain in summaries and prompt caches until derived copies are invalidated
Figure 3: Propagating deletion Open full-size diagram ↗

First commit the deletion state in the authority. Until physical cleanup finishes, reads exclude candidates whose source is deleted. Dispatch deletion to asynchronous processing, remove index entries, summaries, and caches, and track each cleanup destination for retry.

A job started before deletion is especially dangerous. When it completes, changed source or deletion generations must prevent publication. Cache keys include user identity and source version; a cache hit must not replace authorization.

TTL defines retention and does not replace explicit deletion. Check expiry during reads as well as cleanup so a delayed sweeper cannot make expired content usable. Backups, audit records, and external copies have separate retention contracts. Track them at a level that supports a concrete explanation of what disappears and when. This is a design policy, not a claim about legal requirements.

Tradeoffs

Representation Useful for Design concern
Original records Corrections, audit, rebuilding Retrieval volume and retention
Summary Compressing context Information loss and update lag
Vector index Similarity search Permission and freshness checks after retrieval
Cache Reducing repeated reads Stale values and cross-user exposure

Worked design: remembering writing preferences

A user says “keep explanations short,” then clarifies “research articles should be detailed.” Store scoped preferences rather than overwriting a single preference line. Separate scope=research and scope=default, giving the narrower applicable scope precedence. A model's inferred “always prefers short text” remains a candidate pending confirmation.

Evaluate summaries for preserved negations, corrections, numbers, and unresolved questions, rather than compression alone. Keep links back to source records for recovery. Different derivatives can tolerate different consistency delays, but permissions and deletion are checked immediately before releasing content.

Exercise and suggested answer

A user deletes a conversation while a summary job is running. The job finishes and finds no summary with that ID. May it insert a new one?

No. Distinguish never-created absence from deletion using a tombstone or generation. Discard output derived from an invalid source version. Authoritative deletion, derivative cleanup, and invalidation of in-flight generation together close the resurrection path.

What to take away

Distinguish authoritative records from derived data and propagate versions and deletion.

Sources & further reading

  1. PostgreSQL 18: Transaction isolationContent reviewed: 2026-10-23
  2. PostgreSQL 18: ConstraintsContent reviewed: 2026-10-23

These are conceptual reference architectures for learning and design discussions. They do not describe the deployed Kumyu infrastructure.

← Browse all chapters