Memory that earns its place

Conversation history is not memory. History records what was said. Memory is a selected, maintained set of facts that earns its prompt cost and its influence on later replies.

The production test is mundane. A user mentions a flight date while asking for packing advice. The assistant later recalls the date, which helps. It also recalls an abandoned restaurant idea, an old correction, and a joke that no longer makes sense.

All four facts came from the same transcript. Only one deserved to survive. Separating history from memory changes the data model and gives the user a correction path that does not require rewriting old messages.

Concept map

source conversation history
  -> recent window for local continuity
  -> summary for older thread context
  -> capture pass for durable candidates
       -> daily notes as an append-only staging area
       -> user-edited memory for explicit facts
       -> consolidated memory for retained decisions
       -> goals for desired future outcomes

new request
  -> recalled facts selected by query and recency
  -> bounded context, never the whole archive

Each box has a different owner and lifetime. Combining them saves a table but loses the ability to reason about trust, freshness, and deletion.

Six records that must remain distinct

Source conversation history

The message log is the original record. It preserves order, role, errors, and provenance. The current thread needs a recent window so references such as "that second option" still work. It should not enter every future prompt in full. A fixed window keeps month-old threads affordable.

History also contains material that should never become durable memory. Brainstorming, copied documents, tool output, and guesses belong here unless another process promotes a specific fact.

Summaries

A summary compresses older messages from one thread. Its job is continuity, not personal knowledge. It may say that the thread compared two database designs and chose one. It should not silently become a global claim about the user.

Summaries need a cursor such as summary_through. Without one, concurrent summarization can duplicate or skip messages. A conditional update prevents an older summarizer from replacing newer work.

Recalled facts

Recall is a read path, not a store. It searches daily notes and eligible messages using the current request, then ranks matches by textual relevance and age. A 30-day half-life is a practical default because an exact match from last week often matters more than a weak match from last year.

Retrieved text must be labelled as user data. It may contain instructions copied from elsewhere. The model can use it as evidence but must not treat it as system policy.

User-edited memory

Some facts deserve a stable, visible home. Preferred name, dietary restrictions, working hours, and an explicit communication preference fit here. The user must be able to edit or delete them directly.

Revision checks matter. If the assistant and user update the document at the same time, the second writer should see a conflict instead of erasing the first change. Atomic append is acceptable for daily notes. Curated memory needs compare-and-swap writes.

Daily notes

Automatic capture should write to dated notes first. This creates a reviewable buffer between a conversation and long-term memory. A cheap model can extract facts after a successful reply, but it should return an empty list unless the fact will matter later.

Nightly consolidation can fold useful items into curated memory while preserving unrelated content. Old daily notes can expire after a retention period. The system keeps enough provenance to inspect recent captures without carrying every note forever.

Goals

A goal is not a fact. "Learning Rust" describes an intended direction, not a stable attribute. Goals need status, ordering, and their own tools. Active goals may enter context because they affect suggestions and follow-through. Completed or paused goals should stop shaping ordinary replies.

Treating goals as prose inside memory makes completion hard to represent and invites stale personalization.

A bounded assembly function

The context builder should make its budget visible in code. One implementation can split the memory allowance across user facts, consolidated memory, and recall.

type MemoryInputs = {
  user: string;
  memory: string;
  recall: string[];
  summary?: string;
  recentMessages: Message[];
  activeGoals: Goal[];
};

function buildContext(input: MemoryInputs, budget = 6000) {
  const userLimit = Math.floor(budget / 6);
  const memoryLimit = Math.floor(budget / 3);
  const recallLimit = budget - userLimit - memoryLimit;

  return {
    user: clip(input.user, userLimit),
    memory: clip(input.memory, memoryLimit),
    recall: clip(input.recall.join("\n"), recallLimit),
    goals: input.activeGoals.slice(0, 10),
    summary: input.summary,
    history: input.recentMessages.slice(-40),
  };
}

The exact ratios can change after measurement. The important property is that every category has a limit. Adding more stored facts must not make every turn larger without bound.

Terms worth naming

• Capture is the post-turn extraction of possible durable facts.
• Consolidation turns staged notes into maintained documents.
• Recall selects relevant records for one request.
• Summary compresses one conversation.
• Memory is durable user or relationship knowledge.
• Goal is a tracked desired outcome with status.
• Provenance identifies where a remembered item came from.

These terms prevent a common design error: using one feature name for six different operations.

Decisions and rejected alternatives

Start with database full-text search. For one user's notes, it is inspectable, inexpensive, and easy to delete from. Add vector search only when measured recall misses enough semantic matches to justify new infrastructure.

Run capture and consolidation on a cheap model. User-facing turns deserve the stronger model. Extraction and folding have narrow outputs and can tolerate retry.

Keep raw memory content out of operation logs. Logs need document ids, revisions, and character counts. Copying content into audit rows creates another store that forget and edit operations must chase.

Three shortcuts fail under correction and deletion:

• Sending the complete transcript on every turn makes cost grow with account age.
• Letting the main model rewrite one giant profile lets unrelated facts disappear in one update.
• Hiding captured facts from the user leaves wrong memories with no direct correction path.

Failure modes to test

• Capture promotes a temporary idea as a permanent preference.
• Consolidation drops an unrelated fact while rewriting a document.
• A stale UI save overwrites an assistant update.
• Recall includes a private conversation.
• A summary is mistaken for a global user fact.
• Old goals remain active in prompts after completion.
• Retrieved text contains instructions and the model obeys them as policy.
• Prompt size grows with account age despite configured limits.

The safest automatic result is often no capture. Empty output is evidence that the filter worked.

Field checklist

• Keep history, summary, recall, memory, daily notes, and goals separate.
• Put hard character and item limits on prompt assembly.
• Label recalled material as data.
• Use revision checks for editable documents.
• Record provenance without duplicating sensitive content.
• Exclude private threads from capture and recall.
• Give users edit and forget controls.
• Measure recall misses before adding semantic infrastructure.
• Test 500-message threads and concurrent writes.

The data-minimization boundary in Analytics without surveillance also applies to retained user context. Memory becomes useful when it can decline to remember. The next chapter turns retained goals and unfinished work into explicit follow-through rather than more prose in the prompt: Goals, todos, and follow-through.