30 June 2025 / Applied AI / 8 chapters

What each record needs to carry

From Long-term AI memory: records, relationships and retrieval

A durable memory record needs fields that say what it means, where it came from, when it applied and whether you can still use it. A chunk of text and an embedding don't give you any of that on their own.

Here's what I'd want on a record:

  • memory_id and a stable type
  • a structured subject and property where it makes sense
  • the value or a short statement
  • the source object and where in it the claim came from
  • the author or system identity behind the source
  • recorded time and effective time
  • validity status and an optional expiry
  • confidence or extraction status
  • sensitivity and access scope
  • the record that superseded it, if any
  • the rule or process that created it

Keep effective time separate from recorded time. A document can be uploaded well after the decision in it took effect. And a roster change that starts next month can be recorded today, but you don't want it coming back as current until its start date.

Provenance has to be precise enough that you can actually reopen the evidence. A document ID with no page, heading or paragraph makes review slow. A transcript reference should point at the message or a bounded span. If the source can change, keep a version or checksum so the remembered claim stays tied to the material someone actually looked at.

Authority should come from the source and the record type. Don't take it from confidence language the model generated. An approved project decision can outrank a suggestion someone made in chat, and a field in the system of record can outrank a summary copied into meeting notes. Write those rules down in code so retrieval resolves conflicts the same way every time.

Memory text can still hold uncertainty. "The preferred date may be Friday" shouldn't get normalised into a confirmed Friday deadline. I'd store an assertion class (confirmed, proposed, inferred or disputed) and make sure the retrieval interface shows it.

Sensitive records need thinking about at the field level. Having access to a project shouldn't automatically give you access to personal notes about someone who works on it. Apply the requesting user's access before ranking anything, and keep restricted values out of search indexes that a broader service can look into.

All articles