TopolearnWithout deliberate memory an agent forgets everything between sessions and loses track during long tasks.
Models are stateless, so any "memory" is built by the application. It helps to separate two kinds.
Short-term memory is the current context window: the conversation and the tool results of this session. It is fast and automatic, but limited in size and gone when the session ends. Long sessions need trimming: dropping old tool results or replacing earlier turns with a summary.
Long-term memory is information saved outside the model, in files, a database or a vector store, and loaded back when relevant. Typical contents:
- Facts and preferences: "This user prefers metric units", "Our fiscal year starts in April".
- Progress notes: a to-do list or notes file an agent updates so a later session can continue.
- Past episodes: summaries of earlier conversations or solved problems.
How long-term memory gets back into context:
- Always load a small core, such as a profile, into the system prompt.
- Retrieve on demand, exactly like RAG: embed the memories and fetch those similar to the current task.
- Let the agent manage it with tools: give it tools to read and write memory files and let it decide. Anthropic's memory tool works this way, with the files kept in your own storage; ChatGPT and Claude apps offer user-facing memory features too.
Pitfalls to design around: stale facts that were true once, wrong things saved and then trusted forever, and private data stored without the user knowing. Give memories dates and sources, let users see and delete them, and never store secrets. Also treat stored memory as untrusted input when it may contain text from outside, a point the security lessons return to.
Example
import json, pathlib, datetime
MEM = pathlib.Path("memory.json") # long-term store (a file here; often a database)
def remember(fact: str) -> None:
items = json.loads(MEM.read_text()) if MEM.exists() else []
items.append({"fact": fact, "saved": datetime.date.today().isoformat()})
MEM.write_text(json.dumps(items, indent=2))
def recall() -> str:
items = json.loads(MEM.read_text()) if MEM.exists() else []
return "\n".join(f"- {m['fact']} (saved {m['saved']})" for m in items[-20:])
remember("User prefers reports as a one-page PDF")
system_prompt = "You are a reporting assistant.\nKnown about this user:\n" + recall()
print(system_prompt) # short-term: this text now sits in the context windowRemember
- Short-term memory = the context window; long-term = stored outside and reloaded
- Load long-term memory always (small core), by retrieval, or via memory tools
- Date memories, let users see and delete them, and never store secrets