Your Agent Forgot You Again. That Is Not a Bug — It Is the Architecture.
Context windows are not memory and RAG is not memory. How agent memory is really assembled, why recall is the hard half, and what breaks in production.
You spent twenty minutes last Tuesday explaining that your fiscal year starts in April, that the client is "Nordwind" and not "Nordwynd", and that finance wants revenue figures net of refunds. The assistant was excellent about it. It thanked you for the clarification.
This Tuesday you are typing all of it again.
Everyone has hit this, and almost everyone diagnoses it wrong. The model is not being careless and it is not too small. It genuinely never knew you — not for a moment, not even during the conversation where it seemed to. Understanding exactly why is the difference between shopping for a bigger context window, which will not fix it, and shopping for a memory architecture, which will.
The goldfish is behaving exactly as designed
A language model API call is stateless. You send a prompt, you get tokens back, and the moment the response ends the model retains nothing whatsoever. There is no "session" on the model's side to store anything in.
What creates the illusion of memory inside a chat is brute repetition: the client resends the entire conversation with every single turn. Turn forty is not the model recalling turns one through thirty-nine. It is the model reading them again, from scratch, for the fortieth time.
Two consequences fall out of that picture, and they explain most of the frustration.
Nothing crosses a session boundary unless someone wrote it down. Close the tab, and the only record of those twenty minutes is your own recollection of them.
A bigger context window buys you a longer conversation, not a longer relationship. Going from 200k tokens to 2M means you can hold more of this thread. It changes nothing about next Tuesday. People conflate the two constantly, because inside a single long thread the model behaves exactly as a system with memory would.
Four things get called "memory", and they are not interchangeable
This is where most product comparisons fall apart, because the same word is doing four jobs.
The context window is working memory. Fast, complete, expensive per token, and gone at the end of the session. Think of it as the desk: everything you are actively using is on it, and it gets cleared nightly.
Retrieval — RAG — is a library. You index a corpus and fetch passages that match the query. Retrieval is genuinely useful and genuinely not memory, and the tell is the write path: a RAG corpus is authored by someone else and loaded ahead of time. Nothing about your conversation goes into it. RAG can tell your agent what the refund policy says. It cannot tell your agent that you have asked about refunds three times this month and always want the figure net.
Semantic memory is the set of durable facts learned about you and your business — the fiscal year, the spelling of the client's name, the preference for net over gross. Short, structured, high-value notes, written by the system as a by-product of working with you.
Episodic memory is what actually happened: past runs, past conversations, the decision you made in August and the reason you gave. It answers "what did we do last time?", which is a different question from "what is true?"
There is a fifth that rarely makes the list but does much of the heavy lifting: procedural memory — how this job is done, standing instructions, the role the agent occupies before you ask it anything. On our platform that is what a role is, and roles are why a scheduled run at 6am behaves like the same colleague you briefed in June rather than a stranger with your login.
All five converge on the same physical destination. They become tokens in a prompt.
Look at that picture and the product question sharpens considerably. "Does it have memory?" is unanswerable. "Which of those blocks does it assemble, and how does it decide what goes in them?" is a question with a real answer.
Storage is the easy half. Recall is where systems actually fail
Writing notes is a solved problem. Any competent team can append facts to a table. The hard part — the part that determines whether the thing feels like a colleague or a stranger — is getting the right three facts out of nine hundred and into this particular prompt, in the fifty milliseconds before the model call.
There are two schools, and the choice between them shapes the entire product.
Search-tool recall gives the model a search_memory tool and lets it decide when to reach for it. It is elegant, it costs nothing on turns that do not need it, and it has one severe flaw: the model has to suspect it has forgotten something in order to look. It usually does not. You ask for the quarterly summary, the model has no reason to think a preference exists, so it never searches, and it produces a fluent, confident, gross-not-net answer. Nothing errored. Nothing was logged. You just get a colleague with amnesia who is too polite to mention it.
Pre-turn injection flips the responsibility. Before the model is called at all, the runtime queries memory and drops the results into a clearly delimited block in the prompt. The model does not choose; it simply always has the context. This is the approach we took on Burrak: every turn pulls learned facts, relevant excerpts from other sessions, and agent notes concurrently, and folds them into a fenced memory block before the first token is generated.
The costs are real and worth naming. You pay retrieval latency on every turn, including the ones that did not need it. You spend tokens on facts that turn out to be irrelevant. And if your relevance ranking is poor, you have not given the model memory — you have given it distraction. The engineering is all in the ranking, which is why so many systems that technically "have memory" do not feel like it.
Five things that break once real users arrive
The demo always works. Here is what the demo does not tell you.
1. Pure vector search loses exact strings. Embeddings are wonderful at "the thing about invoices being paid late" and unreliable at "invoice INV-2291". Semantic similarity is not string equality, and business memory is full of identifiers, product codes and proper nouns. The fix is a hybrid: run semantic search first, then let keyword matching fill the remainder of the result set. Neither alone is enough, and teams usually discover this the week a customer searches for an order number.
2. Memory has to fail soft, or it becomes your biggest outage. Recall sits in the hot path of every single turn. If the embedding service is slow, or the vector index is rebuilding, or the store is briefly unreachable, the correct behaviour is to degrade to keyword search — or to no memory at all — and let the run finish. The wrong behaviour is to raise. An agent that forgets your preferences produces a mildly worse report; an agent that crashes produces nothing at all. Every retrieval path in our stack is written to swallow its own failures for exactly this reason.
3. Two writers, one store — or you get a split brain. This one cost us real time. Memory can be written from several places: an interactive session on the desktop, a scheduled run on managed infrastructure, a chat on the mobile app. If those paths do not converge on a single store, everything appears to work while quietly forming two separate brains. The symptom is maddening and easy to misdiagnose: your agent remembers the thing you told it on your laptop, but the 6am scheduled version of the same agent does not, and the memory view in the dashboard shows nothing at all from the overnight runs. Nothing is broken. There are simply two stores, and each is faithfully remembering half your life. If you are building this, decide where the single source of truth lives before the second writer exists — the migration afterwards is considerably less fun.
4. The scope key is the account, never the session. Memory that leaks across tenants is not a bug, it is an incident. Every read and every write should carry the owning account as a mandatory filter, enforced in the store rather than remembered by each caller. The related trap is scoping too tightly: key memory to a session and you have reinvented the context window with extra steps.
5. Memory costs money, and the bill is not obvious. Every note written is an embedding call. Every turn recalled may be another. These are model calls, they meter like model calls, and unlike your chat completions they run in the background where nobody is watching. Any platform serious about this should be able to show you that line item — ours prices embeddings as a normal metered call on the same credit system as everything else, precisely so it shows up rather than hiding in overhead.
Forgetting is a feature, not a missing one
Here is the counterintuitive part. A memory system that only ever appends becomes confidently wrong, and it gets worse the longer you use it.
You told it in March that the client contact was Marta. In July, Marta left. If both facts sit in the store with equal standing, retrieval will cheerfully hand the model a March-flavoured answer for the rest of time — and the more you have used the system, the more stale facts it has to trip over. Long-lived memory does not decay gracefully on its own; it silts up.
So a real memory system needs an opinion about at least three things. Recency, so that a later statement outranks an earlier one about the same subject. Correction, so that "actually, it's net of refunds" supersedes rather than sits beside the original. And importance, so that a deliberate standing instruction outranks a passing remark you made once. Superseded notes are better marked than deleted, incidentally — you often want to know what the agent used to believe when you are working out why it did something strange in August.
The uncomfortable implication for builders: you cannot evaluate a memory system on whether it retains things. Retention is trivial. Evaluate it on whether it drops the right things.
Why this is the actual personalisation unlock
Strip away the architecture and the business case is simple arithmetic.
Without memory, every task costs you the same amount of attention forever. The thirtieth weekly report needs the same briefing as the first — the same corrections, the same re-pasted context, the same fifteen minutes. You are not accumulating anything. You are renting a very capable stranger, repeatedly.
With memory, the cost curve bends. The first run is expensive in your time. The fifth needs a correction. By the twentieth you are reading the output rather than steering it, because the standing instructions, the corrections and the house style have all been absorbed. That gap is where the actual return lives, and it is why memory — not model quality, not tool count — is what separates an assistant you use from an agent you delegate to.
It also explains a shift in how these products should be judged. Agents that take real actions on your behalf are only as safe as their grasp of your context; we wrote about the wider gap between chatbots and agents separately, but memory is the load-bearing piece of it. An agent that acts confidently on a stale fact is worse than one that asks.
Be clear-eyed about the trade-off, though: memory is exactly what makes these systems personal, and "personal" means your business's specifics are now sitting in someone's database. You should expect per-account isolation as an architectural guarantee rather than a policy promise, a way to see what has been remembered about you, and a way to delete it. If a vendor cannot show you the contents of your own memory store, that is a product answer as well as a privacy one.
Ten minutes to test any memory system
Ignore the marketing page. Run this.
- Cross the session boundary. State a specific, non-obvious preference. Close everything. Come back tomorrow and ask for related work without restating it. This alone eliminates most tools.
- Cross the surface boundary. Tell it something on the web app; ask on the phone, the desktop app, or the CLI. Half-remembering is the split-brain signature.
- Cross the autonomy boundary. Give it a preference in chat, then let a scheduled run produce the same artefact unattended. Scheduled runs are where memory plumbing is most often not wired up.
- Contradict yourself. Change your mind about something you established earlier. See which version comes back. A system that hands you the March answer in July is appending, not remembering.
- Ask it what it knows. You should be able to read your own memory store, and remove things from it. If the answer is a shrug, the memory is not really yours.
Test 3 is the one worth dwelling on. Memory inside a chat window is table stakes now. Memory that survives into work happening while you are asleep is the part that changes what you can hand over — and it is where the wiring is most often missing, because it never shows up in a demo.
What nobody has solved yet
Being honest about the frontier: conflict resolution is still largely heuristic, and no one has a principled answer for which of two contradictory memories should win. Evaluation is worse — there is no accepted benchmark for "did the right thing get recalled", so most teams are tuning against intuition. Sharing memory across a team without leaking one person's context into another's work is an unsolved product problem as much as a technical one. And deletion is genuinely hard: removing a row is easy, but a fact that has already been embedded, summarised into a note, and folded into a downstream summary is not a single row any more.
Anyone claiming these are handled is selling. The realistic goal today is a system that remembers the things that matter, forgets on purpose, degrades quietly when its dependencies fail, and lets you look inside.
Try it on something with a memory of its own → — free to start, no card. Give an agent a preference on Monday, then check on Tuesday whether you have to say it twice. That is the whole test. If you would rather see it first, the video tutorials walk through setup end to end, and we compared our approach to on-device memory layers like Pieces if you want the contrast.
Tags: AI agent memory, persistent AI memory, long-term memory for LLM agents, RAG vs memory, AI memory architecture, Burrak AI