Updated October 21, 2026: the first version had a benchmark table comparing local and cloud memory that cannot be reproduced from anything published, so it is gone; the numbers left are ones you can check. OpenMemory has been discontinued and Letta and Zep have changed since April, so those parts are updated too.
The default for AI tooling is cloud-first. APIs, hosted services, "we manage the infrastructure for you". For most things this is the right default — model inference is genuinely hard to run locally at the quality cloud providers offer.
AI memory is the exception.
For AI coding agents specifically — Claude Code, Codex, Cursor, OpenCode, Gemini in agent mode — local memory wins decisively. Not because cloud is bad, but because the workload pattern of dev memory is uniquely well-suited to local inference and uniquely poorly-suited to cloud APIs.
This post is the case for that. For a feature-by-feature view of the servers, see the MCP memory servers comparison or the comparison page.
Three taxes that cloud memory imposes
Every cloud memory API charges you three taxes you cannot opt out of:
1. The latency tax
A cloud memory call is at least one network round trip: your query goes out to be embedded and matched, and the results come back. Every recall pays it, however fast the service is behind it.
A local memory call never leaves the machine. In SynaBun's own memory benchmark (run on an Apple M4, September 2026), a hybrid recall took 1.6 ms at the median over its test corpus, and stayed at 10.5 ms at p95 with 10,000 documents and fresh queries, embedding included.
On a single recall the round trip does not matter much. On a coding session where you call recall over and over, it adds up — and it is the difference between memory that feels invisible and memory that feels like it has a loading state. Interactive flow dies fast when every memory check stalls.
2. The cost tax
Cloud embeddings are cheap but not free: OpenAI's text-embedding-3-small costs $0.02 per 1M tokens, and hosted memory services add plans of their own on top.
For a single developer it is rounding error. For a team it becomes a line item — and the value you get over local memory is small on the dev memory workload.
3. The privacy tax
Cloud memory means your code, project notes, debugging traces, design decisions, and (often) full conversation history get sent to a third party. Even if the provider is reputable:
- It is one more vendor in your data flow
- Compliance reviews get harder
- You depend on the provider's retention policy
- You depend on the provider's existence
For consumer apps this is fine. For dev work at any company that does compliance reviews, it is a meaningful headache.
Local memory makes all three taxes go to zero.
Why dev memory is different from production memory
Cloud-first AI memory is a great fit for production agents — customer support bots, sales copilots, in-app AI assistants. Those workloads have:
- Centralized state (memory belongs to the product, not the user)
- Heavy concurrency (1000s of users querying memory simultaneously)
- Cross-instance consistency requirements
- Compliance + audit needs that benefit from a managed provider
Dev memory is the opposite. Single user. Single device. No concurrency. No multi-instance state. The "memory" is conversational notes about your own codebase, not user data you have a duty of care for.
For that workload, the architecture that wins is also the simplest: a single SQLite file on disk + a small embedding model that runs on CPU. SynaBun ships with exactly that: SQLite with vectors in a BLOB column, keyword search through FTS5, and all-MiniLM-L6-v2 computed locally. Mem0's library gets close once you point it at local models. The hosted products (Mem0 Platform, Letta Cloud, Zep Cloud) run infrastructure this use case does not need.
What the numbers say
The first version of this post had a table of mean latency and recall@5 for five setups. Those runs cannot be reproduced from anything published, so the table is gone. Here is what can be checked:
| What | Value | Source |
|---|---|---|
| Hybrid recall, 59 documents, 23 queries | 1.6 ms p50, 2.4 ms p95 | SynaBun memory benchmark |
| Hybrid recall, 10,000 documents, fresh queries | 10.5 ms p95, embedding included | SynaBun memory benchmark |
| Relevance, hybrid vs vector-only (nDCG@10) | 0.87 vs 0.64 | SynaBun memory benchmark |
OpenAI text-embedding-3-small | $0.02 per 1M tokens, 1536 dimensions | OpenAI |
The SynaBun benchmark (benchmarks/memory-v2.mjs, not yet in the public repository) runs on engineering fixtures, not production traffic. It does not measure hosted services; the point about them is structural — every call crosses the network.
What local-first does NOT mean
Three things people get wrong about "local-first AI memory":
- It does not mean offline-only. Local memory works offline, but it is still files you can back up to git, S3, or wherever you point them.
- It does not mean you give up search quality. Retrieval design matters as much as the model: in SynaBun's benchmark, adding keyword search and passage refinement to the same small model raised relevance from 0.64 to 0.87.
- It does not mean you give up the UI. SynaBun ships a 3D Neural Interface for browsing memory locally. The cloud-only-UI-experience is a choice, not a constraint.
When cloud is the right answer
I am not religious about this. Use cloud memory if:
- You are building a multi-tenant product where memory belongs to the product, not the developer
- Your team has a hard "no SQLite in prod" rule (more common than you would expect)
- You need cross-region replication out of the box
- You want a managed service to handle backups + scale
- You are running on infrastructure that cannot host a small model (rare in 2026)
For everything else, local is the simpler answer.
The setup tax is also lower
People assume "self-hosted" means complex setup. For dev memory specifically, it is the opposite.
SynaBun setup:
npm install -g synabun
synabun
That is it. SQLite is created on first use. The embedding model downloads once and caches. No Postgres. No Qdrant. No Docker compose. The quick start has the details.
Mem0 self-hosted server:
docker compose up -d # starts the Mem0 API + Postgres with pgvector
# configure your model providers in .env
# wire the MCP server
Functional but heavier.
Zep, self-hosted: the Zep Community Edition was discontinued in April 2025; self-hosting now means running Graphiti, Zep's open-source knowledge-graph framework, with a graph database such as Neo4j or FalkorDB.
Where complexity comes from is not "local vs cloud", it is the storage + service architecture each project chose. SynaBun's choice of SQLite is the actual reason its setup is one command.
What I would build if I were starting today
If I were building a new MCP memory server in 2026, I would copy these decisions verbatim:
- SQLite. Zero ops. Backupable. Embeddable. Inspectable. Vectors fit in a BLOB column next to the text.
- all-MiniLM-L6-v2 or another small local model by default. Free, local, fast. Provide an adapter for users who want a hosted one.
- MCP-native, not a wrapper. Be a real MCP server. Implement the tools fully.
- Categorical memory. Vectors alone are too flat. Categories + projects + tags + importance let users actually navigate memory.
- Local UI. Browser-based, served from the same Node process. No required cloud login.
That is roughly the SynaBun architecture.
The category is moving local
Watch the next 12 months. The trend lines are clear:
- Small embedding models keep getting better.
- Local vector search keeps getting faster.
- MCP keeps standardizing. Wrappers around cloud services will lose ground to native MCP servers that treat memory as a first-class concept.
- Privacy-conscious teams are explicitly asking for local-first AI tooling now.
The cloud-first AI memory products are not going away — they will own the production-agent and multi-tenant-app categories. But for the dev workload, local has already won. Most developers just have not switched yet because the default is sticky.
Closing
If you are using a cloud memory API for your AI coding workflow, try a local one for a week. The latency difference is the kind of thing you cannot un-feel. Once memory becomes invisible, going back to a loading state on every recall feels worse than it sounds.
SynaBun is the local-first option I built. Letta and Zep are great if you need their specific features — an agent platform, a knowledge graph. Their hosted services, Letta Cloud and Zep Cloud, add a network round trip to every call; Letta Code also runs locally, and Graphiti, the open-source graph behind Zep, runs on your own graph database. OpenMemory, which used to be the other local option, has been discontinued.
Pick local. Your flow will thank you.
Related reading: