Patrick John Kelly

The Case for Open AI Agent Memory

I’m obsessed with the idea of a portable, platform-agnostic AI agent memory solution.

If agent quality is determined by context quality, we should have full control and ownership over our memory. Right now, your agent’s memory is locked inside whatever platform you’re using. Switch models, switch providers, and you start from zero. That’s a problem.

Here’s what I think this should look like:

The basic premise

A third-party open-source platform your agent can outsource its memory to via a simple API. Your agent stays smart. It just doesn’t need to worry about how memory is stored.

Retrieving memories

Your agent sends a search query with terms and keywords. The memory platform searches a structured knowledge graph using hybrid semantic and keyword search, and returns the relevant memories.

There are two options for how this works. The agent can receive the raw list of results and parse them itself, or the memory platform can run a final AI filter, returning a clean, minified response with only the data you actually need.

Storing and updating memory

Your agent sends arbitrary JSON data. The memory platform does all the complex work - finding related memories, determining if and how they should be updated or added to, tagging and organizing them, and tracking their conceptual relationships.

Your agent’s memory-related responsibilities are now limited to two things: knowing when it needs context, and knowing when something meaningful happened that could be worth saving.

Why this matters

Portability. You now have the freedom to switch between AI models and agent providers without losing your memory. At any time, you can download and export your full memory. No lock-in of any kind.

Quality. A dedicated memory system can be optimized specifically for storage and retrieval, rather than being an afterthought bolted onto a chat interface.

Control. An admin interface lets you visually manage your memory, use version control to roll back changes, and run built-in evals to quantify improvements on specific runs.


This is one of the ideas that eventually led me to build Curator - a context engineering platform that takes this concept further by not just storing memories, but actively auditing, optimizing, and routing business knowledge to AI agents.

The core insight remains the same: if your agent’s output quality is bounded by its context quality, then context deserves its own infrastructure.