One of the biggest issues I've had building with AI is giving it long-term memory that works across any knowledge domain and is simple enough for the agent to use without mistakes.
So I decided to solve this problem, and I'm putting it out there for anyone to use.
Why should you care?
If you are building agents, creating smart workflows, or need to store a lot of data for your AI, keep reading.
What is Deep Memory:
Deep Memory is an implementation of GraphRAG. If you are not familiar with Graph, it's a storage technology that uses vertices and edges (or Entities and Relationships if you like).
But, this one is a little different. I found that AI needs the proper guidance to work effectively with external systems. Heck that's why Anthropic created the MCP (Model Context Protocol) in the first place, to allow an AI to understand the tools.
So I extended this concept to the database itself. This one has a Vocabulary. It's kind of like giving a dictionary that explains the nodes and edges to the AI before it's done a single query. What this means is it knows all the entity types and relationships straight off the bat. In addition the system contains some instructions helping it to query the database efficiently. We call this progressive disclosure.
As you've probably encountered AI has a limited context window. We can't just give it everything. What we want to do, is help it get to just the context it needs quickly. Deep Memory does this by providing the Vocab when the AI opens a memory graph repository, and in usage instructions that helps it get started.
What can you do with Deep Memory?
1. You can connect it to your AI using the MCP tools available in NPM, such as Claude Desktop.
2. You can run it directly in a clone of the repository, and index your own content if you want to.
3. You can export/import your deep memory repositories as .dkg files, meaning they are very portable.
4. You can bypass the MCP tools completely and integrate the deep memory library directly in your own app or system. Anything that can support npm packages.
Here's some examples of what I've built with Deep Memory with outstanding results IMO:
1. I indexed council planning rules from PDF files and created an Expert AI Agent using a deep memory graph - it became an expert council planning advisor telling me exactly what permits I need, what rules apply.
2. A legal expert - I indexed a set of Legal Documents, building a Legal Memory graph. I then used this to help me refine, strengthen and improve legal documents. The AI was able to quickly see variations and very relevant context to aid the drafting of new clauses.
3. A mining equipment expert - By downloading the Caterpillar and Komatsu Equipment technical specs and manuals from the manufacturer websites, I created an expert in maintenance and equipment specs, compatibility for parts, procedures etc.
4. I fully implemented Deep Memory into our flagship AI Productivity platform at https://ucm.utaba.com.au. Along with a graph viewer, and the built in Agent system, I was able to get the agents up and running in no time flat! (quick plug, reach out if you want to work with me)
Looking for feedback on this. What domain would you point this at? The council, legal, and mining ones surprised me with how well they worked — I'm curious where else this lands.
tjwheeler•3h ago
So I decided to solve this problem, and I'm putting it out there for anyone to use.
Why should you care? If you are building agents, creating smart workflows, or need to store a lot of data for your AI, keep reading.
What is Deep Memory: Deep Memory is an implementation of GraphRAG. If you are not familiar with Graph, it's a storage technology that uses vertices and edges (or Entities and Relationships if you like).
But, this one is a little different. I found that AI needs the proper guidance to work effectively with external systems. Heck that's why Anthropic created the MCP (Model Context Protocol) in the first place, to allow an AI to understand the tools.
So I extended this concept to the database itself. This one has a Vocabulary. It's kind of like giving a dictionary that explains the nodes and edges to the AI before it's done a single query. What this means is it knows all the entity types and relationships straight off the bat. In addition the system contains some instructions helping it to query the database efficiently. We call this progressive disclosure.
As you've probably encountered AI has a limited context window. We can't just give it everything. What we want to do, is help it get to just the context it needs quickly. Deep Memory does this by providing the Vocab when the AI opens a memory graph repository, and in usage instructions that helps it get started.
What can you do with Deep Memory?
1. You can connect it to your AI using the MCP tools available in NPM, such as Claude Desktop.
2. You can run it directly in a clone of the repository, and index your own content if you want to.
3. You can export/import your deep memory repositories as .dkg files, meaning they are very portable.
4. You can bypass the MCP tools completely and integrate the deep memory library directly in your own app or system. Anything that can support npm packages.
Here's some examples of what I've built with Deep Memory with outstanding results IMO:
1. I indexed council planning rules from PDF files and created an Expert AI Agent using a deep memory graph - it became an expert council planning advisor telling me exactly what permits I need, what rules apply.
2. A legal expert - I indexed a set of Legal Documents, building a Legal Memory graph. I then used this to help me refine, strengthen and improve legal documents. The AI was able to quickly see variations and very relevant context to aid the drafting of new clauses.
3. A mining equipment expert - By downloading the Caterpillar and Komatsu Equipment technical specs and manuals from the manufacturer websites, I created an expert in maintenance and equipment specs, compatibility for parts, procedures etc.
4. I fully implemented Deep Memory into our flagship AI Productivity platform at https://ucm.utaba.com.au. Along with a graph viewer, and the built in Agent system, I was able to get the agents up and running in no time flat! (quick plug, reach out if you want to work with me)
Looking for feedback on this. What domain would you point this at? The council, legal, and mining ones surprised me with how well they worked — I'm curious where else this lands.
https://github.com/TjWheeler/deep-memory