I started this project as a need to have a stable state for my AI before any chat memory systems were released back in March or so. I wanted a system that could remember things persistently throughout the session. I developed a method for maintaining state by modifying how the system prompt was used and leveraged KV cache to cache mass quantities of tokenized state. This method injects the snapshot of the entire state that's saved into the llm before every response so the llm always knows what the current context is. I also wipe the context window and reinject everything, though only the deltas and the currently saved verbatim chat pairs are saved.

So while building this system I had limitations like having no GPU and only edge CPU only devices. This lead me to strive for efficiency in every way including using basic IPC and removing all server and API layers.

Currently this runs on PC, Mac, and Linux. I have taken a small break from work to manage life but really would like some feedback and questions about the system. There are white papers in the repo that are the initial design. Also this runs just about every model though I need to see if the role display was fixed by my friend as the main branch has an issue with hard coded roles. Gemma and OpenAI OSS work fine but other models don't display with the proper formatting of the displayed roles. This doesn't break the function just the visual so it's not something I immediately focused on.

There is a video tutorial and dev notes YouTube video, it's about 30min but it goes over the process and future of this tool. https://youtu.be/t3TozyYGNTg?si=amwuXg4EWkfJ_oBL

Note: I am not a strong coder at all and have primarily used jules coding agent from Google to create most of the actual syntax.

https://github.com/bsides230/LYRN