One of two ways I think this can be accomplished:

1. The entire context of all your conversations is stored in some highly compressed form, basically summarized conversations that get appended and edited constantly

2. There is some kind of small summary for each conversation and the LLM can query any other conversation to read the contents, kind of like agentic mode in Cursor that can query any files in your directory

This seems useful. I don't think it's any less respectful of privacy since OAI already has all these conversations, just doesn't let your LLM access them in your current chat