There seems to be a need to be some kind of hierarchy to contextualization brought to LLMs and not a flat history stuffed into the context. For comparison, human memory is not just a compressing of linear history of which we rewind the tape when needed. Its more like a Russian doll of bounded contexts which we break out of, when our inner context is no longer sufficient to solve the problem.