It is counterproductive to force cache on all tools of an agent
1•istvan_intrnd•2mo ago
To make agentic cache work, LLM providers force frameworks to add all tools into the cached prompt. This is the main reason why I cannot properly use cache in my own agentic framework, where the set of tools changes in function of objectives. It confuses most base models, and however many times they say it doesn't my observations contradict this. How about yours?
Comments
sshadmand•1mo ago
Don't they have to cache in order to access the tools quickly enough to run the agents? I imagine not doing slow would slow down the progress considerably. Or, maybe I misunderstand which part of the flow/agent/work you mean by "cache on tools"
sshadmand•1mo ago