As Simon Williamson said, an LLM agent is something that runs tools in a loop to achieve a goal. https://simonwillison.net/2025/Sep/18/agents/

I might be overthinking this, but:

For a coding agent like Devin, the output of a tool use is usually in the system environment like the terminal state.

When assigned a task, Devin would (probably) first pull the github repo onto a ephemeral firecracker-like container instance that is hardened and isolated. It would then connect to and call an LLM API service with context of (user query, terminal state, existing code) and execute tool-calls recommended by the LLM.

It'd continue doing so in a loop until it's goal (user query) is reached.

Now, how are they storing effective 'agent state' to fight against container reliability concerns. I've always read that stateful API services are a bad idea.

In a way the sandbox service is also an API, but if user sessions are mapped 1:1 to sandboxes, how do you achieve statelessness in the compute layer without storing or caching entire system states somewhere?