Inference is compute. If you rent an EC2 instance you pay by the hour and it has a fixed cost at the end. It's up to you to use it to the best of your abilities. Think about this is a lever you can use to say how much more concurrency you want.

We are all running lots of sessions at this point where we don't look at everything that is happening every second and if something takes an hour or an hour and 30 minutes, there is no real change. Agents can wait.

Our value added will come from more than just inference. Stay tune for more :)