I have a feel for how long a query takes when it runs on a local model on my laptop but I wanted to understand if we are all running these queries concurrently in the cloud, how much compute is that, how many GPUs, how much land.
All the sources, confidence ranges and assumptions are listed (for inference only) on the page and on GitHub. Feedback, corrections, collaboration is really welcome!