It's easy to scrape the pricing; you have to do some leg work to estimate the token efficiency, which factors into the ultimate cost. Ideally you'd compare reasoning efficiency too; given them all the same task.

How do you calculate the effect of caching?

Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?