The coding and math tabs seem to be missing the latest models...

Is there a cheaper model than Gemini 3.8 Flash (High) that maybe/kind-of is on-par with it? For me it works really good but hit the limit in two hours tops... last week was the first time I hit the weekly limit and had to wait 4 days... Claude patches OK, but that is also getting drained really fast these days...

1. In real life, most of us use token packages like OpenCode Go etc.

It would be handy to have a site like this one that takes into account the various deals and attempts to calculate the number of tokens per monthly fee for a chosen model. I realize this makes the task a lot more difficult.

2. It would be handy to have a chart like that for the AI hardware that people own. It helps you decide which model to run (resulting in different levels of intelligence and speed). Also difficult to please everyone (preprocessing vs token generation for example) and to keep updated!

I found https://llm-list.com/ yesterday and when I had a detailed look, I quickly found outdated entries, for example looking at GLM 5.3 flash it listed several providers as "free" that weren't free any longer.

If you maintain this over time, maybe include other sources than just AA and update the design to look less like zero-shot claude styling (I know that font! I know that color! Lol) it's genuinely useful :)

This definition of cost is not particularly useful. You want cost per task, not cost per 1m tokens. Artificial Analysis does a good job of this.

I'm sure that local LLM will be far cheaper

all I ever wanted is an updated website where I can see the best models I can run on my different devices locally, I don't get why people are throwing money at these companies

It's missing Opus 5.5 which was released over a day ago (and also is clearly on the pareto frontier).

As the time goes on it only becomes harder to differentiate between model capabilities with just one or two numbers. I would love to see some kind of multi-axis placement of all the models on less objective attributes, like wordiness, willingness to give up, an ability to "think ahead" and pre-solve possible problems in code, for example, that I didn't think of or didn't think of talking about, etc etc etc.

For example I've been really enjoying Deepseek v4.1 Flash, it's very "straightforward" to the point of being almost dumb sometimes, but it's absolutely relentless and would solve almost any problem no matter how inefficient the solution is.

No idea how to measure all that, just average CoT length per task is probably a good approximation for some things, but not others.

Does anyone actually pay API costs out of their own pocket? It's about 10x cheaper to just get a codex or chat gpt subscription, it's so heavily subsidized compared to the API that I'm sure it would be cheaper to use frontier models on a subscription plan rather than paying API prices for deepseek flash.

You're better off going directly to artificial analysis, this is a feature-poor/misleading/outdated repackaging

Ex. this type of price estimation is quite naive - some models can require 2-3x the number of tokens to achieve the same level of intelligence. Artificial Analysis' own cost per task is a more fair estimation of cost.

If you have a 24-64GB mac, consider running Qwen3.8 27B locally at night. It's a bit slower to run locally, but if you're sleeping it's less of a problem.

Depending on your memory, you'll need to use the weaker Q4 versions but they still perform well.

It ranks higher than GPT-5.3 Codex (xhigh) or Claude Opus 4.6 (max) so is great for pairing with https://github.com/kunchenguid/gnhf for nightly experimentation, cleanup, or recommendation lists for in the morning.

Coding and math graphs are very interesting. Extremely cheap models make it into the upper echelon, delivering 90% of the performance for 1% of the price compared to the #1.

I find it interesting that with the given metric comparison, for coding at min 50 strength, every frontier model brand is from a distinct vendor: Ling, Qwen, Gemini, Muse, Grok, GPT, and Claude in increasing value.

Cost per token is an extremely naive way to index cost, renders this chart essentially meaningless.

I'd really like something that's more oriented around subscription fees.

If I want to spend $100 on LLMs next month, what should I do? Get Claude because Opus 5.5/Fable 5.1 are scoring well? Get Grok because 4.7 is supposedly a good mix of competence and cost? Try out a Chinese model? Don't do a subscription at all like this site is saying?

The page looks Claude generated and ranks Opus at the top.

Fun, but this seems to assume all LLM run on SAAS subscriptions. I would like to compare this to local LLM costs by converting my usage load x hardware costs into a token price. In addition if it games the LLM so often, these eventually are optimized and basically cheat on the score.

Astra worse than 5.6 Sol is crazy work