One thing we found interesting is that Clojure's benchmark-wide performance was nuked by Anthropic models, whereas OpenAI models many other frontier lab releases reasoned quite well in Clojure.
The individual language per-model data is noisy, but aggregated per-provider data gives an interesting view of how companies are training their models (and likely distillation vectors). Data at https://gertlabs.com/rankings?mode=agentic_coding
gertlabs•54m ago
The individual language per-model data is noisy, but aggregated per-provider data gives an interesting view of how companies are training their models (and likely distillation vectors). Data at https://gertlabs.com/rankings?mode=agentic_coding