>example is a text classification task instead of a decision
It's so easy that I question why I would ever pay for JEV when eventually I'll have done enough random things that I will also have a large corpus and likely a general model as well.
It's a fast classifier you can use out-the-box, ~1.5bn tokens is about $40 (I've been hammering it)
It just works ... a whole bunch of low-level/low-importance workflow stuff that was getting farmed out to small/fast LLM models now has a competitive alternative ... and bits that hadn't even been considered to go into some external descision/classifier service can be tested/deployed at ~$0.00003/req
I don't get this wall of negativity on it, it's genuinely innovative/useful tech ... would expect HN to be more positive, regardless of whether it's the absolute best execution
decision model = classifier
system one model = small non-reasoning LLM
noul = boolean
confidence = f(probabilities)
It's sad to see how gullible engineers are today.
I’m sure this has been a gradual and long decline. Maybe it even started with the dot com boom and accelerated with crypto. With AI it seems to have got worse.
Rank System Score Public / sealed accuracy Evidence
1 decider-4b v2 64.13 83.5% / 34.7% Evaluator-run, offline
2 Jev 1.13 63.29 86.6% / 36.7% Evaluator-run API
3 JevK5 v0.2 62.04 85.3% / 33.1% Evaluator-run
4 Cygnet 12B 61.76 87.9% / 33.8% Evaluator-run, offline
5 Hopper 59.43 82.3% / 34.1% Evaluator-run
28 Kev 4B 36.14 66.2% / 22.4% Evaluator-run
41 Laya 421M 30.25 58.4% / 30.8% Evaluator-run
datadrivenangel•1h ago
Smarter move if you have an eval set is to just train a classifier and call it a day.
rgbrgb•59m ago
top open one is trained by perplexity cto for $3k, kinda cool https://x.com/denisyarats/status/2102252088067850507
physicallyIllfr•30m ago
Bro is writing off the H200 lol
On a sidenote I really can't stand the term "swarm" and definately plays into AI doomerism.
cobanov•44m ago