Having run a few head-to-heads vs Jev, both perform fairly similar, but in instances of uncertainty, Mercury Decide has a tendency to provide an incorrect answer with a fairly high confidence value, where as Jev ranks very low. Latency has a very wide delta too, whereas Jev basically never exceeds 600ms, Mercury Decide quite frequently goes beyond 1.3s growing with query size far more aggressively. Remains to be seen whether both improve with optimisations/a commercial release. Neither is massively better in my testing for less opaque queries and pricing for Mercury Decide without training isn't yet known (though given Mercury 2.5, could see it being much more competitive than e.g. Clef which pricing wise is in another dimension vs Jev).
Topfi•1h ago
https://openrouter.ai/inception/mercury-decide:free
Having run a few head-to-heads vs Jev, both perform fairly similar, but in instances of uncertainty, Mercury Decide has a tendency to provide an incorrect answer with a fairly high confidence value, where as Jev ranks very low. Latency has a very wide delta too, whereas Jev basically never exceeds 600ms, Mercury Decide quite frequently goes beyond 1.3s growing with query size far more aggressively. Remains to be seen whether both improve with optimisations/a commercial release. Neither is massively better in my testing for less opaque queries and pricing for Mercury Decide without training isn't yet known (though given Mercury 2.5, could see it being much more competitive than e.g. Clef which pricing wise is in another dimension vs Jev).