I'm not exactly following through with the claim, can someone explain how the built-in classification would not necessitate more tokens used, or be much different from turning on reasoning? Not that I don't see the difference, I just doing see how OpenAI would do it well.

If Typesafe/Jev has 2-3 years of financial runway, this problem might solve itself.

If OpenAI releases something similar to what Jev does, then that would be like admitting defeat. Their whole spin is AGI and world ending danger. Why would somebody with an AGI at home make something like Jev which is intended to be a part of some SW the AGI is going to replace anyway.

Personally, I don't really care what OpenAI does here. What's going to be far more exciting is when DeepSeek, Qwen, or GLM start integrating classifiers into their open models.

I'm confused. Why OpenAI and not Anthropic? I don't see anything here that is specific to OpenAI.

I'm looking forward to next week when we never have to hear about Jev again.

With all the excitement around Jev, I suspect we'll see hundreds of options, it doesn't sound like Jev is that hard to replicate, given all the 3rd parties who are getting pretty damn close, or even better, results within a week.

vLLM has a PR very close to merging: https://github.com/vllm-project/vllm/pull/57250

Kev is an open Jev: https://github.com/jaredpalmer/kev

maybe open source even eats Jevs lunch first

see here: https://news.ycombinator.com/item?id=49800574

There is one benefit that Jev has, that it is not OpenAI and thus it's probably less likely to steal your own work.

I find it unlikely. OpenAI is all in training models with reasoning with RL, and Jev-like models are the total opposite. They are made to not reason at all to be fast. If you want to add reasoning on top, you might as well use a conventional LLM because you lose the price and speed benefits when you output auto-regressive tokens. I don't think OpenAI will even bother with this.

> My main assumption is that Jev is using something quite close to a conventional large language model. As evidence of this, Latent Space reports that many of the early clones are indeed LLM-based.

Not proof that this is the case with Jev though. It might use non causal text encoder for the state, which could make sense given that its very good for its price.

I just had Claude and Jev combine forces last night. I've built a few personal browser extensions in the past and thought it would be fun to copy an experiment I saw on twitter where Jev classifies comments/posts etc as slop or not.

Fed Claude an api key from typesafe and a link to documentation, and within about 10 minutes I had a view of HN that was populated with a little ranking as to sloppiness of each comment.

When your mind has been wired a bit to LLM latency, it feels extremely fast, and for such a subjective rating I think it did a good job.

Feels like it sits in a space between traditional ML classification and the frontier models. I can't think of a 'real' production use case for it in my sphere of influence, but certainly some will. And of course there will be five Jev competitors by the end of the year.

> Back when I was at GitHub working on Copilot I had the opportunity to work with a very new and very raw internal API for GPT-4. Out of the gate, we knew something was way off because, after an initially very coherent response, the model would have trouble wrapping up. It would end every response with something like "Let me know if you have any other questions. Have a nice day. Have a great week. Have a good time. Have a wonderful life. Have a special day. ..." and it would keep on like this until it hit the response token limit.

I love this!!

Not directly related, but still jev related:

Would it be intesting/useful to use jev to generate a block of text like LLMs do ?

Like asking it to pick the n + 1 word given the starting text (using it's choice primitive), but also asking n + 2,n+3 and so on at the same time.

Would it give coherent or useful results ? Or does the fact that it computes it "all at once" means it cannot make one of it's answer influence the other ones ?

Can't they eat everybody's lunch simply by typing "Astra, please copy this product?"

They certainly have the token budget for it.

I think the article is part right and part wrong.

The right part: autoregressive LLMs are indeed generating “probabilities” (scare quotes very much intentional). During pre-training and any SFT steps, those probabilities are nudged toward the probabilities, over the training distribution, of the next token conditioned on the previous tokens. (This is an explicit property of most training recipes: KL divergence is a “proper scoring function”.)

So if you prompt with “Paris is a city in ”, the next token probabilities estimate the probabilities over the input distribution that the next token in the sentence is the first token of France or of something else.

But there are huge caveats:

1. That is not at all the same thing as the probability that Paris is France under any distribution that you care about (the population of the various Parises, for example).

2. None of this necessarily usefully applies to RL or, as the article discusses, tool calling. The output probability of a tool call is not some Platonic idea of a probability that the input is worthy of a tool call. It’s a the result of a training process that tried to teach the model to be useful and to achieve its goals.

3. I suspect that reasoning makes this all much worse. Suppose that you prompt with “a help desk user with IP=a.b.c.d says they’re ‘in Paris’. What country are they in?” The model has been trained to generate a reasoning trace, which may well start with “let me think of where Paris could be. It could be in France or in Texas etc. The user was speaking English…” See the problem? The model is reasoning well, but it reasoned “France” before “Texas”, so the logprob for France was probably higher than “Texas”. At the end of the reasoning trade there will be an answer, but the logprobs for that answer are, at best, some representation of the probabilities of the answer conditioned on the sampled reasoning trace. And that is not the probability distribution that a Jev user wants.

comments are pretty weird here, there's no real moat to what jev is doing, it is certain that frontier labs are going to release their own jev and there are even open source alternatives (although nowhere near as accurate as jev).

so maybe typesafe's real plan is to front run and releasing their own new models for some time until they can get acquired which seems to be the only rational objective

This article is extraordinarily hard to read. It’s tummelvisioned on OpenAI and things like tool calling which are only relevant to the extent that llms have been tuned to make relative choices, but this applies to all LLMs.

Also, moat discussion is the lowest form of discussion. I don’t care if jev has a moat. Did it get the interface right? What other past ideas have we overlooked that if given some love, could kick the door down like jev did?

Really silly stuff.. people wanting to talk about moats when there’s no castle. Moat talk is all an illusion of being engaged without actually engaging in a way that requires thinking.

gosh, for wanting TypeSafe to survive, this fellow just handed OpenAI detailed instructions and ideas for defeating them...

I think the original idea originates from this author : https://laya.convaiinnovations.com/

Jev certainly feels vulnerable but whether it’s oai or someone else that goes after them seems unclear.

Wouldn’t be surprised if every single AI house spins up a copy

But like they usually also have an embeddings endpoint