Inference is then the decompression stage where it generates text from the input prompt and the compressed model.
Now that compressing and decompressing texts is trivial with LLMs, we humans should focus - in business at least - on communicating only the core of what we want to say.
If the argument to get a new keyboard is: "i like it", then this should suffice, for inflated versions of this argument can be trivially generated.
What a horrible technology.
This seems like exactly what LLMs are supposed to be good at, according to you, so why don't they just near-losslessly compress the data first, and then train on that?
Also, if they're so good at this, then why are their answers often long-winded and require so much skimming to get what I want?
I'm skeptical LLMs are accurately described as "near lossless de/compression engines".
If you change the temperature settings, they can get quite creative.
They are their algorithm, run on their inputs, which can be roughly described as a form of compression, but it's unlike the main forms of compression we think of - and it at least appears to have emergent decompression properties we aren't used to.
If you up the lossy-ness on a JPEG, you don't really end up with creative outputs. Maybe you do by coincidence, and maybe you only do with LLMs - but at much higher rates.
Whatever is happening does not seem to be what I think people typically associate with simple de/compression.
Theoretically, you can train an LLM on all of Physics, except a few things, and it could discover the missing pieces through reasoning.
Yeah, maybe a JPEG could, too, but the odds of that seem astronomically lower.
Ah, yes. It is an achievement in signals in a way.
Is this situation in any way realistic one? Because the way companies work in my beck of woods, no one wants your 4 paragraph business case essay about computer. Like, it is funny anecdote.
But, in real world, at least in my experience, pretty much everyone preferred short for emails and messages. They would skim the long ones at best, especially in situation that can be boiled down to "Tom wants a new computer and is verbose about it".
Of course when I went to read them they were 100% slop. The funniest requirement were progress bars for actions that don’t have progress. The tickets were, even if you assume the requirements weren’t slop, at least 15 points a piece.
But ok maybe with all of these new tools we can respond by implementing these insane requirements. The real problem is what this article is discussing. Each ticket was also 500-700 words. Requirements that boil down to a single if statement were described in prose. While this is hilarious the problem is it makes them harder to understand.
I tried to explain this and they just said “ok fine rewrite them then”. Which I did in maybe 15min because there wasn’t actually much to write.
At this point I’m at a loss for how to even work with people that are so convinced these things will save time because they look at the volume of the output.
But project plan dates have always been fiction. Getting there faster is an efficiency win.
That said I’ve found that llms are good as interrogators. If used to guide a conversation, research background information and then be explicitly told to tersely outline the steps in something I’ve had very good results.
And if an LLM is also used at the other endpoint to parse the longer text, that creates a broken telephone. Congrats, your communication channel is now unreliable.
isn't this the opposite? Enabling compression will INCREASE the load on your server as you need more CPU to compress/decompress the data.
The 4 paragraphs requirement was not introduced 'because LLM'. It was there all along for what just should have been 'gimme 2 -3 bullet points'. They wanted Bob to hold back on requesting the new machine he needed, not by denying his request openly, but by making the process convoluted. Now Bob can cut through the BS, they want to blame the LMM for wasting their* time and resources? BS!
I expect smaller models to become incrementally better at compressing what truly matters in terms of information. Books, reports, blog posts… all kinds of long-form content can be synthesized in just a few words or pages. It’s no wonder that even small LLMs can provide accurate results for many queries.
What a depressing belief. Human communication is about a whole lot more than just getting your point across as quickly and efficiently as possible.
This is one example of the "horseless carriage" AI solutions. I've begun questioning further that actually we're going into a generation where a lot of the things we are doing now are not even necessary.
I'll give you one more example. The whole "Office" stack of ["Word", "Excel", "Powerpoint"] can also go away. But we still use it because change is hard.
Answer me this question. In the near future if we could have LLMs that can traverse to massive amount of data why do we need to make excel sheets anymore? Will we as a society continue to make excel spreadsheets because we want the insights the sheet provides or do we make excel sheets to make excel sheets.
The current generation of LLM products I find are horseless carriages. Why would you need agents to make spreadsheets when you should just be able to ask the agent to give you answers you are looking for from the spreadsheet.
> Bob’s manager receives 4 paragraphs of dense prose and realises from the first line that he’s going to have to read the whole thing carefully to work out what he’s being asked for and why. Instead, he copies the email into the LLM.... The 4 paragraphs are summarised as “The sender needs a new computer as his current one is old and slow and makes him unproductive.” The manager approves the request.
"LLM inflation" as a "bad" thing often reflects a "bad" system.
In the case described, the bad system is the expectation that one has to write, or is more likely to obtain a favorable result from writing, a 4 paragraph business case. Since Bob inflates his words to fill 4 paragraphs and the manager deflates them to summarise, it's clear that the 4 paragraph expectation/incentive is the "bad" thing here.
This phenomenon of assigning the cause of "bad" things to LLMs is pretty rife.
In fact, one could say that the LLM is optimizing given the system requirement: it's a lot easier to get around this bad framework.
jasode•1h ago
>Bob’s manager receives 4 paragraphs of dense prose and realises from the first line that he’s going to have to read the whole thing carefully to work out what he’s being asked for and why. Instead, he copies the email into the LLM du jour and types at the start “Please summarise this email for me in one sentence”. The 4 paragraphs are summarised as “The sender needs a new computer as his current one is old and slow and makes him unproductive.”
Sam Altman actually had a concise tweet about this blog's topic (https://x.com/sama/status/1631394688384270336)
>something very strange about people writing bullet points, having ChatGPT expand it to a polite email, sending it, and the sender using ChatGPT to condense it into the key bullet points 2:42 PM · Mar 2, 2023 · 1.2M Views
unglaublich•1h ago
Now that decorating any message with such fluff is automated, we can as well drop the requirement and just state clearly what we want without fluff.
watwut•53m ago