What quotes? This is an AI summary that may or may not have summarized actual quotes from the researchers, but I don't see a single quote in this article, or a source.

The point is to look at who is making a claim and asking what they hope to gain from it. This is orthogonal to what the thing is, really. It’s just basic skepticism.

Even if the article is accurate, it still makes sense to question the motives of the publisher. Especially if they’re selling a product.

"This company predicts software development is a dead occupation"

Citation needed?

Closest I've seen to that was Dario saying AI would write 90% of the code, but that's very different from declaring the death of software development as an occupation.

I use Gemini almost obsessively but I don't think feeding the entire corpus of a subject would work great.

The problem is so much of consensus is wrong and it is going to start by giving you the consensus answer on anything.

There are subjects I can get it to tell me the consensus answer then say "what about x" and it completely changes and contradicts the first answer because x contradicts the standard consensus orthodoxy.

To me it is not much different than going to the library to research something. The library is not useless because the books don't read themselves or because there are numerous books on a subject that contradict each other. Gaining insight from reading the book is my role.

I suspect much LLM criticism is from people who neither much use LLMs nor learn much of anything new anyway.

I'm just a noob but lecun seems obsessed with the idea of world models, which I assume means a more rigorous physical approach, and I don't understand (again, confused noob here) how are t would help precise abstract thinking.

Old "agged Technological Frontier" but explains a bit the challenge https://www.hbs.edu/faculty/Pages/item.aspx?num=64700 namely... it's hard and the lack of reproducibility (models getting inaccessible to researcher quickly) makes this kind of studies very challenging.

> Finding patterns in large datasets is one of the things LLMs are really good at.

Where by "good at" you mean "are totally shit at"?

They routinely hallucinate things even on tiny datasets like codebases.

As a scientist, the two links you provided are severely lacking in utility.

The first developed a model to calculate protein function based on DNA sequence - yet provides no results of testing of the model. Until it does, it’s no better than the hundreds of predictive models thrown on the trash heap of science.

The second tested a models “ability to predict neuroscience results” (which reads really oddly). How did they test it? Pitted humans against LLMs in determining which published abstracts were correct.

Well yeah? That’s exactly what LLMs are good at - predicting language. But science is not advanced by predicting which abstracts of known science are correct.

It reminds me of my days in working with computational chemists - we had an x-ray structure of the molecule bound to the target. You can’t get much better than that at hard, objective data.

“Oh yeah, if you just add a methyl group here you’ll improve binding by an order of magnitude”.

So we went back to the lab, spent a week synthesizing the molecule, sent it to the biologists for a binding study. And the new molecule was 50% worse at binding.

And that’s not to blame the computation chemist. Biology is really damn hard. Scientists are constantly being surprised at results that are contradictory to current knowledge.

Could LLMs be used in the future to help come up with broad hypotheses in new areas? Sure! Are the hypotheses going to prove fruitless most of the time? Yes! But that’s science.

But any claim of a massive leap in scientific productivity (whether LLMs or something else) should be taken with a grain of salt.

In reference to the second article: who cares? What we care about is experimental verification. I could see maybe accurate prediction being helpful in focusing funding, but you still gotta do the experimentation.

Not disagreeing with your initial statement about LLMs being good and finding patterns in datasets btw.

How exactly are we asking for the confidence level?

If you give the model the image and a prior prediction, what can it tell you? Asking for it to produce a 1-10 figure in the same token stream as the actual task seems like a flawed strategy.

A blind mathematician can do revolutionary work despite not being able to see

> If the Magic Disruption Box is incapabele of knowing whether or not it read "42/A" correctly

Are you implying that science done by humans is entirely error-free?

Yes and no at the same time, depending on what you intend to get from asking. I don't know what you were doing with this project, obviously, so I don't speak to that, but science (well, stats in general, but science needs stats) has a huge dependency on being sure the question was the correct one and not just rhyming.

Reading hand-written digits was the 'hello world' of AI well before LLMs came along. I know, because I did it well before LLMs came along.

Obviously a simple model itself can't know if it's right or wrong, as per one of Wittgenstein's quote:

  If there were a verb meaning 'to believe falsely', it would not have any significant first person, present indicative.

That said, IMO not (as Wittgenstein seemed to have been claiming) impossible, as at the very least human brains are not single monolithic slabs of logic: https://www.lesswrong.com/posts/CFbStXa6Azbh3z9gq/wittgenste...

In the case of software, whatever system surrounds this unit of machine classification (be it scripts or more ML) can know how accurately this unit classifies things in certain conditions. My own MNIST-hello-world example, split the test set and training set, the test set tells you (roughly!) how good the training was: while this still won't tell you if any given answer is wrong, it will tell you how many of those 40 million is probably wrong.

Humans and complex AI can, in principle, know their own uncertainty, e.g. I currently estimate my knowledge of physics to be around the level of a first year undergraduate course student, because I have looked at what gets studied in the first year and some past paers and most of it is not surprising (just don't ask me which one is a kaon and which one is a pion).

Unfortunately "capable" doesn't mean "good", and indeed humans are also pretty bad at this, the general example is Dunning Kruger, and my personal experience of that from the inside is that I've spent the last 7.5 years living in Germany, and at all points I've been sure (with evidence, even!) that my German is around B1 level, and yet it has also been the case that with each passing year my grasp of the language has improved, so what I'm really sure of is that I was wrong 7 years ago, but I don't know if I still am or not, and will only find out at the end of next month when I get the results of an exam I have yet to sit.

I say hacker news should just be for low information clickbait headlines and daily pop culture articles that have nothing to do with technology, not AI advertisements.