Should be possible to backtest by training LLMs on historic datasets and then probing them to see if they can re-discover things that were discovered after their training data cut-off. What sort of prompts could push them to make a breakthrough.

AlphaFold already did. Or do we only count AI if it’s an LLM now?

Is this a recruiting attempt by Los Alamos? AI/ML for science as this broad field used to be known is interesting. Some five years ago there as a real craze where every STEM lab at my university was doing some form of ML project. I think by now people have learned what works and what doesn't. Climate models for example have been quite successful. Possibly the reason is that they learn directly from collected data, rather than trying to emulate the output of simulations. Attempts to build similiar models for fluid dynamics have been rather dismal. In general, big models and big data result in useful models, even if only because these models seem to be somehow interpolating based on similiar training data points. Trying to replace classical physics based models with ML models trained on simulation data does not seem to work. The model is only ever capable of emulating a physically plausible output when the input is close enough to the training data, and that too, only when the system isn't chaotic. For applications where you are generating a sample to be used in a downstream task, ML models trained on lots of data can be very useful. You only need a few lucky guesses, that you can verify downstream, to end up with some useful result. In short, there is no magic to it. It's a useful tool that can be regarded as both a search algorithm and an optimization algorithm.

Bit short on details other than "Let's see what LLMs can predict when we train them on various scientific data sets."

Certainly a good thing to try, but the article feels like a PR piece more than anything else, as it's not answering anything, just giving a short overview of a few things they're trying with no data on those things whatsoever.

It does fit in with the "Throw LLM spaghetti at a wall and see what sticks" trend these days though.

I think our creativity has not yet been duplicated in AI, so for maximum results, we need to pair AI with a human expert or a panel of human experts and innovate by committee. AI brings to the table vast memory, instant recall and most importantly, tired-less pursuit and the human element can provide creative guidance and prompt. The trick is in curating the BOK(body of knowledge) used to train GENERATIVE AI. I wonder what a curricula designed specifically for AI would look like?

Yes. ML has advanced many fields related to modelling - meteorology, climate, molecular. Classification models have done much for genomics, particle physics, and other fields where experiments produce inhumane amounts of data.

DeepVariant, Enformer, ParticleNet, DeepTau, etc. are some well-known individual models that are advanced branches of science. And there are the very famous ones, like AlphaFold (Nobel in Chemistry 2024).

We need to think of AI not as a product (chats, agents, etc.), but as neural nets (AlexNet). Unfortunately, large companies are "chat-washing" these tremendously useful technologies.

I would be interested in machine learning for scientific research. Something more “physical” than optimizing software.

I checked some of the nuclear fusion startups and didn’t see anything.