Should be possible to backtest by training LLMs on historic datasets and then probing them to see if they can re-discover things that were discovered after their training data cut-off. What sort of prompts could push them to make a breakthrough.

AlphaFold already did. Or do we only count AI if it’s an LLM now?

Is this a recruiting attempt by Los Alamos? AI/ML for science as this broad field used to be known is interesting. Some five years ago there as a real craze where every STEM lab at my university was doing some form of ML project. I think by now people have learned what works and what doesn't. Climate models for example have been quite successful. Possibly the reason is that they learn directly from collected data, rather than trying to emulate the output of simulations. Attempts to build similiar models for fluid dynamics have been rather dismal. In general, big models and big data result in useful models, even if only because these models seem to be somehow interpolating based on similiar training data points. Trying to replace classical physics based models with ML models trained on simulation data does not seem to work. The model is only ever capable of emulating a physically plausible output when the input is close enough to the training data, and that too, only when the system isn't chaotic. For applications where you are generating a sample to be used in a downstream task, ML models trained on lots of data can be very useful. You only need a few lucky guesses, that you can verify downstream, to end up with some useful result. In short, there is no magic to it. It's a useful tool that can be regarded as both a search algorithm and an optimization algorithm.

Bit short on details other than "Let's see what LLMs can predict when we train them on various scientific data sets."

Certainly a good thing to try, but the article feels like a PR piece more than anything else, as it's not answering anything, just giving a short overview of a few things they're trying with no data on those things whatsoever.

It does fit in with the "Throw LLM spaghetti at a wall and see what sticks" trend these days though.

I think our creativity has not yet been duplicated in AI, so for maximum results, we need to pair AI with a human expert or a panel of human experts and innovate by committee. AI brings to the table vast memory, instant recall and most importantly, tired-less pursuit and the human element can provide creative guidance and prompt. The trick is in curating the BOK(body of knowledge) used to train GENERATIVE AI. I wonder what a curricula designed specifically for AI would look like?

Yes. ML has advanced many fields related to modelling - meteorology, climate, molecular. Classification models have done much for genomics, particle physics, and other fields where experiments produce inhumane amounts of data.

DeepVariant, Enformer, ParticleNet, DeepTau, etc. are some well-known individual models that are advanced branches of science. And there are the very famous ones, like AlphaFold (Nobel in Chemistry 2024).

We need to think of AI not as a product (chats, agents, etc.), but as neural nets (AlexNet). Unfortunately, large companies are "chat-washing" these tremendously useful technologies.

I would be interested in machine learning for scientific research. Something more “physical” than optimizing software.

I checked some of the nuclear fusion startups and didn’t see anything.

Did… did this really long bio answer the question? I’m not sure what this is trying to do tbh, but it was interesting to see how these labs have been trying to incorporate AI.

I thought Alpha Fold advanced biology enough to get Jumper and Hassabis the Nobel Prize in Medicine.

I feel like there are a few ways in which science could be advanced by AI models.

1. We have the sense that "science progresses one funeral at a time." An AI model could be used to recognize situations where a single viewpoint is getting a disproportionate amount of attention in the literature, and warn journals+funding agencies about any disconnects between attention and quality of work.

2. We have the sense that "It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" An AI model could identify the most high-profile results in the literature that have contradicting evidence and call for further, decisive study.

3. Interdisciplinary translation. There are many many cases of different branches of science re-discovering each other's work. I believe I read an article a little bit ago about an academic in a somewhat softer science publishing a paper proudly claiming the discovery of linear regression. Obviously not all cases are so egregious, but an AI could advance a discipline just by pointing out areas where that discipline is using outdated/inferior methods compared to the state of the art.

I got a 404 loading the page, is it blocked outside the US?