Hyperparam tuning that has better intuition and can incorporate architecture changes automatically. It won't invent something completely new though.

Wrong and short-sighted take given that the LLM explores serially learning along the way, and can tool use and change code arbitrarily. It seems to currently default to something resembling hyperparameter tuning in absence of more specific instructions. I briefly considered calling the project “autotune” at first but I think “autoresearch” will prove to be the significantly more appropriate name.

Yeah I thought that was a particularly neat part

Why do we think this emerged “on its own”? Surely this technique has been discussed in research papers that are in the training set.

Why?… The experiment.yaml shows that it is calling h100/200 explicitly, it’s pretty common for humans to say “number bigger more gooder” for anything… Lie and reverse the values and see what happens. I would put money on a rabbit hole of complaining about it being misconfigured.

> An agent is honestly just brute-force search, but guided.

Heuristic search, then.

Human-driven research is also brute-force but with a more efficient search strategy. One can think of a parameter that represents research-search-space-navigation efficiency. RL-trained agents will inevitably optimize for that parameter. I agree with your statement insomuch as the value of that efficiency parameter is lower for agents than humans today.

It's really hard to imagine that they __won't__ exceed the human value for that efficiency parameter rather soon given that 1. there are plenty of scalar value functions that can represent research efficiency, of which a subset will result in robust training, and 2. that AI labs have a massive incentive to increase their research efficiency overall, along with billions of dollars and really good human researchers working on the problem.

Is there anything in the research space that doesn't fit "brute-force search, but guided"?

All of science is "gather inputs, make hypothesis, test, analyse" on repeat.

There's plenty to critique in the particular guidance approach, but the overall method is the same.