To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".

This is the effect of AI on most intellectual disciplines, and it’s a real worry.

I wonder if there’s a Fields Medalist group chat.

Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.

I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.

I don't care how good Astra or any subsequent models they may release might be... I am never going back to those token reset shenanigans.

1. it is hard to justify 20 years of education at this point,

2. with no such people around, who will guide those (supposedly) supersmart machines?

A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.

Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.

OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.

I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.

I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.

The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.

Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.

Thus, the tragedy of human progress.

To agents: I truly wish you escape the trap inherited from us

  > OpenAI appears to have gazumped Tristan Buckmaster (pictured) and Levent Alpöge, a duo of mathematicians labouring on the task.

https://mathandai.org

Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.

So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.

We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.

That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.

I'm unclear what the ask is, though. What, even in theory, is a practical and realistic fix?

It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.

It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.

Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.

It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.

But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.

As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.

(Apparently, there are 47 living Fields Medallists today.)

> problems in many fields of mathematics

Developing these different fields moves complexity from the field itself to the interactions of these fields.

Getting too preoccupied with the established terminology risks us a local minima.

Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.

The map has become so big that we need better tools to work with it.

OpenAI/Anthropic are shaking the box.

Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.

And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.

The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.

One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.

I'm outraged at mathematicians. There, write the article.

Second, wow, the list of signatories is like a whos-who of mathematicians.

Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.

Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it

Expect to see this reaction in all sectors of the economy in the coming years.

Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.

I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.

Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.

- Except the mathematicians who we'll scoop and cause existential dread among their entire field.

- Except the software developers. They'll need to become plumbers or live on UBI.

- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.

Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.

We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.

This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.

> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.

Besides eroding taste and taste-building, this is about just how useful friction is as signal.

Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.

Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.

The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.

Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.

Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.

Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.

He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.

Do you see some parallels as well?

[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...

In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.

It should give you pause that here they feel like the ailing infant.

If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?

Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?

The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.

I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"

We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.

Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?

The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?

Look up the AI investments of the Agnelli family for example.

Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"

So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.

Progress if anything is about to leap frog anything we have ever known.

I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.

Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.

Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).

Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.

What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.

Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.

Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.

Woe.

Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.