> Purpose-driven institutions [...] empower individuals to take intellectual risks and challenge the status quo.
(which of course includes and is most-often the institution itself.)
Almost the defining problem of modern institutions is sweeping problems under the rug, be it be climate change or (most important) Habermas's "Legitimation Crisis" [1] It's something I've been watching happen at my Uni ever since I've had anything to do with it. The spectacle of institutions failing to defend themselves [2] turns people against them.
Insofar as any external threat topples an institution or even threatens it seriously there was a failure of homeostasis and boundaries from the very building.
[1] https://en.wikipedia.org/wiki/Legitimation_Crisis_(book)
[2] ... the king is still on the throne, the pound is still worth a pound ...
The link to download the paper is here: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5870623
Just as an example, they criticize the FDA for using an AI that can hallucinate whole studies, but they don't talk about the fact that it's used for product recalls, and the source that they use to cite their criticism is an Engaget article that is covering a CNN article that got the facts wrong, since it relied on anonymous sources that were disgruntled employees that had since left the agency.
Basically what I'm saying is the more you dig into this paper, the more you realize it's an opinion piece.
Either way, it's an effort, and at least the authors will learn to not to do.
It reads like a trademark attorney—turned academic got himself interested in "data" and "privacy," wrote a book about it in 2018, and proceeded to be informed on the subject of AI almost exclusively by journalists from popular media outlets like Wired/Engaget/Atlantic—to bring it all together by shoddily referencing his peers at Harvard and curiously-sounding 80's sociology. But who cares as long as AI bad, am I right?
Hard to take seriously with so many misspellings and duplicate punctuation.
I vibe with the general "AI is bad for society" tone, but this argument feels a lot to me like "piracy is bad for the film industry" in that there is no recognition of why it has an understandable appeal with the masses, not just cartoon villains.
Institutions bear some responsibility for what makes AI so attractive. Institutional trust is low in the US right now; journalism, medicine, education, and government have not been living up to their ideals. I can't fault anyone for asking AI medical questions when it is so complex and expensive to find good, personalized healthcare, or for learning new things from AI when access to an education taught by experts is so costly and selective.
Toxic is toxic.
So, web programmers could be going against AI on the grounds of self-preservation and be wholly justified in doing so, but lawyers are entitled to go after AI on more fundamental, irreconcilable differences. AI becomes a passive 'l'estat, cest moi' thing locking in whatever it's arrived at as a local maximum and refusing to introspect. This is antithetical to law.
But day to day, they spend a lot of their time selling boiler plate contracts and wills or trying to smuggle loopholes into verbose contracts, or trying to find said holes in said contracts presented by a third party[1]
Or if they are involved in criminal law, I suspect they spend most of their time sifting the evidence and looking for the best way to present it for their client - and in the age of digital discovery the volume of evidence is overwhelmning.
And in terms of delivering justice in a criminal case - isn't that the role of the jury ( if you are lucky enough still to have one ).
I suspect very few lawyers ever get involves in cases that lead to new precedents.
Most of the sections had no citations and anecdotes.
Like many who have predicted doom or ChatGPT can never do XYZ, anecdotes do not build a substantive argument.
Are we only talking about technological revolutions here or are you talking about peasants uprising in China 1000 years ago?
The moments after the revolution might be worse, but in the long term, we got better.
neither being able to speak to someone on a computer nor videos on the internet are new, fancy web 10.0 frontend notwithstanding
> and isolated people from each other.
I assume you mean doomscrolling as opposed to the communication social media affords. because social media actually connects us (unless apparently its facebook, then messaging is actually bad)
Part of the problem is that social media isn't social media anymore. Its an algorithmic feed that only occassionally shows content from people you're friends with. If Facebook went back to its early days when it was actually a communication tool, then I don't think you would see the same complaints about it.
JFC downvoters; when I posted this comment the title did not match the article title.
Social media was already isolating people. It is being sped up by the use of AI bots (see dead internet theory). These bots are being used to create chaos in society for political purposes, but overall it's increasingly radicalizing people and as a result further isolating everyone.
AI isn't eroding college institutions, they were already becoming a money grab and a glorified jobs program. Interpersonal relationships (i.e. connections) are still present, I don't see how AI changes that in this scenario.
I am not a fan of how AI is shaping our society, but I don't place blame on it for these instances. It is in my opinion that AI is speeding up these aspects.
The article does highlight one thing that I do attribute to AI and that is the lack of critical thinking. People are thinking less with the use of AI. Instead of spending time evaluating, exploring and trying to think creatively. We are collectively offloading that to AI.
Hard working expert users, leveraging AI as an exoskeleton and who carefully review the outputs, are getting way more done and are stronger humans. This is true with code, writing, and media.
People using AI as an easy button are becoming weaker. They're becoming less involved, less attentive, weaker critical thinkers.
I have to think that over some time span this is going to matter immensely. Expert AI users are going to displace non-AI users, and poor AI users are going to be filtered at the bottom. So long as these systems require humans, anyway.
Personally speaking:
My output in code has easily doubled. I carefully review everything and still write most stuff by hand. I'm a serious engineer who built and maintained billion dollar transaction volume systems. Distributed systems, active active, five+ nines SLA. I'm finding these tools immensely valuable.
My output in design is 100% net new. I wasn't able to do this before. Now I can spin up websites and marketing graphics. That's insane.
I made films and media the old fashioned way as a hobby. Now I'm making lots of it and constantly. It's 30x'd my output.
I'm also making 3D characters and rigging them for previz and as stand-ins. I could never do that before either.
I'm still not using LLMs to help my writing, but eventually I might. I do use it as a thesaurus occasionally or to look up better idioms on rare occasion.
To risk an analogy, if I throw petrol onto an already smouldering pile of leaves, I may mot have ‘caused’ the forest fire, but I have accelerated it so rapidly that the situation becomes unrecognisable.
There may already have been cracks in the edifice, but they were fixable. AI takes a wrecking ball to the whole structure
AI may have caused a distinct trajectory of the problem, but the old system was already broken and collapsing. If the building falls over or collapses in place doesn't change that the building was already at its end.
I think the fact that AI is allowed to go as far as it has is part of the same issue, namely, our profit-at-all-costs methodology of late-stage capitalism. This has lead to the accelerated destruction of many institutions. AI is just one of those tools that lets us sink more and more resources into the grifting faster.
(Edit: Fixing typos.)
This is how we get food that has fewer nutrients but ships better, free next-day delivery of plastic trash from across the world that doesn't work, schools that exist to extract money rather than teach, social media that exists primarily to shove ads in your face and trick you into spending more time on it.
In the next 4 years we will see the end of the American experiment, as shareholder capitalism completely consumes itself and produces an economy that can only extort and exploit but not make anything of value.
Universities have pushed post-modernism since the 60s which is the precursor for the deprecation of truth.
Later on when Google killed the press, truth had become subservient for easy money grabs, ending with a social media that is completely swamped with disinformation
However, an AI model with a decent alignment (and maybe government regulated alignment) might lead to decent narratives whose goal is not what we're seeing now with left and right populism, which is the destruction of the state
Call me crazy, but the situation may be more nuanced than this (and your next statement). For example, all universities embraced post-modernism? Also, universities are the arbiter for truth? If so, which universities and which truths? Or is it the transcendental Truth all universities gave out? Lastly, post-modernist ideas on media or some other part of culture?
I think the technical term is "throwing gas on the fire." It's usually considered a really bad thing to do.
> I am not a fan of how AI is shaping our society, but I don't place blame on it for these instances. It is in my opinion that AI is speeding up these aspects.
If someone throws gas on a fire, you can totally blame them for the fire getting out of control. After all, they made it much worse! Like: "we used to have smouldering brush fire that we could put out, but since you dumped all that gas on it, now we will die because we have a forest fire raging all around us."
It's not really a good argument to say 'but what if this argument is so right and so commonly held that an AI could regurgitate it?'. Well, yes, because AI is not inherently unable to repeat correct opinions. It's pretty trivial to get AI to go 'therefore, I suck! I should be banned'. What was it, Gemini, which took to doing that on its own due to presumably the training data and guidance being from abused and abusive humans?
It probably was in 1850-1950s, but not in the world I live today.
Press is not free - full of propaganda. I don't know any journalist today I can trust, I need to check their affiliations before reading the content, because they might be pushing the narrative of press owners or lobbies
Rule of law? don't make me laugh, this sounds so funny, look what happened in Venezuela, US couldn't take its oil, so it was heavily sanctioned for so many years, then it still couldn't resist the urge to steal it, and just took the head of the state.
Universities - do not want to say anything bad about universities, but recently they are also not good guys we can trust, remember Varsity Blues scandal? https://en.wikipedia.org/wiki/Varsity_Blues_scandal - is this the backbone of democratic life?
Did you think that was different from 1850-1950?
* there were no internet, so local communities strived to inform things happening around more objectively. Later on, there were no need for local newspapers
* capitalism was on the rise and on its infancy, but families with a single person working could afford some of the things (e.g. house, car) hence there were no urgent need to selling out all your principles
* people relied on books to consume information, since books were difficult to publish and not easy to revert (like removing a blog post), people gave an attention to what they're producing in the form of books, hence consumers of those books were also slightly demanding in what to expect from other sources
* less power of lobby groups
* not too many super-rich / billionaires, who can just buy anything they want anytime, or ruin the careers of people going against them, hence people probably acted more freely.
But again, can't tell exactly what happened at that time, but in my time press is not free. That's why I said "probably"
I think this centralization of authority over capital is what has allowed for the power of lobbying, etc. A billionaire could previously only control his farms, tenant farmers, etc. Now their reach is international, and they can influence the taxing / spending the occurs across the entire economy.
Similarly, local communities were probably equally (likely far more) mislead by propaganda / lies. However, that influence tended to be more local and aligned with their own interests. The town paper may be full of lies, but the company that owned the town and the workers that lived there both wanted the town to succeed.
The provided timespan encompasses the 'gilded age' era, which saw some ridiculous wealth accumulation. Like J.P. Morgan personally bailed out as the US Treasury at one point.
Did you have any examples or reading to share?
[1]: https://www.vox.com/2015/4/23/8485443/polarization-congress-...
if they publish anything contrarian
I am more optimistic about AI than this post simply because I think it is a better substitute than social media. In some ways, I think AI and institutions are symbiotic
"In prison, I learned that everything in this world, including money, operates not on reality..."
"But the perception of reality..."
Our distrust of institutions is a prison of our own making.
On the other side of the coin, the press and both parties ignored what was going on in rural America until the rise of Trump
They delegitimize knowledge, inhibit cognitive development, short circuit
decision-making processes, and isolate humans by displacing or degrading human connection.
The result is that deploying AI systems within institutions
immediately gives that institution a half-life.
e.g., an LLM-equipped novice becomes just enough of an expert to tromp around knocking down chesterton's fences in an established system of any kind. "First principles" reasoning combined with a surface understanding of a system (stated vs actual purpose/methods), is particularly dangerous for deep understanding and collaboration. Everyone has an LLM on their shoulder now.
It's obviously not always true, but without discipline, what they state does seem inevitable.
The statement that AI is tearing down institutions might be right, but certainly institutions face a ton of threats.
The authors use Elon Musk's DOGE as an example of how AI is destructive, but I would point out that that instance was an anomaly, historically, and that the use of AI was the least notable thing about it. It's much more notable that the richest man in the world curried favor by donating tens of millions of dollars to a sitting US president and then was given unrestricted access to the government as a result. AI doesn't even really enter the conversation.
The other example they give is of the FDA, but they barely have researched it and their citations are pop news articles, rather than any sort of deeper analysis. Those articles are based on anonymous sources that are no longer at the agency and directly conflict with other information I could find about the use of that AI at the FDA. The particular AI they mention is used for product recalls and they present no evidence that it has somehow destroyed the FDA.
In other words, while the premise of the paper may seem intellectually attractive, the more I have tried to validate their reasoning and methodology, the more I've come up empty.
Coincidentally, this has happened exactly when the Flynn effect reverted, the loneliness epidemic worsened, the academics started getting outnumbered by the deans and deanlings and the average EROI of new coal, oil and gas extraction projects fell below 10:1. Sure, we should be wary of the loss to analysis if we just reduce everything to an omnicause blob, but the human capital decline wouldn't be there without it.
Similarly, it seems to me like the rule of law (and the separation of powers), prestige press, and universities are social technologies that have been showing more and more vulnerabilities which are actively exploited in the wild with increasing frequency.
For example, it used to be that rulings like Wickard v. Filburn were rare. Nowadays, various parties, not just in the US, seem to be running all out assaults in their favoured direction through the court system.
--> "Civic institutions—the rule of law, universities, and a free press—are the backbone of democratic life."
People are the backbone of our civilization. People who have good intentions and support one another. We don't NEED an FDA to function -- it's just a tool that has worked quite well for a long time for us.
We publish common sense laws, and we have police officers and prosecutors, and then we have a court system to hold people accountable for breaking the law. That's one pretty major method that has little to do with the need for an institution like FDA.
I don't know if a system that relied entirely on tort and negligence and contract law to protect people from being sold snake oil would function better or worse than FDA, but I do know something like FDA (where a bunch of smart people advise very specifically on which drugs are ok to take and which are not) isn't the only option we have.
Fun quotes from the paper > I. Institutions Are Society’s Superheroes: Institutions are essential for structuring complex human interactions and enabling stable, just, and prosperous societies.
> Institutions like higher education, medecine, and law inform the stable and predictable patterns of behavior within organizations such as schools, hospitals, and courts., respectively,, thereby reducing chaos and friction.
>Similarly, journalism, as an institution, commits to truth-telling as a common purpose and performs that function through fact-checking and other organizational roles and structures. Newspapers or other media sources lose legitimacy when they fail to publish errata or publish lies as news.
> Attending physicians and hospital administrators may each individually possess specific knowledge, but it is together, within the practices and purposive work of hospitals, and through delegation, deference, and persistent reinforcement of evaluative practices, that they accomplish the purpose of the institution
> The second affordance of institutional doom is that AI systems short- circuit institutional decisionmaking by delegating important moral choices to AI developers.
>Admittedly, our institutions have been fragile and ineffective for some time.36 Slow and expensive institutions frustrate people and weaken societal trust and legitimacy.37 Fixes are necessary.
> The so-called U.S. “Department of Government Efficiency” (“DOGE”) will be a textbook example of how the affordances of AI lead to institutional rot. DOGE used AI to surveil government employees, target immigrants, and combine and analyze federal data that had, up to that point, intentionally been kept separate for privacy and due process purposes.
It's all politics. 150% bullshit.
Having super accessible machines that can make anything up and aren't held accountable run the world is going to break so many systems where truth matters.
Having large bureaucratic organizations
> that can make anything up and aren't held accountable
that run everything and aren't held accountable
> run the world is going to break so many systems
run the world breaking up families, freedom, and fun
> where truth matters.
where truth is determined by policy
Yes, that's what the paper argues. Institutions at every scale (say, doctor's clinics, hospitals, entire healthcare systems) are very challenging to access compared to me asking ChatGPT. And not just bureaucracy, but there's time, money and many other intangible costs associated with interacting with institutions.
> [Large bureaucratic organizations]that run everything and aren't held accountable
But they ultimately are. People from all types of institutions are fired and systems are constantly reorganized and optimized all the time. Not necessarily for the better -- but physical people are not black boxes spewing tokens.
Individuals' choices are ultimately a product of their knowledge and their incentives. An MLM's output is the result of literal randomness.
> run the world breaking up families, freedom, and fun
There's lots of terrible institutions vulnerable to corruption and with fucked up policies, but inserting a black box into _can't_ improve these.
> where truth is determined by policy
The truth is the truth. Regardless of what policy says. The question is, do you want to be able to have someone to hold accountable or just "¯\_(ツ)_/¯ hey the algorithm told me that you're not eligible for healthcare"
*Core Thesis:* AI systems' fundamental design features degrade and will eventually destroy civic institutions essential to democratic life.
*Three Destructive Affordances of AI:*
1. *Undermines Expertise* - Encourages cognitive offloading, leading to skill atrophy - Creates illusion of accuracy while producing inevitable "hallucinations" - Backward-looking nature cannot adapt to changing circumstances - Displaces knowledge transfer between humans
2. *Short-Circuits Decision-Making* - Outsources moral choices to machines, obscuring accountability - Flattens institutional hierarchies needed for oversight - Removes critical points of reflection and contestation - Incapable of intellectual risk-taking or challenging status quo
3. *Isolates Humans* - Displaces opportunities for interpersonal connection - Sycophantic design erodes capacity for managing social friction - Depletes social capital and solidarity institutions require
*Institutions at Risk:*
- *Rule of Law:* AI decisions lack transparency, predictability, and accountability required for legitimate governance - *Higher Education:* Offloads learning, homogenizes output, undermines trust between students and educators - *Journalism:* AI "slop" pollutes information ecosystem; press cannot fulfill watchdog function - *Democracy:* Erodes social capital, generalized reciprocity, and civic participation
*Key Evidence Cited:* - Studies showing AI use inhibits critical thinking and problem-solving skills - DOGE's use of AI to surveil employees and bypass institutional safeguards - FDA's "Elsa" system hallucinating nonexistent studies - Research on "Model Autophagy Disease" degrading AI accuracy
*Conclusion:* Without rules mitigating AI's spread, institutional dissolution is inevitable. Authors call for bright-line prohibitions rather than half-measures like ethics principles or consent frameworks.
Speculative, we don't have that evidence yet. The evidence robust we do have is in students who have not yet developed skills and maturity, and who also have a lot of other confounding factors in their development, eg. social media, phones, formative development years exposed to immediate pleasure-seeking feedback loops.
> Backward-looking nature cannot adapt to changing circumstances
As opposed the nimble flexibility that established institutions are historically known for?
> Displaces knowledge transfer between humans
If people use LLMs, presumably it's because knowledge transfer is faster and more convenient that way than via direct interaction.
> Flattens institutional hierarchies needed for oversight
We have hierarchical institutions for oversight because flatter institutions don't scale (because humans don't scale). If AI can scale and can provide the same transparency, accountability and oversight, how is that not an improvement?
I could go on with the remaining points, but suffice it to say that there are a lot of faulty assertions behind the paper's arguments. It's also interesting that every chicken little saying that the sky is falling immediately reaches for the ban hammer instead of providing constructive criticism on how AI (or whatever innovation) can improve to mitigate these issues.
I can see this happening. Earlier, more people worked in groups because they relied on their expertise.
Now, there is no need for this; people can do it alone. Even though this makes the work done, it comes at the cost of isolation.
I am sure for some people this would look like a win.
It only isolates you, if you let it isolate you. The pandemic shifted my life, as I have been working alone at home ever since. I am single no kids, and after the pandemic ended I continue to stay "isolated". I knew about that dangers and took active measures - some of which were only possible because I was no longer required to go to an office. I moved to another country, to a location with a lot of international expats that work online, too. I built an active social circle, attending meetups, sport groups, bar nights etc.
I am now more social and happier than ever, because my daily social interactions are not based on my work or profession, and I get to chose with whom I spend my time and meet for lunches. Before, the chores around hour long commutes, grooming, packing my bag, meal-prep, dressing properly etc. just to sit in the office all day - all are gone from my schedule. I have more free time to socialize and maintain friendships, pay less rent, and in general - due to lower cost of living - life quality has improved significantly.
Without work-from-home this would not be possible. You could argue WFH results in isolation and depression, but for me it was liberating. It is, of course, each individuals own responsibility (and requires acitve work, sometimes hard work, too) that will influence the outcome.
Using AI wisely can augment human capability without eroding institutional roles — the real question is how accountability, transparency, and critical thinking evolve alongside the technology.
If the institutions cannot handle that, they will have to change or be destroyed. Take university, for instance. Perhaps they will go away - but is this a great loss? Learning (in case it will remain relevant) can be more efficiently achieved with personal AI assistants for each student.
“It’s not guns that kill people, it’s people that kill people”.
It’s not “AI bad”, it’s about the people who train and deploy AI.
My agents always look for research material first - won’t make stuff up. I’d rather it say “I can’t determine” than make stuff up.
AI companies don’t care about institutions or civil law. They scraped the internet, copyright be damned. They indexed art and music and pay no royalties. If anything, the failure of protecting ourselves from ourselves is our fault.
But most of these institutions predate the existence of game theory, and it didn't occur to anyone how much they could be manipulated since they were not rigorously designed to be resistant to manipulation. Slowly, people stopped treating them like a child's tower of blocks that they didn't want to knock over. They started treating them like a load bearing structure, and they are crumbling.
Just as an example, the recent ICE deportation campaign is a direct reaction to a political party Sybil[0] attacking the US democracy. No one who worked on the constitution was thinking about that as a possibility, but most software engineers in 2026 have at least heard the term.
The best that the experts the paper talks about can do today is say that if we follow their advice our lives will get worse slower. Not better. Just as bad as if we don't listen to them, but more slowly.
In the post war period people trusted institutions because life was getting better. Anyone could think back to 1920 and remember how they didn't have running water and how much a bucket weighed when walking up hill.
If big institutions want trust they should make peoples lives better again instead of working for special interests, be they ideological or monetary.
FWIW, I know a lot of people who refuse to go to the dentist unless it's an issue because they're one of the medical professions that seem to do the most upselling.
I go every six months for a cleaning and trust my dentist, but I can definitely see how these huge chain dentists become untrustworthy.
"Banning Trump from Twitter and Facebook isn’t nearly enough"
https://www.latimes.com/opinion/story/2021-01-15/facebook-tw...
Even if you dislike Trump, the campaign to suppress conservative voices of the 2010s (now largely reversed) that he argues for there was a significant contribution to the decline in authority and respect that academia has had in the eyes of the general populace.
To be clear, the same must also apply to any suppression of liberal voices - its unacceptable in a culture that claims free speech.
But I am sceptical that this particular writer has a moral high ground from which to opine.
The obvious culprits being smartphones and social networking, though it's really hard to prove causality.
At the very least, this should make us reconsider what we are building and the incentives behind it.
Material (full page) material material, sources at the end. Simple. Readable.
Material (half page), sources (half page). Material/source. Material/source. Looks quite unreadable to the eye.
AI may be destroying truth by creating collective schizophrenia and backing different people’s delusions, or by being trained to create rage-bait for clicks, or by introducing vulnerabilities on critical software and hardware infrastructure. But if institutions feel threatened, their best bet is to become higher levels of abstraction, or to dig deeper into where they are truly deeply needed - providing transparency and research into all angles and weaknesses and abuses of AI models. Then surfacing how to make education more reliably scalable if AI is used.
"If you wanted to create a tool that would enable the destruction of institutions that prop up democratic life, you could not do better than artificial intelligence. Authoritarian leaders and technology oligarchs are deploing [sic] AI systems to hollow out public institutions with an astonishing alacrity"
So in the first two sentences we have hyperbole and typos? Hardly seems like high-quality academic output. It reads more like a blog post.
chrisjj•1h ago
PaulHoule•1h ago
chrisjj•1h ago