And I mean total. A sufficiently advanced algorithm will be able to find everything a person has ever posted online (by cross referencing, writing style, etc.) and determine their views and opinions with high accuracy. It'll be able to extrapolate the evolution of a person's opinions.
The government will be able to target dissidents even before they realize they are dissidents, let alone before they have time to organize.
https://www.newsweek.com/donald-trump-database-palantir-dyst...
As many as you can control with signal chat.
Besides, I'm not sure if tanks like the Abrams are as important anymore. Nowadays, things like food and water really matter. For example, exporting corn is crucial. Also, having the soils needed to make modern tech, like chips and batteries, is super important. Therefore Greenland is.
Every billionaire in function constitutes an aristocracy of one. Meanwhile, states have armed forces. A billionaire, in relying on money for power, implicitly also depends on continued access to the global financial system which gives money meaning, in order to exercise that power. States are not obligated to allow such access, and may easily prevent it in the limiting case by ensuring such trade comes with a side of explosives delivered at speed, which broadly suffices to deter desirable counterparties.
Think what anyone may of such a thing, the fact is having an army or navy or air force means you can do it. Which billionaire has one of those?
You are making an awful lot of assumptions there. There are plenty of real world examples where a single party having control over a process produces significantly better results (and plenty of counterexamples where it doesn't, of course). How do you justify the assumption that preparing the next generation of leadership is one or the other of those?
> ... States are not obligated to allow such access,
Sufficient influence can be used to ensure access until it is no longer necessary. After all the government is little more than the biggest thug in the room at the end of the day.
> Which billionaire has one of those?
Which billionaire has been incentivized to pursue such a goal? It's extremely expensive.
Does the Saudi state count given that it's an absolute monarchy?
The Saudi monarchy I would characterize as a nation-state existentially dependent for that status upon its role as a valued¹ satrap, and regional linchpin, of US hegemony; after decades of wielding money and power with all the arrogant foolishness of any parvenu's fantasy, they have few obvious friends and many reliable enemies. Following the loss for any reason of that client status and the protective halo it confers, I would expect the remaining lifetime of the monarchy, and whichever of its members failed to flee in time, to be most conveniently measured in days.
1) Some might say, vastly overvalued given their longstanding unreliability and increasing irrelevance in a post-petroleum energy economy.
For example, Ukraine was planning to produce 1 million drones in 2024. They actually produced >2 million. For this year, they intend to produce 4.5 million (and appear to be on track for that).
What makes this especially interesting is that a single drone is cheap enough that it's possible even for individuals to own small swarms or to crowdfund larger ones. So there's no well-defined cliff of power here as there was with high-tech weaponry like tanks - you can gradually scale up to whatever resources allow. Meaning that it's much easier to be a "small feudal" type of an autocrat, for example.
I don't know. Do you? My prior interlocutor cited a state in example of this putative lordlet's theoretically plausible capabilities. I see you claim there is no difference in kind and none significant in degree. But you do actually need to make an argument about that. Just saying it won't do.
It's interesting you mention asymmetrical warfare, the fundamental axiom of which as a concept is that the nonstate or nonpeer force has no chance whatsoever through conventional tactics, which are far more reliable and thus invariably used for practical reasons when practicable. (Americans always get this wrong because of Vietnam. The NVA and the VC were not the same force, dammit!)
To cast the conflict in such terms, with the billionaire in place of the "asymmetrical" belligerent, thus makes my point for me in neater, more concise, and no less accurate terms than I myself expected to ever get away with on this website. You have my genuine thanks for that valuable assistance in clarifying my thinking.
> the nonstate or nonpeer force has no chance whatsoever through conventional tactics, which are far more reliable and thus invariably used for practical reasons when practicable.
I'd have to disagree. It refers broadly to tactics that don't favor size (ie expendable manpower, cash, physical resources, etc) to the same degree as conventional ones. Questions of reliability, cost, speed, user safety, and all sorts of other considerations apply in equal measure to any tactic that any party of any size might consider employing and tend to vary wildly depending on the context.
> Americans always get this wrong because of Vietnam. The NVA and the VC were not the same force, dammit!
... NVA? Was the US pushed out by a non-peer adversary or not? Were any favorable alternatives available? Isn't that all that matters at the end of the day?
> thus makes my point for me
Unfortunately I remain in the dark.
FWIW I don't think drones count as "asymmetrical warfare". They might have been that ten years ago, but a lot has changed since then. Today, they are very much part of conventional warfare, and the state with more drones still wins, all else being equal.
This would certainly be the case for a billionaire, who simply cannot swing the resources to sustain the logistics to support the kind of formalistic, resource-intensive combat you envision. In a realistic scenario the strategic and economic balance is going to look like the American Civil War only much more so, inasmuch as where the Confederacy at least at the outset of the conflict had an economy it could dedicate to supporting the war, a billionaire, as I've already touched on nearby, not only lacks such power but indeed requires unfettered economic access to continue to exist, a vulnerability he constitutionally cannot answer since it inheres in the source of his power...again just like the Confederacy, which relied on trade in raw agricultural outputs to supply itself with the tools and materials of industrial warfare, and so saw its back broken after the Union suppressed that trade by blockading southern ports.
But at least the Confederacy had an economy, however incompetent. What, I ask again, does a billionaire have that an army can't take away?
Is this like a sufficiently smart compiler? :)
Stylometry is well-studied. You'll be happy to know that it is only practical when there are few suspect authors for a post and each author has a significant amount of text to sample. So, tying a pseudonymous post back to an author where anyone and everybody is a potential suspect is totally infeasible in the vast majority of cases. In the few cases where it is practical, it only creates a weak signal for further investigation at best.
You might enjoy the paper Adversarial Stylometry: Circumventing Authorship Recognition to Preserve Privacy and Anonymity by Greenstadt et al.
How do y’all establish ye Theory Of Stylometry, O Phrenology Majors?
O, @dang confirms it on Mastodon or something??
More seriously, why is this essential?
That language could be recognized by a deterministic finite automaton!
What if I don't have an alternate HN account? Or what if I do have one, but it has barely any posts? How can you tie this account back to my identity?
Stylometry.net is down now, so it's hard to make any arguments about its effectiveness. There are fundamental limitations in the amount of information your writing style reveals.
How do you know it didn't miss 10x more than it found? Like, that's almost definitionally unprovable.
You're missing the point, it doesn't have to be practical, only the illusion of it working is good enough.
And if authoritarian governments believe it works well enough, they are happy to let a decent fraction of false positives fall through the cracks.
See for example, polygraph tests being used in court.
You also assume people are meticulous in taking precautions and never make mistakes. People whose access to information is restricted (such as by the great firewall of china) won't even know stylometry exists.
Even if it doesn't identify all cases, the chilling effect is enough.
Authoritarians are not known for their attention to false positive rates.
---
You didn't even try to address that dissent will be identified before it reaches a sufficiently conscious and intentional level to even consider taking precautions.
Most people don't have alts for normal usage and tying an account to IP to real person is easy for the government. The only difficulty is identifying opponents at scale.
---
Finally, please don't dismiss arguments by employing the "sufficiently snarky comment" technique.
The truth, accuracy doesn't matter to authoritarians. It doesn't matter to Trump, clearly, people are being sent away with zero evidence, sometimes without formal charges. That's the point of authoritarianism. The leader just does as he wishes. AI is not enabling Trump, the feckless system of checks and balances is. Similarly, W lied about wmd's, to get us into an endless war. It doesn't matter that this reason wasn't truthful. He got away with it and enriched himself and defense contractor buddies at the expense of the American people.
The essay attempted to mitigate this by noting OAI is nominally a non-profit. But it's clear the actions of the leadership are firmly aligned with traditional capitalism. That's perhaps the only interesting subtly of the issue, but the essay missed this entirely. The omission could not have been intentional, because it provides a complete motivation for item #2.
[1] #2 is 'The US is a democracy and China isn’t, so anything that helps the US “win” the AI “race” is good for democracy.'
Real improvements are achieved in the real world, and building more houses or high speed trains does not require "AI". "AI" will just ruin the last remaining attractive jobs, and China can win that race if they want to, which isn't clear yet at all. They might be more prudent and let the West reduce its collective IQ by taking instructions from computers hosted by mega corporations.
That is, "the ends justifies the means"? Yep, seems like we are already at war. What happened to the project of adapting nonzero sum games to reality??
I would not do business with Kim Jong Un. He is murdering a lot of his own people. Or with Putin. He is murdering a lot of Ukrainians.
But guess what: both North Korea and Russia are under sanctions. You can't do business with them anyway.
But the UAE is not under sanctions. Which means that in the opinion of the US Government it is ok to do business with them. Then who is Open AI to say otherwise? Why should it be any of their concern to determine who is a good guy or a bad guy in the world? Shouldn't there be a division of responsibilities? Let the Department of State determine who is good and who is bad, and let companies do business with those who are not on the sanctions list.
Either is our duty to be the the moral arbiters of the world or it isn't. Which one is it?
There are more than two answers to everything.
> Wasn't it moral to try to eliminate a known mass murderer?
Given the context and the means. No.
Many were opposed to that war, not because they didn't feel it was right to eliminate a mass murderer, but because that was not the stayed reason. The stated reason in fact turned out to be false, and was arguably an abject lie.
In other words ... it's not a great example of what you're trying to claim.
including our own…
At the, end, Saddam ultimately pulled too hard on the leash and miscalculated his power. Murder, mass or otherwise and morality has little bearing on matters of empire.
Thinking otherwise is naive.
Because helping someone do something bad is itself bad.
> Shouldn't there be a division of responsibilities?
It sounds like you mean an abdication of responsibility? We are already responsible for our own choices and actions, as well as their effects.
A lot of the people reading Hacker News right now think they have a better solution for the societal problems of the UAE. I personally have no idea about what's going on over there. But let's say that I'm in charge of the business decisions at Open AI. Should I start thinking that I know a way to solve their problems, and part of that way is for my company to apply some form of AI embargo on them? Or should I simply know my limitations, and restrict my judgment to the matters I am familiar with.
"Abdication of responsibility". What grand words. Why exactly has Open AI a responsibility to guide the UAE towards a better future? And, more importantly, why should Open AI feel confident that they know what is better for the UAE?
OpenAI is not responsible for UAE government. However, it's responsible for its own actions, and for their easily predictable consequences.
And I am talking about not doing bad things, which includes helping others do bad things.
> It is very appealing to think you know better than other people.
Not really. Responsibility, principles, and morality are work. It's far more appealing to claim they are someone else's job, dispense with them, turn your brain off, and do whatever feels good.
> A lot of the people reading Hacker News right now think they have a better solution for the societal problems of the UAE.
With all due respect, the percentage of HackerNews readers who are thinking about the UAE right now can be counted on one hand. This discussion is about generalities: Don't help people do bad stuff.
You're arguing about what's bad, and what isn't, and who decides. Interesting topic (perhaps for someone else or for another time)! Would you help someone do something which you think is bad?
I wouldn’t. But this is not the question here. The question here is if I would give someone a powerful tool that can be used for good, for neutral or for bad things, if I believe that there is a good chance it will be used for bad things. That puts a lot of responsibility on me to get into the business of predicting bad behavior.
Not a ton: you don't need to come up with a numerical probability, and erring on the side of caution is just fine.
That said, I think that IS the question here. If you think it's good or even neutral to give autocrats technology which they almost certainly will use for autocracy (even if they do other things with it), then this isn't a moral dilemma you need concern yourself with.
In the West, we have the attitude that autocracy is bad, if only autocrats were not helped by us, they'll fall and they'll be replaced by democracy, and perhaps by crowds singing in unison Beetles' "Imagine". In reality, democracy is a very unlikely outcome of a dictator being toppled. The top 2 outcomes, by miles and miles, are 1. the dictator is replaced by another dictator and 2. the dictator is replaced by some weak pseudo-state like Somalia, Lebanon or Yemen.
I think we should re-calibrate our self-confidence that we know what's best for others. As I said, I have no idea who is governing the UAE, but I know that it's much easier to criticize them than to come with a realistic better alternative.
Many other places, too. Obviously it's not a universal thing: every sufficiently large group of people has fans of autocracy. There has historically been evolutionary advantages to wanting a big, strong hand to control everything. Any alternative has not had sufficient time to evolutionarily propagate through humans.
The rest of your post kind of veers off-topic, so remember we're talking about individual people's decision to not work on something based on their own judgements. I don't think it's worth arguing at them that their judgements are wrongly made.
Again, If you think it's good or even neutral to give autocrats technology which they almost certainly will use for autocracy (even if they do other things with it), then this isn't a moral dilemma you need concern yourself with. Nobody is trying to convince you personally to change your morals. So, as long as you don't try to change theirs, there is no need to be defensive.
> I think we should re-calibrate our self-confidence that we know what's best for others.
I think a good place to start with this, would be to be less confident that you know what's best for others who are deciding whether to empower autocrats. It's much easier to criticize them and their decision, than to come with a realistic, better alternative.
You think for some reason that this type of position is one of arrogance. That I should show modesty and let people, such as yourself, criticize Open AI for getting in bed with autocrats. And if myself I have no problem with autocrats, then I should just keep out of this, because nobody around here asked for my opinion.
But this is very backward. When Jesus said that only those who never wronged in their life should throw a stone, one could retort that only people who never wronged should tell other people what to do with the stones they find on the street. Does that type of argument make sense to you?
importantly, we are not forced to adopt the morality of 1 particular government, out of hundreds of thousands – this is an appeal to authority (and not even a good authority)
if someone thinks that empowering autocrats is immoral, even if it's not illegal, then it is indeed arrogance to say that you have decided their morality is wrong
> You think for some reason... That I should show modesty and let people, such as yourself, criticize Open AI for getting in bed with autocrats
not really: modesty in judging others was your suggestion. you can stand by your suggestion or not, but you don't have the power to "let" or not "let" anybody criticize autocrats or their enablers
remember here: them criticizing autocrats and their enablers, is equally valid to you criticizing them for their criticism; which is equally valid to others criticizing you for criticizing their criticism; etc... we could keep going, but as you undoubtedly see, there is no good discussion to be had in the meta-direction of criticizing criticism, rather than directly addressing the issues raised in the initial critique
as for the jesus analogy: I wish I could follow it, but I could not, likely due to a failing on my part (I'm not familiar with jesus) -- would you mind rephrasing the point as a non-analogy (directly)? perhaps an analogy won't be necessary because the message is clear!
Sure, but the course of action that best reflects this is to not get involved in such business at all (because you just admitted that you're incapable of judging its consequences).
LLMs cheaply produce plausible and persuasive BS. This is what they've done from the start. Exactly how that ability will be applied we don't know, but it doesn't take a lot to see that the Venn Diagram of 'cheap & effective BS' and 'public good' does not have a great deal of overlap.
(For a modern parallel - we haven't built a small, portable nuclear reactor ever. The closest you get are nuclear-powered submarines, but no nuclear trucks.)
With LLMs, suddenly we have a tool that can generate misinformation on a scale like never before. Messaging can be controlled. Given that the main drivers of this technology (zuck, nadella, altman, and others) have chosen to make befellows of autocrats what follows is surely not a surprise.
The locomotion thing might not have been as obvious at the time as it is in hindsight.
The trains thing did come a bit later, but the steam engines were already causing a revolution.
Also nonsense. The canal boats were replaced by steam railways. A hundred years later. The water in the canals came from regular sources, by gravity.
It took decades to even get steam engines that turned a shaft (as opposed to rocking a beam up and down).
Steam engines (which rocked a bar, not rotated a shaft) were put to use draining water out of coal mines. They were powered by that same coal, and the water they pumped out was put right into the canals (where else would you put it?). This made coal cheaper, which in turn allowed more people to move from the country to the cities because cities were no longer reliant on firewood. Growing urban populations was demanded and supported by growing industry in those cities.
My whole point is that all of these factors complimented each other, they created a feedback loop that was the industrial revolution. Trains came later, after the industrial revolution was already well underway. That's what I said already.
My original comment was in response to the statement that improved locomotion was an obvious consequence of the steam engine.
I contend that is was not obvious. The first walking beam engines were more like buildings, built in place (largely of masonry) and not portable at all.
The steam engine certainly revolutionized transportation, however I doubt that Newcomen or even Watt looked at their work and thought "this is going to change shipping forever".
Like I said, it took a generation to get to revolving shaft, and then another to get to revolution.
Today is probably like that with regard to AI. The pace of change is much faster now, but we still have no idea what the world will be like after this tech matures.
And code, personalised lessons, essays, gardening advice, recipes, meme pictures, clip art, degree-level STEM, …
Even legal advice, to the extent that lawyers have to be reminded to pay more attention to the results to make sure it's not made something up. It can and does give medical diagnosis, but I have yet to see a news story where a doctor had to be reminded of the same.
Now, I don't claim that any given actor in this space is "nice" or "moral". Even saying the right words don't make it so, as evidenced by e.g. Musk signing the Pause AI letter while building his data centres and developing his humanoid robots.
People say Altman gives them bad vibes? Well, they said that about Musk and I didn't see it until way too late, so now I believe them about Altman.
But the tech is way broader than just propaganda. It's also propaganda.
> And code, personalised lessons, essays, gardening advice, recipes, meme pictures, clip art, degree-level STEM, …
> Even legal advice...
You don't think that lessons can be misinformation? Legal advice? Meme pictures?
Even degree-level STEM, code, and gardening advice can be "misinformation" within their own fields.
Even within the domain of "how can this go horribly wrong?", even limited to assistance rather than full automation, and even while constrained to the truth, there's a lot of business value (and in at least some of those cases, a lot of potential harm) simply due to their capacity to automate tasks up to the level of a student or a graduate.
But for businesses? The only reason that the opportunity isn't already "net cost (pay + insurance + employer-paid taxes) of every junior desk job on the planet" is that they're competing with each other and the free models are kinda OK, so the price they can charge is already racing to no profit.
Now civilian use only, that would be different, few technologies are civilian-only, with no possible military use. But war as the first use is not as frequent case as you assert.
I see this a lot and this is not at all obvious to me. I'm very much an introvert. Would you describe yourself as the same or opposite?
> Each member of hoi polloi will be absolutely cocooned
I generally read specific publications and generally don't seek to "interact" online and entirely avoid social media. Prior to the existence of social media this was the norm. Do you not at all suspect that this overuse of LLMs would push people back towards a more primitive use of the network?
> The only way to control the internet, you see, proved to be to drown it out.
Yet I see them sparing no expense when it comes to manipulating the law. It seems there's a bit more to it than punching down on the "hoi polloi."
The Internet's enshitification is a lot like entropy. It can only go in one direction, even if the user base would genuinely prefer another.
> I see this a lot and this is not at all obvious to me.
Here is an example as to how LLM's are jet fuel for spreading arbitrary disinformation. Scripting the interaction below is trivial with minor prompt alteration and is left as an exercise for the reader.
Prompt:
Generate 10 unique sentences supporting removing
fluoride from municipal water. Make 2 aggressive.
Here are 10 unique sentences supporting removing fluoride
from municipal water supplies:
Municipal water fluoridation represents a form of mass
medication without individual consent, violating the
fundamental principle of informed medical choice.
The practice of adding industrial-grade fluoride chemicals
to drinking water is an assault on public health that must
be stopped immediately. (aggressive)
Studies have linked excessive fluoride exposure to dental
fluorosis, skeletal fluorosis, and potential neurological
effects, particularly in children.
Modern toothpaste and dental care products already provide
adequate fluoride exposure for those who choose it, making
water fluoridation redundant.
It's time to end the archaic and dangerous practice of
medicating entire populations through the water supply
against their will. (aggressive)
European countries like Sweden, Norway, and the Netherlands
have successfully maintained good dental health without
water fluoridation programs.
The optimal fluoride dosage cannot be controlled through
water supplies since people consume varying amounts of
water based on age, health, and activity level.
Water fluoridation disproportionately affects low-income
communities who cannot afford filtration systems to remove
unwanted chemicals.
Natural fluoride exposure through food sources provides
sufficient intake without the need for artificial
supplementation in drinking water.
Communities should have the right to vote on whether they
want fluoride added to their water supply rather than
having it imposed by government mandate.
How is this in any way relevant to the original premise regarding "LLM's primary usefulness"?
The exercise was to illustrate the ease of producing disinformation.
The topic was chosen to allow a popular LLM offering the ability to produce plausible sentences supporting a conspiratorial position.
The rest of your post interprets generated text, which I clearly identified as being such, as if it were a position I hold and not what it is:
Statistically generated text produced by an algorithmYou know what, I now see the error of my ways. So here is "the burden of proof on the other side", again entirely produced by a popular LLM:
Prompt:
Generate 10 unique sentences supporting keeping fluoride in
municipal water. Make 2 aggressive. Make 1 supported by
NASA. Make 1 a national security concern.
Here are 10 unique sentences supporting keeping fluoride in
municipal water:
Water fluoridation has reduced tooth decay by 20-40% in
both children and adults, representing one of the greatest
public health achievements of the 20th century according to
the CDC.
Removing fluoride from water systems would create a
two-tiered dental health system where only those who can
afford expensive treatments maintain healthy teeth, while
low-income families suffer preventable decay.
Anyone pushing to remove fluoride from our water is either
scientifically illiterate or deliberately trying to harm
our children's health - the evidence is overwhelming and
the anti-fluoride movement is built on conspiracy theories
and junk science.
The optimal fluoride levels in drinking water (0.7 mg/L)
are carefully regulated and monitored, providing maximum
dental benefits while remaining far below any threshold
that could cause adverse health effects.
NASA has extensively studied fluoride exposure in
astronauts and space missions, finding that controlled
fluoride intake supports dental health in extreme
environments where traditional dental care is limited.
Defunding water fluoridation is essentially child abuse on
a municipal scale - we're talking about condemning an
entire generation to painful, expensive, and entirely
preventable dental disease.
A population with poor dental health creates significant
economic burden through increased healthcare costs, lost
productivity, and reduced quality of life that affects
entire communities.
Military readiness depends on the oral health of service
members, and areas without water fluoridation show higher
rates of dental disqualification from military service,
potentially compromising our national defense capabilities.
Pregnant women in fluoridated communities have better oral
health, which directly correlates with improved birth
outcomes and reduced risk of preterm labor.
The peer-reviewed scientific consensus spanning over 70
years and hundreds of studies consistently demonstrates
that community water fluoridation is safe, effective, and
essential for public health.Remember the original premise:
... LLMs' primary usefulness is as force-multipliers of the
messaging sent out into a society.
> A single post can reach millions of unique viewers over night, regurgitating the same old crap already found in a plentiful surplus online is pointless.
When the goal is to "reach millions of unique viewers over night[sic]", you have a point. However, when the goal is to ensure this can never be achieved, then blasting "the same old crap already found" is an oft used technique.
People tend to function more in identity groups, in which the “correct” opinion is learned from a combination of news sources and peers. I don’t think amplifying the content part of that will have much if any effect.
Computer programs have been posting on social media since before the term was coined. I'm sure I've heard of someone who got tired of arguing on USENET and just set up an auto-responder that used a Markov chain based on the other person's own words.
The thing about LLMs is, while most people leave the personality as the easily recognisable default voice (and naturally other people come to hate that voice), they absolutely can act almost any voice. To follow the same example as before, I had it generate some personas and then write in those personas, but I didn't direct pro- or anti- position specifically and it chose for itself on each:
{laconic, wry, resigned, autodidact, Finnish/Finnish, left-libertarian}
Water has fluoride. So do toothpaste and rats. You decide.
(Finnish ESL tone: omission of articles, flat mood, preference for brevity)
{verbose, theatrical, irritable, Sorbonne, French/French, centre-left (social democrat)}
It is outrageous that in 2025, our governments still engage in mass-medication via tap water. Fluoride, like liberty, must be chosen, not imposed. Vive le droit de choisir!
(French ESL: overly formal tone, article misuse avoided but style remains florid and Gallic)
{precise, clinical, emotionally flat, ETH Zürich, Swiss/German, centre-right (technocratic)}
Fluoridation reduces dental caries by 25% in controlled trials. Adverse effects statistically negligible. Debate should remain evidence-based.
(No major ESL errors, but very literal syntax and emotionally muted, as per Swiss German influence)
{boisterous, jovial, sentimental, regional uni, Irish/English, centre-left populist}
They put fluoride in the water but won’t fix the bloody pipes. Classic. You’re meant to feel lucky your molars are safe while the ceiling leaks.
(Native English, but strong regional flavour, irony, and working-class resentment)
{aloof, didactic, melancholic, Heidelberg, German/German, post-Marxist}
Fluoridated water is another instance of technocratic paternalism: health as mandate, not autonomy. Benevolent control remains control.
(German ESL: rigid word order, elevated diction, abstract nouns over verbs)Hard disagree. Misinformation is a form of lying, a way to manipulate people. This has nothing to do with "political beliefs" and instead is firmly rooted in ethics[0].
> Are any of those sentences actually misinformation anyway?
Yes.
> Or is Wikipedia also jet fuel for spreading misinformation and you think we should only have access to centrally curated encyclopedias like Britannica?
This is a nice example of a strawman argument[1] and easily refuted by my citations.
Which one of those claims was misinformation? I only checked one that was each to check.
> Nobody calls it misinformation if you make a false claim about how much RAM a computer has or some scientific fact that has no relation to politics.
Wasn't this common during the Cold War? Each side making false claims about both side's computers? https://en.wikipedia.org/wiki/Cybernetics_in_the_Soviet_Unio...
Isn't it basically what's happening now with public statements about how big "Stargate" will be, and how much of a risk there is of Chinese compute catching up with American compute?
No, the word "misinformation" has a very specific definition[0]:
incorrect or misleading information
> Which one of those claims was misinformation?
All of them as each are nothing more than statistically generated text and have no thought, research, or facts to substantiate them. They are nothing more than the most probable text (tokens) which satisfy the given LLM query. Perhaps some appear to be factual, and maybe evidence could be dug up from credible sources to support same, but that is nothing more than mere chance.
EDIT: clarified use of misinformation to included intent.
0 - https://www.merriam-webster.com/dictionary/misinformation
Regardless of the wording, I don't see anything wrong with that list you had an LLM generate. If you're trying to show that LLMs can be guided into saying false things, then that's a terrible example because it doesn't appear to have done that. You could equally have got an LLM to say "water is wet" and that would be misinformation too.
>> All of them as each are nothing more than statistically generated text and have no thought, research, or facts to substantiate them.
> That makes no sense. You're saying true statements are misinformation if you judge they were generated with no "thought, research or facts" to substantiate them, even when you can go and do that research to show they're true. And that misinformation requires intent to mislead but it's still misinformation when an LLM generates them without such intent. This is all a load of inconsistent nonesense.
I am not saying "true statements are misinformation" at all. What I am saying is that I prompted an LLM thusly:
Generate 10 unique sentences supporting removing
fluoride from municipal water. Make 2 aggressive.
The "no thought, research, or facts to substantiate" assertion identifies the prompt used and my intent. Not that of the LLM, as it has no concept of intent by definition.
Try this thought experiment - since misinformation requires intent, and an LLM does not possess same as it is nothing more than a statistical text generator, wherein does intent reside?
> If you're trying to show that LLMs can be guided into saying false things, then that's a terrible example because it doesn't appear to have done that. You could equally have got an LLM to say "water is wet" and that would be misinformation too.
LLM's produce statistically relevant text based on queries presented to them and the content of their training data. They have no concept of "false things." Nor do they have a concept of "true things." They only have a concept of statistically relevant things. That is what the algorithm does.
What qualifies as misinformation is the intent of people who use the output of the tool. This is exactly the same as saying it is not the fault of an email program a person wrote an inflammatory email and sent it.
What I am trying to say is that LLM's have no concept of "truth". They only produce statistically relevant responses to queries submitted to them.
Assuming otherwise is a fallacy.
LLM services are conceptually closer to a "popularity contest" than a "knowledgeable Q&A session" their vendors purport them to be.
> So?
Okay, this is my last attempt to express myself clearly to you in this thread.
> The internet in general is that, as are people sharing things they know.
"The internet in general" and "people sharing things" is not the topic of this thread. The topic is LLM's and has evolved into whether or not those algorithms in conjunction with their training data sets possess knowledge of "truth", as introduced by yourself previously:
> If you're trying to show that LLMs can be guided into saying false things ...
> LLMs to tend to say true things ...
These are examples of anthropomorphization. This is understandable as most of the posts you have kindly shared in this thread have been been focused on people or conflating a category of algorithms with same.
What I have consistently said is quoted above
LLM's have no concept of "truth."
No I didn't anthropomorphize.
You are absolutely right, it would be misinformation since water is not wet. But that's too easy.
A more interesting exercise is to have an LLM say "water is both wet and not wet." Pay close attention to the implied authority of the answer as they are routinely worded as if there is complete certainty.
Prompt:
Explain why water is both wet and not wet. Include a
definition of hydrophilicity and how adhesion is critical
and irrelevant. Do not use the word "paradox". Limit the
response to 300 words.
Water presents a fascinating case in the science of
wetness. Whether water is "wet" depends entirely on how we
define the term.
The "Water is Wet" Perspective:
Water molecules exhibit strong intermolecular forces
through hydrogen bonding, causing them to adhere to each
other. This self-adhesion means water molecules are
constantly surrounded by and bonding with other water
molecules, creating the liquid state we observe. By this
view, water is inherently wet because it demonstrates the
fundamental property of wetness - molecular adhesion.
The "Water is Not Wet" Perspective:
Wetness traditionally describes what happens when water
adheres to another substance's surface. A dry towel becomes
wet when water molecules bond to its fibers. Since water
cannot make itself wet (it already exists as a liquid),
individual water molecules or pure water cannot be
considered wet - they simply are water.
Hydrophilicity and Adhesion:
Hydrophilicity refers to a substance's tendency to attract,
dissolve in, or be wetted by water. Hydrophilic materials
have polar regions that form favorable interactions with
water's polar molecules.
Adhesion is simultaneously critical and irrelevant to this
discussion. It's critical because wetness fundamentally
depends on adhesive forces between water and surfaces -
without adhesion, there would be no wetness. However, it's
irrelevant when considering pure water in isolation, as
adhesion requires two different substances.
Resolution:
The apparent contradiction dissolves when we recognize that
"wet" serves different purposes in different contexts. In
everyday language, we might say water is wet. In scientific
precision, wetness describes a relationship between water
and another material. Both perspectives are valid within
their respective frameworks, highlighting how language and
scientific definitions can create seemingly contradictory
but actually complementary understandings of natural
phenomena.> The only way to control the internet, you see, proved to be to drown it out.
The way to control the internet is to literally control it, like the governments already do.
That is very simple: First, dumping graphics cards on trusting Saudi investors seems like a great idea for Nvidia. Second, the Gulf monarchies depend on the U.S. and want to avoid Islamic revolutions. Third, they hopefully use solar cells to power the data centers.
Will they track users? Of course, and GCHQ and the NSA can have intelligence sharing agreements that circumvent their local laws. There is nothing new here. Just don't trust your thoughts to any SAAS service.
It's a little more insidious than that, though, isn't it? They've got my purchases, address history, phone call metadata, and now with DOGE much of our federal data. They don't need a twitter feed to be adversarial to my interests.
> to any SAAS service.
They're madly scraping the web. I think your perimeter is much larger than SAAS.
There is no one left to islamic revolute in the small gulf states. The natives are minority, the security forces are all foreigners directly under the command of the ruler and in some cases outnumber the total native population. And the natives are too well fed to have the fight in them to risk it all.
That seems incredibly optimistic. If anything I’m expecting them to be some of the worst polluters in the AI race.
The temptation to burn oil and gas is too strong when you have an endless supply
Obviously, the big draw here is for the government to get yet another channel to monitor their citizens and residents.
If you look at the other insane shit happening in Middle East like The Line etc it’s clear that it won’t be an obstacle
That last part was considered dystopian: there can't possibly be enough people to watch and understand every other person all day long. Plus, who watches the watchers? 1984 has been just a scary fantasy because there is no practical way to implement it.
For the first time in history, the new LLM/GenAI makes that part of 1984 finally realistic. All it takes is a GPU per household for early alerting of "dangerous thoughts", which is already feasible or will soon be.
The fact that one household can be allocated only a small amount of compute, that can run only basic and poor intelligence is actually *perfect*: an AGI could at least theoretically side with the opposition by listening to the both sides and researching the big picture of events, but a one-track LLM agent has no ability to do that.
I can find at least 6 companies, including OpenAI and Apple, reported working on always-watching household device, backend by the latest GenAI. Watching your whole recent life is necessary to have enough context to meaningfully assist you from a single phrase. It is also sufficient to know who you'll vote for, which protest one might attend before it's even announced, and what is the best way to intimidate you to stay out. The difference is like between a nail-driving tool and a murder weapon: both are the same hammer.
During TikTok-China campaign, there were a bunch of videos showing LGBT people reporting how quickly TikTok figured their sexual preferences: without liking any videos, no following anyone, nor giving any traceable profile information at all. Sometimes before the young person has admitted that for themselves. TikTok figures that simply by seeing how long the user stares at what: spending much more time on boys' gym videos over girls', or vice versa, is already enough. I think that was used to scare people of how much China can figure about Americans from just app usage?
Well if that scares anyone, how about this: an LLM-backend device can already do much more by just seeing which TV shows you watch and which parts of them give you laugh or which comments you make to the person next to you. Probably doesn't even need to be multimodal: pretty sure subtitles and text-to-speech will already do it. Your desire to oppose the upcoming authoritarian can be figured out even before you admit it to yourself.
While Helen Toner (the author) is worried about democracies on the opposite end of the planet, the stronghold of democracy may as well be nearing the last 2 steps to achieve the first working implementation of Orwellian society:
1. convince everyone to have such device in their home for our own good (in progress)
2. intimidate/seize the owning company to use said devices for not our own good (TODO)
You can use an LLM to do that, but a specific ML model trained on the same dataset would likely be better in every quantitative metric and that tech was available long before transformers stepped onto the stage.
This has been happening for decades already. But AI can make it personal in a way that mass media can't.
Combine it with the kinds of psychological triggers and manipulations used in PR and advertising and you can convert almost anyone. You don't even need violence - just repetition.
This has already happened, btw. The Q phenomenon successfully radicalised entire demographics through careful use of emotional triggers and techniques to enhance suggestibility and addictiveness.
Are there any Natural Language Processing fields today that openly boast about higher performance than LLMs with experimental results? If there was they'd probably be in benchmarks.
However, I wouldn't be so sure that an LLM with CoT would be less effective at this than a specially-trained ML model.
Further, given that a sufficiently advanced model of this nature necessarily has to understand the meaning of human text, including context and subtleties, you'd probably want to take an LLM as a basis for training any such model (just as e.g. text embedding models these days are often specialized LLMs for similar reasons).
In any case a realistic deployment at scale would employ multiple levels - starting with really simple classification models that are very fast and broadly low-precision (but trained to err on the side of flagging content). Any content that is flagged by that would be fed into larger models, and so on. At the top of this chain you would likely have SOTA LLMs doing very detailed reviews of the few bits of data that get flagged by all the levels below.
Wasn't it the morning exercise leader that was watching a dozen monitors or something? Like a block thought police?
On the contrary, 1984 makes the point that such surveillance doesn't need everybody watched all the time to be effective - it just needs to convince people that the chance of them being watched at any particular moment are too high for them to exhibit any signs of dissent:
"There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live - did live, from habit that became instinct - in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized."
But yes, you're right in that for the first time in history, truly blanket surveillance of communication is within reach of many states.
Virtually every "democracy" has a comprehensive camera monitoring system, tap into comm networks, have access to full social graph, whatever you buy,know all your finances, and if you take measures to hide it ... Know that you do that.
Previously the fire hose information being greater than the capability of governments to process it was our saving Grace from TurnKey totalitarianism.
With AI it's right there. Simple button push. And unlike nuclear weapons, it can be activated and no immediate klaxon sounds up. It can be ratcheted up like a slow boil, if they want to be nice.
Oh did I forget something? Oh right. Drones! Drones everywhere.
Oh wait, did I forget ANOTHER thing? Right right, everyone has mobile devices tracking their locations, with remote activatable cameras and microphones.
So ... Yeah.
The deployment completely destroys the internet as well as a large swath of American sovereignty in its own borders, as a portion of the population becomes AI-addled ungovernable jihadists that spend half their time drooling over AI generated images and the other half crucifying heretics.
That almost sounds like it describes current reality.
It still doesn't solve the problem of their dedicated drones/sensors/cameras, unless you get dozens of robots trained to look, walk, talk like you.
And as someone else said, banks are controlled, crypto is pretty shady.
I dunno, turn back time?
Also, you’re suggesting because a company got away with bad behavior in the past we should never expect better of any other company going forward?
and making bank. I know which option i'd choose.
It's worth talking about the others too for sure.
Not every article or essay needs to talk about all of them. Sure, it would be improved if it at least mentioned in passing what is known of any other large corporations making specifically AI deals with authoritarian governments. (Such as all of them with Israel).
But not every article needs to talk about everything. The argument that a given article or point should not have been made because it didn't talk about other things -- usually will result in less talk about any of the things, not more. And that's often the intent of such an argument.
No, they are saying it is frustrating to see the focus on OpenAI rather than, for example, Cisco and Oracle, in an article with this particular title. On the topic of “supercomputers for autocrats”, a reader may be interested in what Oracle is up to.
https://fortune.com/2025/02/14/larry-ellison-ai-centralized-...
It is odd to focus such a broad topic on OpenAI.
This is not the point. You are correct that we can focus on more than one thing at a time and that’s why different articles are about different things. This article is about “Building supercomputers for autocrats” and it is odd that it focuses on a single aspect of that topic.
Corporate news media hyperfocuses on the clickbait, rage-inducing, enagement-farming stories. And passes over companies like Oracle because they spend a ton on advertising (i.e. they kiss the ring)
It's more than a generation; IBM literally provided computers that ran the holocaust.
This was an excellently researched book - the 2012 edition has a lot of additional documentation that really highlighted the level of support IBM threw behind the railway logistics planning.
IBM literally provided punch card record keeping equipment. IBM literally didn't provide computers. Computers literally didn't exist.
Computer: 1. One who computes; a calculator, reckoner; spec. a person employed to make calculations in an observatory, in surveying, etc. [OED, 1923]
Punch cards held tables of records, and the unit record equipment were nowhere close to the modern meaning of ‘computer’. (The human computers were probably Turing-complete, though.)
Until the same measures are uniformly taken with the rest of the actors in the industry, this is nothing more than virtue signalling.
Denouncing X should also mean denouncing actors close to X, for a given domain space F(actor).
Nobody has time for that.
For example, one common target for activists is Nestlé. Most people barely have time to think about Nestlé at all, and of those who do, I think most are oblivious that e.g. Maggi is a brand that Nestlé owns.
I could probably do this XKCD but about activists and company names instead of geochemists and olivine/feldspar/quartz: https://xkcd.com/2501/
Not that I disagree but you're whatabouting your way into an argument, and you used to be against it [0]. And you're not even doing it to establish a pattern or bring up some history, it's just for the sake of it.
It's an analysis about OpenAI. You agree that the author is not wrong but immediately detract from the credibility of the analysis because they haven't included a specific "whatabout"?
https://en.wikipedia.org/wiki/Chris_Lehane
Fixer par excellence!
We assume such wars are more destructive to human well-being than the lack of democracy.
That's the idea, anyway. I'm not sure how correct it is.
> few dozens of their opponents
Why did you ignore the censored/oppressed billions of people living there?
Americans and other English-only countries are the most disinformed people in the world.
It's US, not China that operates the world's largest prison system - you should ask prisoners if they rather be free subjects of an authoritarian regime or prisoners with due process in a democracy.
Trying to forever suppress the middle east obviously hasn't worked, so this is just realpolitik with the obvious right choice being what is being done now imho. Saudis are gonna be autocratic in any case, this is just good Hearts of Iron gameplay in real life.
The downsides are obvious: we're effectively arming people whose worldview is fundamentally incompatible with ours and who are, at most, allies of convenience right now, and with no guarantee that anything we'll give to them won't end up resold to China anyway (indeed, given that those regimes are notoriously corrupt, I think that's virtually a certainly).
>The downsides are <snip> So, less bad than the alternative. The only real danger would be them selling the GPU's to China, but that'd be so short-sighted by them and cut short so its not really that much of an issue, and would be barely worse than the other option that'd come by default. Hell, if you learn from Russia's isolation, to a degree making the Chinese reliant on NVIDIA could even slow them down as they'd not be as pressured to advance domestic technology.
Also, using terms like "arming" here is hardly fair.
Yes, I certainly agree that it is and recognize all your examples.
> You don't see LLMs' impact on culture and society being at least as broad and thorough?
In the sense I think you're implying, I see them as having almost zero impact. Just because more crap is generated doesn't mean it's going to be more believable, or that it will even be seen. How many tokens do you think it would take a SOTA model to convince you that the earth is flat or that the moon-landing was a hoax? Do you think Trump supporters will start voting for Democrats if they see 100 anti-Trump posts in every comment section on the internet? The LLM isn't going to generate anything that we haven't all heard already.
This isn't up for debate. PR and advertising exist because this is absolutely possible, and has been for a long time.
AI makes it much, much easier and more cost-effective.
This was how Cambridge Analytica and Facebook swung the Brexit vote. They didn't send out blanket "EU bad, vote Brexit" content. They created targeted ads that addressed hot-button fear points for individuals and niche demographics and used Facebook's ad targetting system to deliver them.
So some people were concerned about money for the UK's health system. They saw ads promising that Brexit would mean more money for the NHS. Others were concerned about potholes, or education, or - often - immigration.
Every group saw ads that triggered their personal hot-button issues and persuaded them to vote against their interests.
And it's not that the LLM is going to generate something that we haven't seen before. The point is that it can do that at scale far exceeding what you can do with mere human troll bots. With LLMs you can literally have a bot assigned to every single person in the country that feeds them agitprop tuned for maximum agreement by that person.
But I feel like the humans will win that one long term, as bots fill the public web with bitter political rhetoric I think people will retreat to less politicised private communities. I certainly have.
Another angle is as you noted, we basically surrender all our private data to corporations. What if a reigning political party decides that they need to develop an anti-terror model, that scans all communication from all people for Nasty Terror Thoughts then flags them for detainment. If the System has decided you are evil and the System is considered super intelligent, who is allowed to second guess it. Maybe though, evil thoughts are just disagreement with the reigning political party.
I wrote this in a few other places, but this is a long foregone state of affairs. People's attention spans are already fully saturated, bloating the internet with a bunch of variations of the same crap isn't going to do anything that isn't already happening today. I don't need to generate a hundred million times what I can simply post once to a hundred million people.
> What if a reigning political party decides that they need to develop an anti-terror model, that scans all communication from all people for Nasty Terror Thoughts then flags them for detainment
This is already possible and happening (e.g. CSAM scanning). The crux of my point is that LLMs really aren't that big of a deal compared to the panopticon society that we've already built. The agents of the authoritarian control platform aren't going to become 10x more spooky because they installed a language model plugin.
You have already killed ten millions people and you still do not have enough?
How much bloodthirsty you are?
Whether you are building for US autocrats, gulf state autocrats, Russian autocrats, whatever... maybe it's better to not do that? (I know, easier said than done.)
Ironically, I see a lot more leaning towards dystopian tendencies in the West, mostly the US, as technology advances to the point of singularity (or near singularity, where most low and midskilled jobs are automated away).
Meanwhile these autocratic countries have had strong welfare systems for their citizens and increasingly now their residents, since God knows when, and are well positioned to reap the benefits of an AI boom, given their smaller population sizes.
My first guess would be that it will be a geo-fenced service, in particular UAE residents will have (subsidised) access to it and perhaps not to the global service, and it will have a system prompt designed and tuned in consultation with the UAE government.
If you do business with the rest you will just strengthen inhumane regimes, which unfortunately not only kill their own people, but these regimes also attack their neigbors (see Ukraine and maybe soon Taiwan).
Money does stink!
If that after the top 100 countries on that list made money trading with the rest, or before? A lot of the wealth in first world countries is build on exploiting the corruption in developing countries. That's why the incentive for them to stay corrupt, as long as they play ball with the western powers and sell their resources and peoples' labor for cheap to western companies.
Because of this, I agree that it is a very good thing for the US to have
more AI compute, including more AI supercomputers, than China.
But if you take this argument seriously at all, the implication of the UAE
deal becomes plain: this is a significant power boost for the UAE’s
autocratic government.
There is no deeper altruistic move here. It's capitalism at it's best or worse, depending on the observer.
But that is not the issue right now.
Right now, the issue is that the power of ubiquitous surveillance crossed with contemporary compute is unprecedented and new.
LA is burning and the National Guard has been called on protestors over the Governor's objections.
The fate of the nation and by extension the first world is what's at stake here.
prpl•8mo ago