Imagine the ads on TV: "Has AI lied about you? Your case could be worth millions. Call now!"
- https://www.msba.org/site/site/content/News-and-Publications...
- https://www.reuters.com/legal/government/judge-disqualifies-...
- https://calmatters.org/economy/technology/2025/09/chatgpt-la...
Simple example: A prospective employer refuses to hire you because of some blatant falsehood generated by an LLM.
At least once a week, there is another US court case where the judge absolutely rips apart an attorney for AI-generated briefs and statements featuring made-up citations and nonexistent cases. I am not even following the topic closely, and yet I just encounter at least once a week.
Here are a couple most recent ones I spotted: Mezu v. Mezu (oct 29)[0], USA v. Glennie Antonio McGee (oct 10)[1].
0. https://acrobat.adobe.com/id/urn:aaid:sc:US:a948060e-23ed-41...
1. https://storage.courtlistener.com/recap/gov.uscourts.alsd.74...
Big tech created a problem for themselves by allowing people to believe the things their products generate using LLMs are facts.
We are only reaching the obvious conclusion of where this leads.
On the other hand, training a small model to hallucinate less would be a significant development. Perhaps with post-training fine-tuning, after getting a sense of what depth of factual knowledge the model has actually absorbed, adding a chunk of training samples with a question that goes beyond the model's fact knowledge limitations, and the model responding "Sorry, I'm a small language model and that question is out of my depth." I know we all hate refusals but surely there's room to improve them.
I always thought that was a correct and useful observation.
Obviously it's asking for a lot to try to cram more "self awareness" into small models, but I doubt the current state of the art is a hard ceiling.
This has already been tried, llama pioneered it (as far as I can infer from public knowledge, maybe openai did it years ago I don't know).
They looped through a bunch of wikipedia pages, made questions out of the info given there, posed them to the LLM and then whenever the answer did not match what was in wikipedia, they went ahead and finetuned on "that question: Sorry I don't know ...".
Then, we went one step ahead, and finetuned it to use search in these cases instead of saying I don't know. Finetune it on the answer toolCall("search", "that question", ...) or whatever.
Something close to the above is how all models with search tool capability are fine tuned.
All these hallucinations are despite those efforts, it was much worse before.
This whole method depends on the assumption that there is actually a path in the internal representation that fires when it's gonna hallucinate. The results so far tell us that it is partially true. No way to quantify it of course.
To fix that properly we likely need training objective functions that incorporate some notion of correctness of information. But that's easier said than done.
But all of their output it literally "made up". If they didn't make things up, they wouldn't have a chat interface. Making things up is quite literally the core of this technology. If you want a query engine that doesn't make things up, use some sort of SQL.
> The consistent pattern of bias against conservative figures demonstrated by Google’s AI systems is even more alarming. Conservative leaders, candidates, and commentators are disproportionately targeted by false or disparaging content.
That's a little rich given the current administration's relationship to the truth. The present power structure runs almost entirely on falsehoods and conspiracy theories.
I think plausiblibly they might, through no fault of Google, if only because scandals involving conservatives might be statistically more likely.
https://en.wikipedia.org/wiki/Stephen_Colbert_at_the_2006_Wh...
0: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
I don't think so, no.
> just an accurate description of party members as a whole?
It wouldn't be. While enough republicans have gotten caught being gay to remove the element of surprise and plausibly be the basis of LLM hallucinations, most of them haven't been, so such an LLM hallucination wouldn't actually be accurate, merely unsurprising.
One potential solution to the accuracy problem is to turn facts into a marketplace. Make AIs deposit collateral for the facts they emit and have them lose the collateral and pay it to the user when it's found that statements they presented were false.
AI would be standing behind its words by having something to lose, like humans. A facts marketplace would make facts easy to challenge and hard to get right.
Working POC implementation of facts marketplace in my submissions.
AI != LLMs. A future version of better AI might become capable of saying "I don't know."
The mess of figuring out what the facts are, in a way, already has deployed solutions. Journalists are a form of fact checkers. Big news companies with multi-million budgets pay journalists to check things. In effect there already is a market for fact-checking. Managing perceptions by twisting facts is something many have been accused of. When news companies are found to have spread lies they possibly face penalties.
So if the big clunky enterprise version of a form of fact-checking is already in place and kind of working, why would something closer to the lean startup-y tail-end of the distribution of fact-checking not work? If a version of a facts marketplace is already working, why doubt it would ever work?
X paying users for engagement is a new variation of this. Prediction markets like Polymarket is another. Polymarket teaming up with TruthSocial could be an interesting result.
It's okay to have many contested subjects our society cannot agree on and which don't lend themselves to scientific inquiry. It's okay to say I don't know.
Unfortunately, it's also worth pointing out that neither Marsha Blackburn nor Robby Starbuck are reliable narrators historically; nor are they even impartial actors in this particular story.
Blackburn has a long history of fighting to regulate Internet speech in order to force them to push ideological content (her words, not mine), so it's not surprising to see that this story originated as part of an unrelated lawsuit over First Amendment rights on the Internet and that Blackburn's response to it is to call for it all to be shut down until it can be regulated according to her partisan agenda (again, her words, not mine) - something which she has already pushed for via legislation that she has coauthored.
Yes, I understand that this was not the intended use. But at some point if a consumer product can be abused so badly and is so easy to use outside of its intended purposes, it's a problem for the business to solve and not for the consumer.
But fundamentally the reason ChatGPT became so popular as opposed to its incumbents like Google or Wikipedia, is that it dispensed with the idea of attributing quotes to sources. Even if 90% of the things it says can be attributed, it's by design that it can say novel stuff.
The other side of the coin is that for things that are not novel, it attributes the quote to itself rather than sharing the credit with sources, which is what made the thing so popular in the first place, as if it were some kind of magic trick.
These are obviously not fixable, but part of the design. I have the theory that the liabilities will be equivalent if not greater to the revenue recouped by OpenAI, but the liabilities will just take a lot longer to realize, considering not only the length of trials, but the length of case law and even new legislation to be created.
In 10 years, Sama will be fighting to make the thing an NFP again and have the government bail it out of all the lawsuits that it will accrue.
Maybe you can't just do things
Calling it "AI", shoving it into many existing workflows as if it's competently answering questions, and generally treating it like an oracle IS being neglectful.
Uhhh… net positive for who exactly?
Or am I not following your logic correctly?
Let's take a weaker example, some sugary soda. Tons of people drink sugary sodas. Are they truly a net benefit to society, or a net negative social cost? Just pointing out that there are a high number of users doesn't mean it inherently has a high amount of positive social outcomes. For a lot of those drinkers, the outcomes are incredibly negative, and for a large chunk of society the general outcome is slightly worse. I'm not trying to argue sugary sodas deserve to be completely banned, but its not a given they're beneficial just because a lot of people bothered to buy them. We can't say Coca-Cola is obviously good for people because its being bought in massive quantities.
Do the same analysis for smoking cigarettes. A product that had tons of users. Many many hundreds of millions (billions?) of users using it all day every day. Couldn't be bad for them, right? People wouldn't buy something that obviously harms them, right?
AI might not be like cigarettes and sodas, sure. I don't think it is. But just saying "X has Y number of weekly active users, therefore it must be a net positive" as some example of it truly being a positive in their lives is drawing a correlation that may or may not exist. If you want to show its positive for those users, show those positive outcomes, not just some user count.
How confident are you that 800M people know what the negative aspects are to make it a net positive for them?
It's worth noting too that how we talk about and use AI models is very different from how we talk about other types of models. So maybe it's not surprising people don't understand them as models.
It told me it can't and I could do it myself.
I told it again.
Again it told me it can't, but here's how I could do it myself.
I told it it sucks and that ChatGPT etc. can do it for me.
Then it went and I don't know, scrapped Airbnb or used a previous search it must have had, to pull up rooms with an Airbnb link to each.
…
After using a bunch of products I now think a common option they all need to have is a toggle between "Monkey's Paw" mode: Do As I Say, vs a "Do What I Mean" mode.
Basically where the user takes responsibility and where the AI does.
If it can't do or isn't allowed to do something when in Monkey Paw mode then just stop with a single sentence. Don't go on a roundabout gaslighting trip.
If it could do what you're suggesting, it wouldn't need to do it in the first place.
You need to gate away useful technology from the normies, usually. E.g. kickstarter used to have a problem where normies would think they were pre-ordering a finished product and so they had to pivot to being primarily a pre-order site.
Anything that is actually experimental and has less than very high performance needs to be gated away from the normies.
Especially in parliamentary democracies where people already take political quizzes to make sense of all the parties and candidates on the ballot.
So uninformed people participating isn't an unfortunate side effect, but rather the point: making everybody feel included in the decision making processes, to make people more likely to accept political change.
"I think we do democracy not because we think the masses are informed and make good decisions, but rather because it's the best system for ensuring peaceful transitions of power, thereby creating social stability which is conducive to encouraging investment in the future.
The lack of warlords leads to peaceful transitions. Trump can feel all he wants about the 2020 election but his sphere of influence was too small to take control.
This isn't the case for all those power struggles when a monarch dies. Each Lord had their own militia they could mobilize to take control and leads to stuff like War of the Roses.
We had this same issue going into the Civil War where the US army was mostly militias so it's pretty easy to grab the southern ones together and go fight the north. This isn't going to work so well post-1812 where a unified federal army exists. Of course, if you start selectively replacing generals with loyalists then you start creating a warlord.
"Let me ask Grok who I should vote for..."
what on earth??
practically every metropolitan area and tons of smaller communities have multiple news sources that publish "voting guides" in addition to voter pamphlets that go out before elections which detail candidates positions, ballot initiatives etc.
barring that you can also just... do your "frantic googling" before the election. it's not a waste of your time to put a little of it toward understanding the political climate of your area and maybe once in a while forming an opinion instead of whatever constitutes a "moderate" position during the largest rightward shift of the overton window in decades.
Maybe the best defense of voting is that there are so many reasons people vote that it is hard to manipulate everyone in the same way.
Of course, that is historically. Voting is quite compromised at this point no matter how you slice it.
"Renfield, you are free now. Yes, master." /s
Why do need a licence to drive but not to vote? Except non-birth citizens effectively had to acquire a licence.
Philosophers have probably thought about this and can provide an answer. Unfortunately I'm not familiar with their literature.
I like Rawls' Veil of Ignorance as a principle for thinking about issues of fairness. Would I choose to live in a society where only informed citizens could vote even if that might preclude me? Probably. Hard to honestly put oneself behind that veil though.
Why ? Don't worry, everything will be fine. Sincerely, FAANG
croemer•3mo ago