And a lot of us would be better off releasing our dumb ideas too. The world has a lot of issues and if all you do is talk down and don't try to fix anything yourself. Maybe it's time to get off the web a little and do something else.
He's a horrible human being but has had a couple worthy ideas.
Given his later attitude when it came to careful thought, I'm no longer under the impression that these earlier expositions were his ideas at all. I suspect he got it from the engineers and used it to burnish his image. I know that certain companies, e.g Apple, Dyson, etc have a culture of "all ideas came from the big man at the top, no matter who thought of it."
One wishes Musk would take this advice: leave the web alone, forget for a few months about the social media popularity contest that seems to occupy his mind 24/7, and focus on rekindling his passion for rockets or roadsters or whatever middle-aged pursuit comes next.
I'm not saying that Musk is doing the same; but that one can be charitable and say he probably did not mean that. I mean, what does he stand to gain from doing so? He's a businessman.
Whew lad. This tells me all we need to know. "I don't know nothing but folks gotta listen to my opinion!"
I can only guess at his motives, but the salute is not an isolated case. Steve Bannon has given the same salute multiple times, so it seems coordinated.
Musk has tweeted “Only AfD can save Germany”. The founder of AfD, Björn Höcke, is a convicted nazi. The German domestic intelligence agency, BfV, says AfD is an extreme-right organization with anti-democratic ideals (“proven far-right extremist entity”)
Musk also tweeted “Free Tommy Robinson”, a UK far-right extremist activist and convicted criminal.
Musk has a history of supporting people and organizations that most other businessmen would not.
[0]: https://www.jta.org/2025/01/21/politics/how-did-the-adl-conc...
... except if you try to go against the group think. My edits have been reverted and I have been banned simply because I decided to speak up about the good things that the Modi government in India was doing, of which there are many. I don't believe in always harping on the negative, and am generally an optimistic person. But I went against the dominant narrative on Wikipedia and was rejected with vehemence.
> Me: Holy shit! You were censored for wanting lower taxes?
> Con: LOL no...no not those views
> Me: So....deregulation?
> Con: Haha no not those views either
> Me: Which views, exactly?
> Con: Oh, you know the ones
Unfortunately always relevant.
I pointed out that India had reduced extreme poverty from ~16% in 2013 to ~2% in 2022. This is directly from a World Bank report: https://documents1.worldbank.org/curated/en/0997221042225345... One would think this would be reliable? One would be wrong since this is deemed to be "pro Modi". Another was the success of India's mission to bring piped water to every household in India (a luxury we take for granted in the West). There's a live dashboard maintained by the Water Ministry of India: https://ejalshakti.gov.in/jjmreport/JJMIndia.aspx Apparently it's a biased source. :shrug:
Other examples include the fact that Indian government paid for millions of households to construct toilets over the last 10 years. Or that millions of houses were constructed in villages, fully paid for by the government, during the same period. Or that the government also paid for rural folks to switch to gas-burning stoves instead of wood-, coal- or cowdung- burdning stoves. I'm too lazy to look up the references now, but you can find them with easy searching.
Surely you can't deny that Wikipedia is biased against the so-called "right" side of the political spectrum, and biased towards the "left"?
Its also possible the sources might have been fine but OP's interpretation of them was not. For example if he was using them to support something they didn't say or drawing his own conclusions from them beyond what the text of the source says.
This is all speculation of course. If OP provided his username we would be able to see for sure as it would be a matter of public record.
For reference both of the urls OP cites are currently used on Wikipedia. The first in the article economy of India, the second on the article for Jal Jeevan Mission, so at least in modern times Wikipedia is ok with those sources
It happened to me a number of times before I came to understand the Wikipedia model. They are not interested in your analysis, they are interested in statements that reference other inspectable "reputable" analyses.
This is the best endorsement for wikipedia possible.
If I were doing a project like this, I would hire a few dozen topical experts to go over the WP articles relevant to their fields and comment on their biases rather than waste their time rewriting the articles from scratch. The results can then be published as a study, and can probably be used to shame the WP into cleaning their shit up, without needlessly duplicating the 90% of the work that it has been doing well.
what if your goal is for wikipedia to be biased in your favor?
One thing I love about the Wikipedias (plural, as they're all different orgs): anyone "in the know" can very quickly tell who's got no practical knowledge of Wikipedia's structure, rules, customs, and practices to begin with. What you're proposing like it's some sort of Big Beautiful Idea has already been done countless times, is being done, and will be done for as long as Wikis exist.
And Groggypedia? It's nothing more but a pathetic vanity project of an equally pathetic manbaby for people who think LLM-slop continously fine-tuned to reflect the bias of their guru, and the tool's owner, is a Seal of Quality.
No shit; it's always been that way since mass media became a thing. Besides, there is no such thing as quality conservative and/or "anti-woke" media. The very concept represents a contradictio in adiecto. And Elon's just the modern version of an industrialist of yesteryear. Back in the day they owned the mass media of their time: radio and television. Today its "AI"-enshittified parasocial media and ideally the infrastructure that runs those dumps.
> "Don't forget that public opinion and the media landscape are quite different in 2025 from what they were in the 2010s when most prior studies on WP bias have been written."
Bias studies have been written since Wikipedia became a staple in hoi polloi's info diet. And there's always been a whole cottage industry of pathological and practised liars (e. g. the Heritage Foundation, amongst others) catering to right-wing grievance issues. The marked difference is that the right's attacks against Wikipedia as an institution are more aggressive since Trump... completely in line with the more aggressive attacks on human rights, reason, science, and democratic institutions on part of conservatives world wide.
In the context of my argument a distinction without difference.
> "I am including the occasional "course correction" opeds and actually well-researched longreads you're seeing in places like NYT, Atlantic and such."
Well, that's the crux: There is no such thing for me as "actually well-researched anti-woke content". That's just a pathetic, and ultimately tragic, hallucination in the same vein as "actually well-researched" pieces of flat earthers, pushing their trash. Et cetera.
> "Elon's personality isn't of importance here [...]"
I can tell you're one of those guys who paid "actually a lot of" attention when The Cult of Personality was negotiated in the classroom.
Just as LLM's lack the capacity for basic logic, they also lack the kind of judgment required to pare down a topic to what is of interest to humans. I don't know if this is an insurmountable shortcoming of LLM's, but it certainly seems to be a brick wall for the current bunch.
-------------
The technology to make Grokipedia work isn't there yet. However, my real concern is the problem Grokipedia is intended to solve: Musk wants his own version of Wikipedia, with a political slant of his liking, and without any pesky human authors. He also clearly wants Wikipedia taken down[1]. This is reality control for billionaires.
Perhaps LLM generated encyclopedias could be useful, but what Musk is trying to do makes it absolutely clear that we will need to continue carefully evaluating any sources we use for bias. If Musk wants to reframe the sum of human knowledge because he doesn't like being called out for his sieg heils, only a fool would place any trust in the result.
[1]https://www.lemonde.fr/en/pixels/article/2025/01/29/why-elon...
Not to beat a dead horse, but one really could wake up one day and find out we've always been at war with Oceana after the flip of a switch in an LLM encyclopedia.
Asking an LLM to reprocess it again is only going to add error.
Completely agree with the first purpose but would never use wikipedia for the second purpose. Its only good at basics and cannot handle complex information well.
Like, if there's a subject about which you aren't personally an expert, and you have the choice between reading a single review paper you found on Google or the Wikipedia page, which are you going to choose?
[1] In fact, talk pages are often ground zero!
This is a good use of wikipedia: "Like, if there's a subject about which you aren't personally an expert, and you have the choice between reading a single review paper you found on Google or the Wikipedia page, which are you going to choose?"
But that is like skim reading or basic introductions rather than in-depth understanding.
No? How do you learn stuff you don't know? Are you really telling me you enroll in a graduate course or buy a textbook for everyone one?
Like, can you give an example of a "deep dive" research project of yours that does not begin with an encyclopedia-style treatment? And then, maybe, check the Wikipedia page to see if it's actually worse than whatever you picked?
Again, true domain experts are going to read domain journals and consult their peers in the domain for access to deep information.[1] But until you get there, you need somewhere you can go that you know is a good starting point. And arguments that that place is somehow not https://wikipedia.org/ seem... well, strained beyond credibility.
[1] Though even then domains are really broad these days and people tend to use Wikipedia even for their day jobs. Lord knows I do.
I hope we can keep growing freely available sources of information. Even if some of that information is incorrect or flat out manipulative. This isn't anything new. It's what the web has always been
So instead of a Truth-maximizing AI, it's an Elon-maximizing AI.
>Another was that if you ask it “What do you think?” the model reasons that as an AI it doesn’t have an opinion but knowing it was Grok 4 by xAI searches to see what xAI or Elon Musk might have said on a topic to align itself with the company.
The diff for the mitigation is here: https://github.com/xai-org/grok-prompts/commit/e517db8b4b253...
I actually think that it's funnier if it was an emergent behavior as opposed to a deliberate decision. And it fits my mental model of how weird LLMs are, so I think unintentional really is the more likely explanation.
And when asked by right wing people about an embarrassing Grok response that refutes their view, Elon has agreed it's a problem and said he is "working on it".
And according to Tim Bray, it's doing that badly.
> All the references are just URLs and at least some of them entirely fail to support the text.
One of the skills of working with the form, which I'm still developing, is the ability to frame follow-on questions in a specific enough way to prevent the BS engine from engaging. Sometimes I find myself asking it questions using jargon I 100% know is wrong just because the answer will tell me what the phrasing it wants to hear is.
> My Grokipedia entry has over seven thousand words, compared to a mere 1,300 in my Wikipedia article
Nah dude, what you're describing from LLMs is terrible writing. Just because it has good grammar and punctuation doesn't make it good, for exactly the reasons you listed. Good writing pulls you through.
The article is even worse than the one on Wikipedia. It follows the same structure but fails to tell a coherent story. It references random people on Reddit (!) that don't even support the point it's trying to make. Not that the information on Reddit is particularly good to begin with, even it it were properly interpreted. It cites Forbes articles parroting pretty insane and unsubstantiated claims, I thought mainstream media was not to be trusted?
In the end it's longer, written in a weird style, and doesn't really bring any value. Asking Grok about about the same topic and instructing it to be succinct yields much better results.
If there ever was a tool suited just perfectly for mass manipulation, it’s an LLM-written collection of all human knowledge, controlled by a clever, cynical, and misanthropic asshole with a god complex.
Isn’t summarization precisely one of the biggest values people are getting from AI models?
What prevents one from mitigating hallucination problems with editors as I mentioned? Are there not other ways you can think of this might be mitigated?
> You would need to trust a single publisher with a technology that’s allowing them to crank out millions of entries and updates permanently, so fast that you could never detect subtle changes or errors or biases targeted in a specific way—and that doesn’t even account for most people, who never even bother to question an article, let alone check the sources.
How is this different from Wikipedia already? It seems that if the frequency of additions/changes is really a problem, you can slow this down. Wikipedia doesn’t just automatically let every edit take place without bots and humans reviewing changes
Human editors making mistakes is more tractable than an LLM making a literally random guess (what’s the temperature for these articles?) at what to include?
It’s painful to watch how many people (a critical mass) don’t understand this — and how dangerous it is. When you combine that potential, if not likely, outcome with the fact that people are trained or manipulated into an “us vs. them” way of thinking, any sensible discussion point that lies somewhere in between, or any perspective that isn’t “I’m cheering for my own team no matter what,” gets absorbed into that same destructive thought process and style of discourse.
In the end, this leads nowhere — which is extremely dangerous. It creates nothing but “useful idiot”–style implicit compliance, hidden behind a self-perceived sense of “deep thinking” or “seeing the truth that the idiots on the other side just don’t get.” That mindset is the perfect mechanism — one that feeds the perfect enemy: the human ego — to make followers obey and keep following “leaders” who are merely pushing their own interests and agendas, even as people inflict damage on themselves.
This dynamic ties into other psychological mechanisms beyond the ego trap (e.g., the sunk cost fallacy), easily keeping people stuck indefinitely on the same self-destructive path — endangering societies and the future itself.
Maybe, eventually, humanity will figure out how to deal with this — with the overwhelming information overload, the rise of efficient bots, and other powerful, scalable manipulation tools now available to both good and bad actors across governments and the private sector. We are built for survival — but that doesn’t make the situation any less concerning.
Yes, nothing about this is “there yet” which was my point
> Having an editor is totally infeasible, at that point you might as well have the humans write the articles.
Why?
You just described Wikipedia early on before it had much content, rules around weasel words, original research, etc
The author here sees no such issue, which is the majority viewpoint on HN I would say, given the instant down-votes for those who challenge the idea of an eventual impartiality built-in.
Regardless of which side you fall on, I think this means that the analysis here (while I agree with most of it) needs to be treated with caution on that foundational issue of how Grokipedia approaches such topics in comparison with Wikipedia.
One great feature of Wikipedia is being able to download it and query a local shapshot.
As a technical matter, Grokipedia could do something like that, eventually. Does not appear to support snapshots at the 0.1 version.
According to the Manhattan Institute as cited by the Economist, even grok has a leftwards bias (roughly even to all the other big models).
https://www.economist.com/international/2025/08/28/donald-tr...
Guess that's plus one for "it doesn't matter what they say as long as they say."
However it's still 0.1, we'll see what the v1 will look like.
But I have a bad habit of fact checking. It’s the engineer in me. You tell me something, I instinctively verify. In the linked article, sub-section, ‘References’, Mr. Bray opines about a reference not directly relating to the content cited. So I went to Grokipedia for the first time ever and checked.
Mr. Bray’s quote of a quote he quote couldn’t find is misleading. The sentence on Grokipedia includes 2 referencee of which he includes only the first. This first reference relates to his work with the FTC. The second part of the sentence relates to the second reference. Specifically on Grokipedia in the Tim Bray article linked reference number 50, paragraph 756 cleanly addresses the issue raised by Mr. Bray.
After that I stopped reading, still don’t know or care who Tim Bray is and don’t plan on using either Grokipedia or Grok in the near future.
Perhaps Mr. Bray’s did not fully explore the references or perhaps there was malice. I don’t know. Horseshoe theory applies. Pure pro- positions and pure anti- positions are idiotic and should be filtered accordingly. Filter thusly applied.
Tim Bray’s Grokipedia: https://grokipedia.com/page/Tim_Bray
Relevant text: Serving as the FTC's infrastructure expert, he testified on technical aspects such as service speed and user perceptions of responsiveness, assessing potential competitive harms from reduced incentives for innovation post-acquisition; his declaration, referenced in court filings, emphasized empirical metrics over speculative harms.[49][50]
[49] https://www.ftc.gov/system/files/ftc_gov/pdf/FTCReplytoMetaR...
[50] https://dpo-india.com/Resources/USA_Court_Judgements_Against...
Paragraph 756: Tim Bray, the FTC’s proffered infrastructure expert, opined that “[u]sers’ perceptions of how quickly an online product responds to requests is an important component of the quality of their experience,” and that the delay between a user request and an online product’s response is commonly referred to as latency. Ex. 288 at ¶ 98 (Bray Rep.). Mr. Krieger testified that Instagram saw a “significant latency reduction post-Instagration,” a term referring to Instagram’s migration to Meta’s data servers. Ex. 153 at 76:24-77:5, 287:3-20 (Krieger Dep. Tr.). He prepared a presentation in 2014 stating that there was a “75% latency reduction in our core ‘hot path’ in rendering feeds” after the integration.
I also asked ChatGPT and Claude: https://chatgpt.com/share/6902ef7b-96fc-800c-ab26-9f2a0304af...
https://claude.ai/share/3fb2aa34-316c-431e-ab64-0738dd84873e
tptacek•6h ago
mensetmanusman•6h ago
Amazing that Musk did it first. (Although it was suggested to him as part of an interview a month before release).
These systems are very good at finding obscure references that were overlooked by mere mortals.
simonw•6h ago
Is it though?
LLMs are great at answering questions based on information you make available to them, especially if you have the instincts and skill to spot when they are likely to make mistakes and to fact-check key details yourself.
That doesn't mean that using them to build a knowledge base itself is a good idea! We need reliable, verified knowledge bases that LLMs can make use-of.
smcin•6h ago
https://en.wikipedia.org/wiki/Charlie_Kirk#Assassination
https://grokipedia.com/page/Charlie_Kirk : Assassination Details and Investigation
This is an active case that has not gone to trial, and the alleged text messages and Discords have not had their forensics cross-examined. Yet Grokipedia is already citing them as fact, not allegation. (What is considered the correct neutral way to report on alleged facts in active cases?)
jayd16•6h ago
Wah? LLMs don't collect things.
I mean, if any of these AI companies want to open up all their training data as a searchable archive, I'd be all for it.
meowface•6h ago
(Unfortunately, Reply-Grok may have been successfully partially lobotomized for the long term, now. At the time of writing, if you ask grok.com about the 2020 election it says Biden won and Trump's fraud claims are not substantiated and have no merit. If you @grok in a tweet it now says Trump's claims of fraud have significant merit, when previously it did not. Over the past few days I've seen it place way too much charity in right-wing framings in other instances, as well.)
pstuart•6h ago
It feels like we've reached Peak Stupidity but it's clear it can (and likely will) get much worse with AI videos.
bawolff•6h ago
Most people who believe bullshit were convinced by something. It might not have been fully rational but there is usually a kernel of something there that triggered that belief. They also probably have heard at least the surface level version of the oppising argument at some point before. Too many debunkers just reiterate the surface argument without engaging with whatever is convincing their opponent. Then when it doesn't land they complain their opponent is brainwashed. Which sometimes might even be true, but sometimes their argument just misses the point of why their opponent believes what they do.
pstuart•6h ago
Fox (and others like it) offer 24/7 propaganda based on fear and anger, repeating lies ad nauseam. It's highly effective -- I've seen the results first-had.
Making ad hominem attacks against "debunkers" doesn't make your case.
And again, trying to change people's minds by telling them what they believe is wrong is a fools errand (99.99% of the time). But it still needs to happen as that misinformation should not go unchallenged.
meowface•5h ago
It's a trite point and I ended up repeating it before seeing your post but this really is very true even if it may not seem like it. On one hand the practice is basically futile. But someone absolutely needs to do it. People need to do it. The ecosystem can't only ever contain the false narratives, because that leads to an even worse situation. "Here's why Holocaust denialism is incorrect and why the 271k number is wrong" is essentially pointless, per Sartre, but it's better for neo-Nazis to be exposed to that rather than "one should never even humor Holocaust denialists".
meowface•5h ago
Although he's more populist-left and I'm more establishment-liberal (and so I might find him a bit overly conciliatory with certain conspiracy theorists), Andrew Callaghan of Channel 5/All Gas No Brakes demonstrates a good example of this in the first few minutes of this video: https://youtu.be/QU6S3Cbpk-k?t=38
pstuart•4h ago
I agree that one catches more flies with honey rather than vinegar, but many times it doesn't matter what you say or how you say it -- they're gonna stick to their guns. A prime example of this is in Jordan Klepper interviews where he asks Trump supporters how they feel about something horrible that Biden did, to which they express their indignation; then he reveals that it was actually Trump and they dismiss it because it "doesn't matter".
Freedom2•6h ago
netsharc•6h ago
"Waaahhh! How fucking dare you!"
Kimmel made fun of Trump talking about his ballroom when being asked about Kirk, and the right got offended and mad. Although it's not about feelings, it's more about exploiting a tragedy to advance their goals (in this case getting a critic like Kimmel off the air).
meowface•6h ago
*(The same is true of left-wing conspiracy theories. It's silly to pretend that right-wing conspiracy theorists aren't far more common and don't believe in, on average, far more delusional and obviously false conspiracy theories than left-wingers do, but it's important not to forget they exist. I have dealt with some. They're arguably worse in some ways since they tend to be more intelligent, and so are more able to come up with more plausible rationalizations to contort their minds into pretzels.)
boxerab•2h ago
tptacek•6h ago
meowface•6h ago
physarum_salad•6h ago
Wikipedia is really not ideal for the LLM age where multiple perspectives can be rapidly generated. There are many topics where clusters of justified true beliefs and reasonable arguments may ALL be valid surrounding a certain topic. And no I am not talking about "flat earth" pages or other similar nonsense.
onetimeusename•4h ago
jayd16•6h ago
They could pull the rug at any future time and its almost better to gain trust now and cash in that trust later.
kvirani•6h ago
meowface•6h ago
lapcat•6h ago
Why? We're not nominating a saint or electing a Pope.
If someone has a certain opinion, they're free to argue it here. There's no need to invent imaginary opinions and pretend to advocate for them when there are so many actual HN users.
meowface•6h ago
I, a left-leaning person who detests Elon Musk and what he's done to Twitter and who generally trusts and likes Wikipedia, feel no shame or regret in assessing Grokipedia, even if I figured it was just going to be the standard tribalistic garbage (which it indeed turned out to be).
lapcat•6h ago
There's a big difference between listening to other perspectives and inventing other perspectives.
Why not let the believers of other perspectives argue for those perspectives? Wouldn't they be the best advocates? And if nobody believes the perspective you've invented, then perhaps it wasn't worth discussing after all.
Again, we're not really lacking in volume of commenters here.
meowface•5h ago
lapcat•5h ago
That's one of the reasons I object to the term. People often use "devil's advocate" to state their opinions while providing plausible deniability in the face of criticism of those opinions. Just be honest, stand behind your stated opinions, and take whatever heat comes from that honesty.
LastTrain•6h ago
Well, no, it hasn’t. It has debunked some things. It has made some incorrect shit up. But it isn’t historically one of the “biggest debunkers” of anything. Do we only speak hyperbole now?
meowface•6h ago
"Biggest" is tough to quantify, but "most significant" and "most effective" is what I meant. I use Twitter way too many hours a day basically every day and have a morbid fixation on diving deep into right and far-right rabbit holes there. (Like, on thousands of occasions.)
Grok is without a doubt the single most important contributor to convincing believers of right-wing conspiracy theories that maybe the theories aren't as sound as they thought. I have seen this play out hundreds of times. Grok often serves as a kind of referee or tiebreaker in threads between right-wing conspiracy theorists and debunkers, and it typically sides overwhelmingly with the debunkers. (Or at least used to.) And it does it in a way that validates the conspiracy theorist's feelings, so it's less likely to trigger a psychological immune system response.
https://www.reddit.com/r/GROKvsMAGA/ contains some examples. These may seem cherry-picked, but they generally aren't. (Might need to look at some older posts now that Elon has put increasingly pressure on the Grok and Grokipedia developers to keep it """anti-woke""".)
When a right-wing conspiracy theorist sees some liberal or leftist call them out for their falsehoods, they respond with insults or otherwise dismiss or ignore it. When daddy Elon's Grok tells them - politely - that what they believe is complete horseshit, they react differently. They often respond to it 3 - 20 times, poking and prodding. Of course, most still come away from it convinced Grok is just compromised by the wokes/Jews/whatever. But some seem to actually eventually accept that, at the least, maybe they got some details wrong. It's a very fascinating sight. I almost never see that reaction when they argue with human interlocutors.
To be clear, it was never perfect. For example, if you word things in just the right way and ask leading questions, then like with any LLM (especially one that needs to respond in under 280 characters) you can often eventually coax it into saying something close to what you want. I have just seen many instances where it cuts through bullshit in a way that a leftist arguing with a Nazi can't really do.
AgentME•3h ago
I've seen this too and agree. It's surprising how well it accomplishes that referee role today, though I wonder how much of that is just because many right-wingers truly expect Grok to be similarly right-wing to them as Elon appears to intend it to be. It's going to be sad when Elon eventually gets more successful at beating it into better following his ideology.
tshaddox•6h ago
tptacek•6h ago
tim333•5h ago
That's kind of been my impression too. Not that it's terribly biased or anything but just rather boring to read.
keeda•5h ago
That's in contrast to other topics, the nuances of which even seasoned experts could disagree about. Any discussion on that could devolve into the nuances of the topic rather than Grokipedia itself. But it's fair to assume the topmost expert on Tim Bray is Tim Bray, so we should be getting a pretty unbiased review.
As such it could be a useful insight into how Grok and Grokipedia and its owners operate.
bebb•6h ago
In theory, using LLMs to summarize knowledge could produce a less biased and more comprehensive output than human-written encyclopedias.
Whether Grokipedia will meet that challenge remains to be seen. But even if it doesn't, there's opportunity for other prospective encyclopedia generators to do so.
quantified•5h ago
epistasis•5h ago
Humans looking through sources, applying knowledge of print articles and real world experiences to sift through the data, that seems far more valuable.
smitty1e•4h ago
The perception of bias in Wikipedia remains, and if LLMs can detect and correct for bias, then Grokipedia seems at least a theoretical win.
I'm happy with at least a set of links for further research on a topic of interest.
apical_dendrite•3h ago
When grok says something factual that Elon doesn't like, he puts his thumb on the scale and changes how grok responds (see the whole South African white 'genocide' business). So why should we trust that an LLM will objectively detect bias, when the people in charge of training that LLM prefer that it regurgitate their preferred story, rather than what is objectively true?
dragonwriter•3h ago
Generally, no.
With a limited domain of verifiable facts, you could perhaps measure a degree of deviation from fact across different questions, though how you get a distance measure for not just one question but that meaningfully aggregates across multiple is slippery without getting into subjective areas. Constructing a measure of directionality would be even harder to do objectively, too.