//A language model hallucinating corresponds to situations in which the model loses track of reality so it’s not able to produce accurate outputs.//

"track of reality"?

The article traffics in the same bullshit as it is calling out.

The conclusion somehow walks by the idea of truth with barely a flinch about the enormity of the term, where in principle the entire scope of the project is meaningless without some stipulations about truth; stipulations which if made would tend to obviate the details of model internal implementation and put focus on the place the models currently store truth: the training data.

So what about the significance of RLHF?! Why does anyone expect truth can be properly arbitrated by human feedback any more so than it was in the OG training sets?

If we switch our concern from language synthesis to images created by diffusion, which according to the underlying machine have the same principles of operation, is anyone asking how "true" AI generated images are? Of course not because the point is meaningless. Images are accepted because they are believable and language is accepted by the same criteria.

If we generalize the idea of human feedback to the operation and profit expectations of the social media industry, a starkly obvious factor for performance sparkles brightly before us: the industry will relentlessly track towards whatever garners attention and truth can go to hell.