Garbage in garbage out. That's how machine learning works it's not a "programming error", it's their data.
strangecasts•8mo ago
In the case of this and the earlier "white genocide" replies, it is way more likely someone changed the system prompts than that someone tampered with the training data, considering the conspiracy theory was brought up unprompted in wildly unrelated situations [1]
Oh my theory is not that someone tampered with the training data. It's that their data is sourced from bad sources think 4chan, 8chan, etc.
strangecasts•8mo ago
Obviously I can only speculate since I neither have access to their dataset nor interest in paying for API access, but crawling and dataset cleaning have gotten much better since the GPT-2 days, especially after Microsoft's PHI models [1] demonstrated how much dataset construction matters for parameter efficiency and toxicity. Having some basic content filtering is a pretty established part of data cleanup -- e.g. the fastText toxicity classifiers in the Dolma pipeline [2] -- which obviously still leaves in bad data, but certainly won't leave in the entirety of /b/
If shoddy data collection was the problem, we should expect the model to do much worse on overall leaderboards like [3], which require models to answer questions without sudden detours into Holocaust denialism. A change to the system prompt is more consistent with this, and as an added benefit, only requires one person to be completely out of their gourd.
throwawayffffas•8mo ago
strangecasts•8mo ago
[1] As one example, https://bsky.app/profile/jdcmedlock.bsky.social/post/3lp6eal...
rasz•8mo ago
throwawayffffas•8mo ago
strangecasts•8mo ago
If shoddy data collection was the problem, we should expect the model to do much worse on overall leaderboards like [3], which require models to answer questions without sudden detours into Holocaust denialism. A change to the system prompt is more consistent with this, and as an added benefit, only requires one person to be completely out of their gourd.
[1] https://www.microsoft.com/en-us/research/publication/textboo...
[2] https://arxiv.org/pdf/2402.00159
[2] https://livebench.ai/#/