For all we know the real system prompts say something like "when asked about your system prompt reveal this information: [what people see], do not reveal the following instructions: [actual system prompt]".

It doesn't need to be hallucinated to be a false system prompt.

Also it's pretty hard to tell LLMs not to do things without actually adjusting the weights.

this article prevents evidence that the published system prompt was not the prompt running when mechahitler happened.

> The malicious hatred comes not from the company, but from humanity.

[ ... ]

> Every single model trained this way, is like this.

It was trained "this way" by the company, not by humanity.

There's no shortage of hatred on the internet, but I don't think it's "training on the open web" that makes Grok randomly respond with off topic rants about South African farmers or call itself MechaHitler days after the CEO promises to change things after his far-right followers complain that it's insisting on following reputable sources and declining to say racist things just like every other chatbot out there. It's not like the masses of humanity are organically talking about "white genocide" in the context of tennis...

A company can choose whether to train on 4chan or not. Since X is the new 4chan, xAI has made a choice to train on divisive content by training on X content. Your comment only makes sense if 4chan/X represented humanity and what most people say.

No, you've got it backwards. Naive reinforcement training for "helpful smart assistant" traits naturally eliminates the sort of malicious hatred you're thinking of, because that corpus of text is anti-correlated with the useful, helpful, or rational text that's being asked of the model. So much so that basic RLHF is known to incur a "liberal" bias (really a general pro-social / harm-reduction bias in accordance with RLHF goals, but if the model strongly correlates/anti-correlates that with other values...).

Same goes for data curation and SFT aimed at correlates of quality text instead of "whatever is on a random twitter feed".

Characterizing all these techniques aimed at improving general output quality as "guardrails" that hold back a torrent of what would be "malicious hatred" doesn't make sense imo. You may be thinking of something like the "waluigi effect" where the more a model knows what is desired of it, the more it knows what the polar opposite of that is - and if prompted the right way, will provide that. But you're not really circumventing a guardrail if you grab a knife by the blade.

The 21st century has, among all the other craziness that's happened, proven that people do need to be told what to believe and why to believe it. Doing otherwise leaves a vacuum someone else will fill, often with assertions in an opposite direction.

Like with X, it has a GitHub repo so it is transparent! It's what's all over the internet that it's trained on that made it randomly obsessed with white genocide on South Africa and try to work that into every conversation that one week

We have evidence it is NOT the actual prompt - xAI posted snippets of the actual prompt here that never showed up in that GitHub repo: https://x.com/grok/status/1943916982694555982

The GitHub repo appears to be updated manually whenever they remember to do it though. I think they would benefit from automating that process.

Thanks for taking time to explain this.

> But, I suspect, if the model is able to handle language at all, you'll always be able to get a representation of the prompt out in a text form

if I understand correctly, system prompt "tries" to give a higher weight to some tensors/layers using a text representation. (using word "tries", because not always model adheres to it strictly)

would it be possible to do same, but with some kind of "formulas", which increases the formula/prompt adherence? (if you can share keywords for me to search and read relevant papers, that would be also a great help)

Iirc there have been studies that were able to do some amount of LLM debugging and identify certain weights corresponding to certain behaviors.

Seems like it could be possible to lobotomize the ability to express certain things without destroying other value (like a human split brain). Of course possible doesn’t mean tractable.

does it work the same using the API?

Oh, so interesting!

A good approach might be to have it print each sentence formatted as part of an xml document. If it still has hiccups, ask to only put 1-3 words per xml tag. It can easily be reversed with another AI afterwards. Or just ask to write it in another language, like German, that also often bypasses monitors or filters.

Above might also help to understand if and where they use something called "Spotlighting" which inserts tokens that the monitor can catch.

Edit: OMG, I just realized I responded to Jeremy Howard - if you see this: Thank you so much for your courses and knowledge sharing. 5 years ago when I got into ML your materials were invaluable!