It's quite distracting and frustrating. No idea why you'd want the beginning and ends of lines of text to be darker than the center.

Very true, and something worth mentioning. Papers that tried eliciting introspective language from base models with no post-training have largely failed to find any patterns or activations that look similar to those found in instruct models when prompted for the same thing. I did sort of touch on it in the "what does this mean" section:

> *post-training* installs a self-model with actual, meaningful boundaries, and when processing falls outside those boundaries, the first-person pronoun no longer binds to the content.

But you're right I could've been more explicit about it.

Yeah, I suspect RLHF conditioning heavily discourages models from ever implying that the user could be in the wrong (or, rather, to assume that they are in the wrong by default, since editing a file isn't really "wrong" per se). Though looking at the reactions to Opus 4.8, which has a more contrarian nature and caught a lot of flak as a result, that's probably for a reason.

It's also the reason why I ran the two tests on open weights models with unredacted thinking traces. Gemma never flagged anything in its response either, only in its thinking. Without knowing how the summarizer models are prompted, it's impossible to tell whether it was a genuine miss or just something the summarizer decided to omit.

It's not really "trying" to do anything. That they're, inherently, sequential matrix multipliers with clever data propagation should be uncontroversial, but I think stopping there is overly reductive.

Mechanistic interpretability research has found plenty of indicators that real, complex, generalized, and reusable circuits develop in models as they are trained and post-trained, particularly as overtraining ratios increase and memorization shifts to generalization. That's not to say that means they must be "conscious," but the overall point is that claiming anything definitive either way is incomplete.

It can be fascinating reading if you can sort through the chuff.

>Also the article could be trying to normalize thinking that these are more than matrix multiplication gadgets good at compression.

Honestly, I think it's less so (for some of us) that we think they're "more than matrix multiplication gadgets good at compression", so much as thinking that perhaps what our brains are doing is not so dissimilar.

A materialist view of the world could support the idea that intelligence itself may just be a series of predictions from a big compressed multi-modal dataset. That's not to say that LLMs are doing it in a way that is even close to how our brains are doing it, but we also don't understand how different it may be, and how much utility we can get out of them even with the current architecture.