Potentially useful for things like innate mathematical operation primitives. A major part of what makes it hard to imbue LLMs with better circuits is that we don't know how to connect them to the model internally, in a way that the model can learn to leverage.
Having an "in" on broadly compatible representations might make things like this easier to pull off.
Edit: to be clear I think these patterns are real and meaningful, but only loosely connected to a platonic representation of the number concept.
"How Different Language Models Learn Similar Number Representations" (actual title) is distinctly different from "Different Language Models Learn Similar Number Representations" - the latter implying some immutable law of the universe.
Saw similar study comparing brain scans of person looking at image, to neural network capturing an image. And were very 'similar'. Similar enough to make you go 'hmmmm, those look a lot a like, could a Neural Net have a subjective experience?'
gn_central•1h ago
OtherShrezzing•1h ago