That sounds a bit like biology; maybe AI will become like that.
GLENDOWE: I can call spirits from the vasty deep.
HOTSPUR: Why, so can I, or so can any man, but will they come when you do call for them?
I can interpret weights too, but will that interpretation be correct?
Chances are much higher than a model can figure out what the patterns are, decipher them, and retain that knowledge in a meaningful way it can reason about. Me and you might take years or months to do all of that on our own, with a "map" of the weight we might be able to discuss it, but not as effortlessly as an LLM.
Honestly, I had told a few different colleagues that reading a model is probably not that exclusive to frontier models. It's probably yet another reason OpenAI / Anthropic don't release any big open weight models, but thankfully Google and Microsoft release reasonably sized models. Imagine a new GPT open weight release, it would get dissected drastically.
The age of source code is definitely not going to end anytime soon if ever. But the future will involve incomprehensible machines editing other models in a "fairly certain" way, but we are going to need a lot of tooling to keep up with the process and not just take the model updates as "trust me bro".
So i asked Qwen 3.8 27B two things:
1. What it is this network all about? I only gave it the ann.c code that implemented a simple artificial neural network library in C and the setup code that specified the network's shape but not its purpose.
Qwen 3.8 27B tried a bunch of stuff, taking the layers one by one, proding numbers through it, etc and finally figured out it is some sort of 5x5 to 2x2 filter, most likely working with pixels (i never mentioned pixels or graphics), though it wasn't certain about the exact purpose. It did rule out a bunch of uses like being a downscaler or classifier, from its behavior.
2. After i revealed what it was, i asked it if it can improve it but keep it 100% compatible with the existing program that used the weights file. And sure enough, it did improve it. I gave it the original training C code and dataset. It made a few experiments (and wrote its own training C code because it found mine too slow :-P) and ended up improving the weights. The difference[1] is small, but it basically eliminated some small noise that was always there and made some transitions less "halo-y". Of course for the purpose i wanted it (use it to upscale games in realtime) it isn't that great (too heavy) but it was learning material anyway. Though one bonus thing i noticed is that it upscales my random doodles much better than any Krita filter[2] :-P.
The above isn't Qwen 3.8 27B observing the weights directly and making sense out of them (ok actually it did seem to make some sense of the weights in the "#1" but i didn't dump the entire network to it - though thinking about it, it is only around 16384 floats so it might work). It just used a bunch of scripts it wrote in Python (and later C) but i still found it interesting to watch it go at it.
[0] https://mastodon.gamedev.place/@badsector/113765818432603610
[1] http://runtimeterror.com/pages/iv/images/a3d94ea369ff95ecf05...
[2] http://runtimeterror.com/pages/iv/images/4e29d027a8aa6e89db5...
It's a blog post which received feedback from collaborators.
It hasn't gone through the academic paper lifecycle ... the authors themselves refer to it as a post and as preliminary.
I feel like I'm taking crazy pills, but can we be precise when talking about work of this form? It started with referring to arXiv pre-prints as papers, and now we're at blog posts with an academic template as papers.
A paper is a fucking paper: something which has been vetted by the scientific community, with at least some form (often, a few rounds) of anonymous review from people outside your circle of collaborators
This is not there yet, and clearly so due to the various feedback from people (including the two pushback posts on HN) critiquing the lack of discussion about the existence of these techniques in mech interp, etc?
I called it a paper here because it's by people with an academic affiliation and is of comparable quality (if not better than) many of the other "papers" we see.
I also like it when a paper isn't a PDF. PDFs are a bad way of communicating information.
(I will note that the authors themselves didn't call it a paper, their own BibTeX reference calls it a "blog post":)
@misc{takehi2026weightspace,
title = {{Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}},
author = {Takehi, Rikiya and Teoh, Jayden and Zhang, Alex and Khattab, Omar},
year = {2026},
month = nov,
howpublished = {\url{https://rikiyat.github.io/blog/weight-space-programming/}},
note = {Blog post}
}But for (1, 2), just to point out, neither of your criteria would pass muster in the scientific communities who are ultimately responsible for vetting this work, right?
The ideal is: we don't accept papers at NeurIPS (or wherever) because of affiliation, and the badge of "quality" is bestowed by peer review, from other scientists.
> is of comparable quality (if not better than)
But the posts that you refer to in your own post point out several ways in which the quality is not good ...
But, in any case, I find it a bizarre choice to call it a paper when the authors themselves do not refer to it as such. We can agree to disagree, but just offering feedback.
My first reaction to your question (rhetorical?) was uncharitable... so, I am curious what you meant that it was "easy."
[1] - https://nitter.netbub.com/AlexKrentsel/status/21173766837246...
Isn't that what this whole technology paradigm is about? How is it surprising?
Granted, it is cool. But I see so much of "look at this cool thing I did with LLMs" and honestly... we need more structured text, more explicit documentation, more ergonomic and intuitive CRUD applications.
The amount of "look at cool things" is growing way faster than before and they are all sort of low effort now because the "agentic" LLM-VM looping apparatus can do a lot of heavy lifting and run with our prompts.
And I mean, if one has jargon, knowledge, and intention, they can also leverage that into great structured text, ergonomic and intuitive CRUD apps etc, all of which add great value to our mundane pursuits of applied research, software products and services and so on.
Honestly, I begin to think the problem is people dazzling themselves instead of doing the housekeeping that is the bulk of life. Technology choice is entirely secondary.
> the 2x2 pixel grid that the input pixel should be replaced for a (width2)x(height2) upscale.
I admit that I don't think formal "papers" deserve their elevated status in today's information economy.
I think they add a whole lot of ceremony that usually results in a format that's harder to understand.
So me calling this a "paper" was a little bit of a dig!
From memory, yolo and roberta and ppo were initially rejected. After a quick google search, pagerank was initially rejected, word2vec got one weak and one strong reject in reviews, LSTM got rejected (90s but still), dropout, knowledge distillation, and many more such cases. All of these are extremely important, foundational and field defining papers that ended up on arxiv or other preprint places.
If that happened to those papers, there's 0 chance today that it'd be a good idea. The field is simply moving too fast, there's too much being published, and the "top venues" are simply swamped. Blog / preprint / carrier pigeon is ok atm, as long as someone can look at it, try it, and hopefully reproduce it.
tarruda•1h ago
awb•59m ago
f311a•40m ago
You can optimize a model by randomly changing its weights and validating results on your task. A monkey can type a book given enough attempts.
stymaar•37m ago
solnyshok•26m ago
latentsea•37m ago
rhdunn•5m ago
For training you would need to hook it up to a LoRA training harness, identify the looping weight direction, and adjust the weights accordingly. I've not (yet) looked into how to do this.
[1] https://news.ycombinator.com/item?id=48555634
TMWNN•8m ago