Lovely explanation!

Just FYI: mouse-scrolling is much too sensitive for some reason (I'm assuming it swipes just fine in mobile contexts, have not checked that). The result is that it jumped from first to last "page" and back whenever I tried scrolling. Luckily keyboard input worked so I could still read the whole thing.

that's soooo coool

This takes me back. 1990, building Boltzman machines and Perceptrons from arrays of void pointers to "neurons" in plain C. What did we use "AI" for back then? To guess the next note in a MIDI melody, and to recognise the shape of a scored note, minim, crotchet, quaver on a 5 x 9 dot grid. 85% accuracy was "good enough" then.

anyone got an archived link?

Typo

“They can be used for generating new data that…”

Oh, this is a neat demo. I took Geoff Hinton's neural networks course in university 15 years ago and he did spend a couple of lectures explaining Boltzmann machines.

> A Restricted Boltzmann Machine is a special case where the visible and hidden neurons are not connected to each other.

This wording is wrong; it implies that visible neurons are not connected to hidden neurons.

The correct wording is: visible neurons are not connected to each other and hidden neurons are not connected to each other.

Alternatively: visible and hidden neurons do not have internal connections within their own type.

Fun article on David Ackley https://news.unm.edu/news/24-nobel-prize-in-physics-cited-gr...

Do check out his T2 Tile Project.

IIUC, we need gibbs sampling(to compute the weight updates) instead of using the gradient based forward and backward passes with today's NNetworks that we are used to. Any one understand why that is so?

Love how this breaks down Boltzmann Machines—finally makes this 'energy-based model' stuff click!

That font with a bit of margin looks fantastic on my phone specifically. Really nailing the minimalist look. What font is that?

> Here we introduce introduction to Boltzmann machines and present a Tiny Restricted Boltzmann Machine that runs in the browser.

nit: should "introduction" be omitted?

The section after the interactive diagrams has no left padding and thus runs off the screen on iOS.

Now the real question: is it you enjoying that nice page or is it a Boltzmann Brain?

https://en.m.wikipedia.org/wiki/Boltzmann_brain

Great site! Would be cool to be able to adjust the speed at which the simulation runs as well.

This is very well made, and so nostalgic to me! My whole PhD between 2012-16 was based on RBMs and I learned so much about generative ML through these models. Research has come so far and one doesn't hear much about them these days but they were really at the heart of the "AI Spring" back then.

Great read!

One nit, a misspelling in the Appendix: derivce -> derive

Nice and clean explanation!

It brings up a lot of memories! Shameless plug: I made a visualization of an RBM being trained years ago: https://www.youtube.com/watch?v=lKAy_NONg3g

My understanding is that the Harmonium (Smolensky) was the first restricted Boltzmann machine, but maximized “harmony” instead of minimizing “energy.” When Smolensky, Hinton and Rummelhart collaborated, they instead called it “goodness of fit.”

The harmonium paper [1] is a really nice read. Hinton obviously became the superstar and Smolensky wrote long books about linguistics.

Anyone know more about this history?

[1] https://stanford.edu/~jlmcc/papers/PDP/Volume%201/Chap6_PDP8...

I mistook the title for "A Tiny Boltzmann Brain"! [0]

My own natural mind immediately solved the conundrum. Surely this was a case where a very small model was given randomly generated weights and then tested to see if it actually did something useful!

After all, the smaller the model, the more likely simple random generation can produce something interesting, relative to its size.

I stand corrected, but not discouraged!

I propose a new class of model, the "Unbiased-Architecture Instant Boltzmann Model" (UA-IBM).

One day we will have quantum computers large enough to simply set up the whole dataset as a classical constraint on a model defined with N serialized values, representing all the parameters and architecture settings. Then let a quantum system with N qubits take one inference step over all the classical samples, with all possible parameters and architectures in quantum superposition, and then reduce the result to return the best (or near best) model's parametesr and architecture in classical form.

Anyone have a few qubits laying around that want to give this a shot? (The irony that everything is quantum and yet so slippery we can hardly put any of it to work yet.

(Sci-fi story premise: the totally possible case of an alien species that evolved one-off quantum sensor, which evolved into a whole quantum sensory system, then a nervous system, and subsequently full quantum intelligence out of the gate. What kind of society and technological trajectory would they have? Hopefully they are in close orbit around a black hole, so the impact of their explosive progress has not threatened us yet. And then one day, they escape their gravity well, and ...)

[0] https://en.wikipedia.org/wiki/Boltzmann_brain