Appreciate the Claude Makelele praise

Generalized Transformers from Applicative Functors

>Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors.

>[...]

>This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means.

https://cybercat.institute/2025/02/12/transformers-applicati...

I really enjoyed that when it was coming out, and used to follow it with some students. It's a shame it seems to have been abandoned.

When I read the first meaty chapter about graphs and commutativity I initially thought he just spends too long explaining simple concepts.

But then ai realized I would always forget the names for all the mathy c' words - commutativity commutativity, qssociativity... and for the first time I could actually remember commutativity and what it means, just because he tied it into a graphical representation (which actually made me laugh out loud because, initially, I thought it was a joke). So the concept of "x + y = y + x" always made sense to me but never really stuck like the graphical representation, which also made me remember its name for the first time.

I am sold.

Years ago when I was reading this (just a couple of chapters, not all of it), it opened my eyes to the power of diagrammatic representation in formal reasoning unlike anything before. I never did anything useful with string diagrams, but it was so fun to see what is possible with this system!

> If the internet has taught us anything, it’s that humans + anonymity = unpleasantness.

Aka one of my favorite axioms: https://www.penny-arcade.com/comic/2004/03/19/green-blackboa...

It's interesting how some of these diagrams are almost equivalent in the context of encoding computation in interaction nets using symmetric interaction combinators [1].

From the perspective of the lambda calculus for example, the duplication of the addition node in "When Adding met Copying" [2] mirrors exactly the iterative duplication of lambda terms - ie. something like (λx.x x) M!

[1]: https://ezb.io/thoughts/interaction_nets/lambda_calculus/202...

[2]: https://graphicallinearalgebra.net/2015/05/12/when-adding-me...