This is a small transformer trained from scratch in 1.5hrs on a 5090 that beats many LLMs. Code is open source.

I want to solve sample efficiency and this work is an attempt to find the limits of transformers and today's methods while keeping costs low so I can iterate fast.

Blog: https://mvakde.github.io/blog/44-on-arc-1/ X thread: https://x.com/evilmathkid/status/2029519274835148829

(This is the public eval. Also it scores 7% on ARC-2)