I wanted to train and run Transformers, but I was exhausted by gigabyte-sized Python environments, opaque C++ build systems, and deep BLAS dependency trees. I wanted to see what it actually takes to execute a forward and backward pass from absolute scratch.
The result is a single-header, stb-style C library written in strict C23.
Architectural decisions I made: - *Zero dependencies:* It requires nothing but a C compiler and the standard math library. - *Strict memory control:* You instantiate a `RaiArena` at boot. The engine operates entirely within that perimeter. There are zero hidden `malloc` or `free` calls during execution. - *The Full Stack:* It natively implements Scaled Dot-Product Attention, RoPE, RMSNorm, and SwiGLU. I also built the backprop routines, Cross-Entropy loss, AdamW optimizer, and a BPE tokenizer directly into the structs.
It is currently public domain (or MIT, your choice). The foundation is stable and deterministic, but it is currently pure C math. I built this architecture to scale, so if anyone wants to tear apart my C23 implementation, audit the memory alignment, or submit SIMD/hardware-specific optimizations for the matmul operations, I'm actively reviewing PRs.