I've started building Neural Foundations, a Python library to explore the first principles of neural networks by constructing them from atomic primitives (cells, layers, activations, etc.). Inspired by the Asimov Institute's "Neural Network Zoo".

This is an early-stage playground aimed at those who really want to explore and internalize NN fundamentals by diving into and hacking the foundational_nn and architecture_nn packages, which are then used to build models in models_nn.

While you won't find Transformer architectures here yet, the library essentially provides all the building blocks to (eventually) create everything from the Neural Network Zoo infographic.

I'm also keen on visualizing simpler architectures (beyond just the TUI), which should be a relatively easy entry point for contributions. So, if you're interested in getting involved, contributions to the visualization (or any other!) module are very welcome!

I'm also open to discussion about the project structure, ways in which it might be run, potential use cases, and any other ideas you might have. Let's build this together!

Check out the code and interactive TUI for experimentation: https://github.com/ndjuric/neural_foundations/tree/main

Note: it's in the earliest possible development stage :)