We built this after studying recent neuroscience research showing that dendrites perform significant nonlinear computation that current AI completely ignores. Traditional artificial neurons are basically weighted sums + activation functions. Real neurons have dendrites that do complex processing before the cell body even sees the signal. Our implementation adds "dendritic support units" that can be dropped into existing PyTorch models with minimal code changes. This open source version focuses on gradient descent training, while we continue research on alternative training mechanisms for future releases.
Early results show models that can be up to 152x cheaper, 10x smaller, and 20% more accurate.
rorry-brenner•6mo ago
We built this after studying recent neuroscience research showing that dendrites perform significant nonlinear computation that current AI completely ignores. Traditional artificial neurons are basically weighted sums + activation functions. Real neurons have dendrites that do complex processing before the cell body even sees the signal. Our implementation adds "dendritic support units" that can be dropped into existing PyTorch models with minimal code changes. This open source version focuses on gradient descent training, while we continue research on alternative training mechanisms for future releases.
Early results show models that can be up to 152x cheaper, 10x smaller, and 20% more accurate.
Code: https://github.com/PerforatedAI/PerforatedAI
Results of our recent hackathon: https://www.perforatedai.com/case-studies
Original Paper: https://arxiv.org/pdf/2501.18018
Happy to answer questions about the implementation or share more benchmarks!