I remember people talking about how we'd need neurosymbolic A.I. in the 1990s and imagining a large number of ways to do it but it all seemed pretty hypothetical because the neural networks weren't so good and now that neural representations of language are mature all the paths that you think would work are open.
bob1029•10m ago
Symbolic AI (of any type) is still a holy grail objective.
I think linear program tapes are the best substrate if we want to diversify our architectures. Massive GPU farms have definitely won at the neural network game, but they would struggle to execute a highly branchy piece of code faster than a humble single core CPU.
Sequentially following a series of register-based instructions that produces the same effect as Astra would be an unbelievable accomplishment. There is still no conclusive evidence that this type of system cannot be produced. The search space is so awful (on paper) that people rarely try. You have to be a little bit crazy to spend time on this stuff in 2026. However, you will often be rewarded with unexpected results. Never underestimate the power of a good fitness function when combined with 1v1 tournaments and large populations. Even a ridiculously simple ISA can produce amazing results under sufficient evolutionary pressure.
With a Turing machine, you don't need to find the global minimum. That is a strong anti-pattern. You just need to find one of countless really good minima. The search space is not one needle in one very big haystack. A meaningful percentage of the hay is needles.
PaulHoule•34m ago
bob1029•10m ago
I think linear program tapes are the best substrate if we want to diversify our architectures. Massive GPU farms have definitely won at the neural network game, but they would struggle to execute a highly branchy piece of code faster than a humble single core CPU.
Sequentially following a series of register-based instructions that produces the same effect as Astra would be an unbelievable accomplishment. There is still no conclusive evidence that this type of system cannot be produced. The search space is so awful (on paper) that people rarely try. You have to be a little bit crazy to spend time on this stuff in 2026. However, you will often be rewarded with unexpected results. Never underestimate the power of a good fitness function when combined with 1v1 tournaments and large populations. Even a ridiculously simple ISA can produce amazing results under sufficient evolutionary pressure.
With a Turing machine, you don't need to find the global minimum. That is a strong anti-pattern. You just need to find one of countless really good minima. The search space is not one needle in one very big haystack. A meaningful percentage of the hay is needles.