A fundamental question I have is whether the current transformer based architectures are powerful enough to allow scaling towards AGI or are there ceilings here that are becoming more evident?

Though folks from the leading LLM companies are in the forefront of this, it’s hard to take their word due to an obvious conflict of interest. And it’s unclear if academia has more theoretical knowledge here as it seems these commercial companies are also leading the research work here.