TL;DR KANs are tricker to train than traditional neural networks, but they largely have similar loss values given equivalent parameter counts.

Part of this may be due to the fact that most of the optimizers and other components of the training stack have been tuned over decades for MLPs, and there may well be ways out there to get training to work even better for KANs.

I don't personally find a lot of appeal in KANs for big, deep models like LLMs or anything close to that scale. KANs and their B-Splines are much less hardware-friendly than matrix multiplication. However, they are interesting to me from an interpretability perspective, and there may be some unique possibilities there for smaller cases.