There are only 4 optimizations in computer science: inlining, partial evaluation, dead code elimination, & caching. It looks like AI researchers just discovered inlining & they already knew about caching so eventually they'll get to partial evaluation & dead code elimination.
fragmede•19m ago
Dead code elimination is already a technique in AI when someone takes an MoE model and removes an unused "E" from it.
measurablefunc•37m ago
fragmede•19m ago