So, if I'm understanding correctly, this is just for lower bandwidth transfers of full BF16 weights over the wire, not for serving, correct? Have you benchmarked the performance against a SOTA general purpose compression algorithm like zstd?
Also, are all that many people handling the BF16 weights directly? GLM-5.2's reference deployment is FP8, and many vendors are even serving at NVFP4 which seems to offer negligible degradation over the FP8 reference deployments.
Sanzig•13h ago
Also, are all that many people handling the BF16 weights directly? GLM-5.2's reference deployment is FP8, and many vendors are even serving at NVFP4 which seems to offer negligible degradation over the FP8 reference deployments.