This is a major leap in technology. I just used it from their github and it is compressing a 24 Billion parameter to just 14GB and that too in 2 minutes. It automatically updates the tokenizer and runs the model for inference in seconds. This is very useful for local AI deployment but not for commercial usage. Its still behind vLLM in multi-user inference.