DeltaTensors compresses fine-tuned model checkpoints by storing the weight difference from the base model instead of another full copy of the model.
If you have a bunch of fine-tunes or checkpoints from the same base model, you keep the base once and store a much smaller `.wdelta` for each version.
On a Qwen2.5-0.5B fine-tune, a 953 MB model compressed to a 294 MB delta. Reconstructing it changed perplexity from 19.11 to 19.22.
It works after training, so you do not need to use LoRA or change how you fine-tune the model. You can take an existing full fine-tune and compress it afterward.
It also supports chained deltas for model version history, Hugging Face Trainer checkpoints, and streaming compression/reconstruction so you do not need to load two full models into memory.
https://github.com/AaravGaurdev/deltatensors