I built a simple tool to estimate how much memory is needed to run GGUF models locally, based on your desired maximum context size.
You just paste the direct download URL of a GGUF model (for example, from Hugging Face), enter the context length you plan to use, and it will give you an approximate memory requirement.
It’s especially useful if you're trying to figure out whether a model will fit in your available VRAM or RAM, or when comparing different quantization levels like Q4_K_M vs Q8_0.
Yes we did, this one is for calculating VRAM/RAM usage as well, to compliment Kolosal CLI and Desktop app and as a quick and easy tool to check if a model would run on your device
SmilingSuccess•4mo ago
You just paste the direct download URL of a GGUF model (for example, from Hugging Face), enter the context length you plan to use, and it will give you an approximate memory requirement.
It’s especially useful if you're trying to figure out whether a model will fit in your available VRAM or RAM, or when comparing different quantization levels like Q4_K_M vs Q8_0.
The tool is completely free and open-source. You can try it here: https://www.kolosal.ai/memory-calculator
And check out the code on GitHub: https://github.com/KolosalAI/model-memory-calculator
I'd really appreciate any feedback, suggestions, or bug reports if you decide to give it a try.