I'm a mechatronics designer with a background in control systems, robotics, PCB design, and embedded hardware. I design physical systems: motors, sensors, microcontrollers, and real-time control loops.

I applied this design thinking to LLM memory management – and it worked.

HotPin is a set of patches for llama.cpp that runs 30B–120B Mixture of Experts (MoE) models on far less RAM than their disk footprint, with bit-identical (lossless) output.

Tested on an AMD Ryzen AI 9 HX 370 (Zen5, AVX512), 23.6GB LPDDR5X, NVMe >1GB/s, CPU-only.

Results: | Model | Disk | Min RAM | Savings | tok/s | |-------|------|---------|---------|-------| | gpt-oss:120b | 58.5GB | 19.1GB | -67% | 3.84 | | qwen3:30b-a3b | 18.0GB | 10.4GB | -42% | 19.7 | | gemma4:26b-a4b | 16.2GB | 10.6GB | -35% | 11.5 | | GLM-4.7-Flash | 19.0GB | 13.3GB | -30% | 12.4 |

Output is SHA-256 bit-identical to full-RAM runs. Verified.

How it works (~50 lines of C++ in llama.cpp): 1. Profile MoE expert routing frequencies. 2. mmap the entire model from disk. 3. mlock only the hottest experts into physical RAM. 4. posix_fadvise / prefetch cold experts from NVMe before they're needed.

Boundary condition: if Disk > RAM, pinning gives +45% speedup (gpt-oss: 2.64 → 3.84 tok/s). If model fits in RAM, pinning adds zero overhead.

Tested on: Linux native, WSL, Windows native (VirtualLock).

Repo: https://github.com/LozzKappa/hotpin-llm Paper (PDF + LaTeX) in the repo – arXiv submission pending endorsement in cs.LG.

I'm looking for: 1. An arXiv endorser (cs.LG) – if you're a researcher and this work interests you, please reach out. 2. Feedback from anyone who tests it on their hardware.

The technique is simple, lossless, and works today. Test it and tell me your benchmarks.

Thanks for reading.