I wanted to share a use case for Noema Overfit, a model-paging system available today in the Noema app. For disclosure, I founded Noema and I am part of the team that has helped develop the system.

The way it works is that the non-expert weights remain resident in memory, while the model’s routed expert weights are read from storage as needed. This makes it possible to run models that would otherwise exceed the device’s available memory, with the expected tradeoff of higher time to first token and slower generation.

For a 699-token initial prompt, I measured:

Prefill speed: 34.4 tokens/s Prefill time: 20.34 seconds Decode speed: 3.5 tokens/s

Photos of Overfit in action: https://noemaai.com/overfit/gemma4-iphone

The complete answer took approximately six minutes to generate, but the answer was correct. This is not intended for interactive, low-latency chat. The more interesting use case is allowing a constrained device to run a substantially more capable model when answer quality matters more than response time.

The same paging system can also be useful on lower-memory MacBooks that cannot ordinarily keep the complete model in unified memory.

Let us know if you can see this feature having any value!

Technical details: https://noemaai.com/overfit

Paged models: https://huggingface.co/NoemaAI-labs/Noema-Overfit