Note to self, Enshittification ahead. Don't use any ollama services unless it's calling an industry standard api.
But please don't use ollama, or their quants. Not only is the app itself slower than pure llamacpp. But their quants are often no where near the best.
I really hope people start with something like unsloth, as their software and quants are really much better all around.
I'm going to uninstall Ollama.
I wish HN would stop sharing drama-seeking posts, especially by anonymous authors, attacking the hard work of people.
Are ollama's authors perfect? No, they are just humans. But many people find their work valuable and we should support Open Source.
I can assure you that you won't like the world where nobody release Open Source projects anymore because of the constant negativity.
A year and still no implementation for such a basic need as offloading MoE layers onto the CPU selectively. On llama.cpp I can get models like Qwen 35BA3B running partially on gpu/cpu with 40t/s on a laptop thanks to --n-cpu-moe but on this VC funded joke it would be simply unusable. I can't quite understand how you make a wrapper so much worse than the code you're ripping out.
>This funding is fuel for what’s ahead. Ollama sits front and center in the open model ecosystem
No.
Then put those LLMs to work and build an open source alternative to everything that Ollama is doing.
After all, open source is a pricing weapon to race everything to $0.
Georgi Gerganov, the author of llama.cpp, is the real hero here.
Were they so enraptured with the “docker for LLMs” line?
fnlsnd•6h ago