I had a great experience with llama-cpp with Nvidia backend on NixOS.
(Sorry for being that guy.)
Git clone llama.cpp and build it, it's not hard.
https://github.com/ggml-org/llama.cpp/blob/master/docs/build...
literally just a few steps for the basics:
git clone https://github.com/ggml-org/llama.cpp
cmake -B build
cmake --build build --config Release
Yeah, 100% and it's becoming more and more of a thing, see rust install for example.
OTOH, if you're installing llama.cpp, you're more than likely planning to run an LLM on you Linux box with an agentic harness, so a curl into bash thing might be the least of your security concerns, :-)
The harness gets the openai-compatible endpoint fed into it to talk to llama-server across the network, but the VM has no access whatsoever to my personal files, mail, backups/deep storage, fileserver, Documents folder, etc.
Yeah I recently tried the coding harness that's recommended here, Pi, in a bubble wrap sandbox and was horrified to learn that it spams multiple warnings at you if you don't give it write access to its own config/extension folder... Everyone else is rawdogging it I guess.
I can understand the desire for the llama.cpp project to want to own the end user relationship, it is true that previous to this they were a tool provider and not really owning the end user experience.
ggerganov and the team have done a stellar job maintaining the quality while still being fast to implement new models/improvements.
Two examples:
- https://github.com/ggml-org/llama.cpp/pull/25863 Someone's few lines change broke the native (ROCm) support for the AMD GPU inside Framework (and other integrated systems), and any rollback or proper fix is pending for almost a month. Fortunately there's workaround (switching to Vulkan rather than ROCm devices), but both the way the bug was introduced and the way it is not fixed just doesn't give much confidencen
- LM Studio is using llama.cpp internally for GGUF, they ship their own build with their closed source system as "runtimes". Their ROCm runtime does not enable the the AMD GPU inside the Framework, even thought the llama.cpp version would support it. So their runtime keeps telling me that there's no supported AMD GPU -- again, the solution is to use the GPU with the Vulkan devices. Not fixed since Jan at least https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1...
I guess overall it's the worst runtime I've seen so far, except for all the other runtimes out there... I'm a fan, though in some cases I don't have enough knowledge, or I don't have access to fix things, and that feels like a bummer...
How is it different than trusting any other method of installation? If URL has https and is from an author you trust i dont see the difference.
One thing I do not do as a matter of practice is install things with a ridiculous number of recursive npm dependencies.
I mean, sure, if there's people who can't figure that out, they're probably better off using a GUI that is a wrapper on top of somebody else's precompiled llama-server, like unsloth studio or lm studio. There's a good sized market for that and I wish them well.
Point Claude Code at a repository and ask how to install it safely. You don’t have to know about make or cryptography of HTTPS or anything, really. It will walk you through the options and risk.
If you have questions about any part of it—i.e. you don’t recognize an acronym or deeply understand why something works—you can ask.
Or ask here! HN is filled with smart humans.
Cloning a repo and building it is not _that_ hard, but easy installation is often the thing that makes or breaks a product. I believe Ollama proves that point in this context.
But yes, still trusting the project with arbitrary code execution on your machine, including build formulas that pull stuff from the internet and suffer from all the above anyways
tosh•1h ago
https://github.com/ggml-org/llama.cpp
bityard•59m ago
(I still deeply distrust curlpipes in general though.)
tosh•33m ago