Ehh, is it cool and time savings that it figured it out? Yes. But the solution was to get a “better” version prebuilt wheel package of PyTorch. This is a relatively “easy” problem to solve (figuring out this was the problem does take time). But it’s (probably, I can’t afford one) going to be painful when you want to upgrade the cuda version or specify a specific version. Unlike a typical PC, you’re going to need to build a new image and flash it. I would be more impressed when a LLM can do this end to end for you.
sh3rl0ck•16h ago
Pytorch + CUDA is a headache I've seen a lot of people have at my uni, and one I've never had to deal with thanks to uv. Good tooling really does go a long way in these things.
Although, I must say that for certain docker pass through cases, the debugging logs just aren't as detailed
ComputerGuru•47m ago
uv doesn’t fundamentally solve the issues. It didn’t invent venv or pip.
What fundamentally solves the issue is to use an onnx version of the model.
simonw•37m ago
Do you know if it's possible to run ONNX versions of models on a Mac?
I should try those on the NVIDIA Spark, be interesting to see if they are easy to work with on ARM64.
BoredPositron•1h ago
Compute well spent... finding out to download a version and hardware appropriate wheel.
syntaxing•1d ago
sh3rl0ck•16h ago
Although, I must say that for certain docker pass through cases, the debugging logs just aren't as detailed
ComputerGuru•47m ago
What fundamentally solves the issue is to use an onnx version of the model.
simonw•37m ago
I should try those on the NVIDIA Spark, be interesting to see if they are easy to work with on ARM64.