We've been working on a project to push local LLM training/inference as far as possible on Apple hardware. It's called PMetal ("Powdered Metal") and its a full featured fine-tuning & inference engine built from the ground up for Apple Silicon.

It's hardware aware (detects GPU family, core counts, memory bandwidth, NAX, UltraFusion topology on M1–M5 chips)

Full TUI and GUI control center (Dashboard, Devices, Models, Datasets, Training, Distillation, Inference, Jobs, etc…)

Models like Llama, Qwen, Mistral, Phi, etc. work out of the box!

It's dual-licensed MIT/Apache-2.0, with very active development (just tagged v0.3.7 today), and I'm dogfooding it daily on M4 Max / M3 Ultra machines.

Would love feedback from the community, especially from anyone fine-tuning or running local models on Apple hardware.

Any models/configs you'd like to see prioritized?

Comments/Questions/Issues/PRs are very welcome. Happy to answer questions!