It ships with auto-mode, which makes a good tradeoff between memory usage and speed.
I'll be implementing and porting the MTP module for speculative decoding next
Local models really are the future of computing!
It ships with auto-mode, which makes a good tradeoff between memory usage and speed.
I'll be implementing and porting the MTP module for speculative decoding next
Local models really are the future of computing!