This article explores trends and tools for deploying large language models (LLMs) in on-premises environments. The authors believe that localized deployment is becoming increasingly important due to the needs of data privacy, cost control, and low latency.

The article summarizes a series of tools covering different needs from individual developers to enterprise-level: Ollama, LM Studio, Jan.AI, LocalAI and LLMOne.

This is a good starting point and overview for those considering how to implement LLMs on their own servers or devices.