LocalAI

LocalAI runs your own AI models locally and exposes them through an OpenAI-compatible API. LLMs, image generation, speech-to-text: all from a single server. No cloud, no API keys, no data leaving your machine. MIT-licensed, free.

Docker-based setup handles most of the complexity. A config file defines which models to load and which backends to use (llama.cpp, whisper, stable diffusion, and more). CPU inference is supported, which means any machine can run it. GPU acceleration is faster but not required. Models download at first startup.

Developers who want to swap out OpenAI API calls with local models point their existing code at LocalAI's endpoint and change nothing else. Good for privacy-sensitive applications, air-gapped environments, and teams that want to control costs without changing application code.

The catch: local inference is slower than cloud for most hardware setups. Model selection lags the frontier. You get privacy and cost control; you give up raw performance and convenience.