ds4

DS4 is a single-model inference engine: it runs DeepSeek V4 Flash locally on Apple Silicon (Metal) or Linux (CUDA), and only that model. Antirez, the creator of Redis, built it as a focused experiment. The point is to run a 284B-parameter frontier-class model on a Mac Studio or a high-end Linux box without going through llama.cpp's generic GGUF loader. With 2-bit quantization the q2 build will fit on a 128GB Mac, q4 needs 256GB plus.

Setup is a download script and a make. You get a CLI with `/think` and `/nothink` modes, and an OpenAI- and Anthropic-compatible HTTP server that drops into any client that already speaks those APIs. On a Mac Studio M3 Ultra Antirez reports 84 tokens/sec prefill and 37 tokens/sec generation at 2-bit. Context window goes up to 1 million tokens.

Use this if you specifically want DeepSeek V4 Flash running locally on serious hardware. The appeal is sovereignty, not portability. Solo developers with a Mac Studio: this is a fun way to burn GPU hours. Anyone else: stick with llama.cpp or vLLM until DS4 ships a stable release.

The catch: this is alpha code, by Antirez's own admission. He notes the implementation leans heavily on GPT 5.5 assistance and acknowledges debt to llama.cpp. One model, one workload, no production claims. Treat it accordingly.