llama.cpp

The C++ engine most local language model software is built on. Runs large models on ordinary hardware, including with no GPU at all.

Self-hosted
Active
Allowed adult content
Open source

Website Source

At a glance

Where it runs: Self-hostedYou run it on your own machine. Nothing leaves it unless you send it.
Adult content: AllowedAdult content is permitted within the published rules.
Trains on your content: NoYour content is not used to train models.
Status: ActiveMaintained and working today.
Account needed: No
Licence: MIT
Cost: Free and open source.
What it keeps: Local only.

What it takes to run

Graphics memory: No GPU needed
System memory: 8 GB, 16 GB recommended
Disk: about 10 GB
Runs on: Linux, macOS, Windows

Find what fits your machine on the tools page — pick your hardware from the first menu.

The project that made local language models practical outside a datacentre. Its quantisation format, GGUF, is why a 70-billion-parameter model fits on a consumer graphics card at all, and why the file you download from Hugging Face has a name ending in Q4_K_M.

Most people meet it indirectly — Ollama, LM Studio and KoboldCpp are all wrappers around it or its ideas. Using it directly gets you every sampler setting and no hand-holding.

Worth knowing before you start

Command-line first The bundled web UI is minimal; it expects you to be comfortable with flags.
Moves very fast Build instructions from six months ago are frequently out of date.

What it does

GGUF quantisation
CPU and GPU inference
Metal, CUDA, Vulkan and ROCm backends
Built-in server

Models it runs

How it relates to other tools

has a frontend Ollama — uses it as one of its engines
has a frontend KoboldCpp

Signs of life

Checked automatically. These are the only figures on this page a machine wrote, and they say when they were taken.

Last answered: Yes, 3 hours ago Its website responded when we asked.
Stars on GitHub: 128,368 23,235 forks.
Last commit: 6 hours ago2026-09-15 23:42 UTC
Latest release: v0.4.1
Archived copy: Wayback Machine A snapshot, in case this one stops answering.