Run it yourself

Runtimes and model servers: the layer that turns a downloaded model into something you can talk to.

Nothing in this category is adult-oriented by itself. They are here because they are the honest answer to "how do I use this without sending it to anyone", and because what you run on them is nobody's business.

10 tools

Ollama

One command to download and run a language model on your own machine. The easiest honest answer to "how do I do this without a cloud".
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
GPT4All

A local chat desktop app with a long history, aimed squarely at ordinary hardware and people who do not want a terminal.
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
Jan

An open-source desktop assistant that runs models locally by default and can be pointed at a hosted one when you choose.
- Hybrid
- Allowed
- No GPU needed
Run it yourself No GPU needed
KoboldAI

The original open-source AI writing interface. Largely superseded by KoboldCpp, and the ancestor of much of this section.
- Self-hosted
- Abandoned
- Allowed
- No GPU needed
Writing No GPU needed
KoboldCpp

A single-file local model server built for storytelling and roleplay, with the sampler controls that community actually uses.
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
LM Studio

A desktop app for finding, downloading and chatting with local language models. Free, closed source, and the gentlest way in.
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
OpenRouter

One API key for a great many hosted models, including ones with looser content policies than their originals.
- Hosted
- Unknown
Finding models Not known
Text generation web UI

The "AUTOMATIC1111 of language models": one interface, many backends, an extension for everything.
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
llama.cpp

The C++ engine most local language model software is built on. Runs large models on ordinary hardware, including with no GPU at all.
- Self-hosted
- Allowed
- No GPU needed
Run it yourself No GPU needed
vLLM

A serious inference server for serving a model to many people at once. Not a chatbot — the thing a chatbot runs on.
- Self-hosted
- Allowed
Run it yourself 16 GB VRAM, 24 GB recommended