guides

Local AI

Run AI on your own hardware. The cluster: Ollama, LM Studio, llama.cpp, Hugging Face Hub. MIT where it matters, MLX on Apple Silicon, GGUF everywhere else.
by cache.directory updated Sep 10, 2026 CC-BY-4.0
← back to shelf

## what it does

tl;dr: “local AI” lands on four things — a runtime (Ollama), a GUI agent (LM Studio), an engine (llama.cpp), and a catalog (Hugging Face Hub). You do not pick one forever; the cluster composes. We do not invent a fourth runtime or mill GGUF files here.

Quick pick

You wantUseLicense (this fetch)
CLI + REST API on :11434, Docker, scriptsOllamaMIT
Desktop agent (docs, slides, skills, MCP, GUI)LM StudioProprietary
GGUF engine, every backend, llama-serverllama.cppMIT
Catalog + CLI + downloads for any open weightHugging Face HubApache-2.0

Current versions (fetched 2026-09-10)

ToolCurrentSource
Ollamav0.34.0 (Sep 5, 2026)GitHub releases
LM StudioBionic 1.1.2 (Sep 8, 2026)lmstudio.ai/changelog
llama.cppv0.4.0 stable, with rolling b10894 pre-release (Sep 10, 2026)GitHub releases
Hugging Face Hub (client)huggingface_hub rollingdocs

llama-server ships with the same binary as llama-cli; the rolling b10xxx pre-release tags are upstream’s daily cut. Treat v0.4.0 as the current stable.

How the cluster composes

llama.cpp          ← engine (GGUF, Metal/CUDA/Vulkan/…)
    ↑
Ollama             ← library + CLI + :11434
LM Studio runtime  ← MLX + llama.cpp packs inside the app
    ↑
Open WebUI         ← chat UI on top of any of the above (separate shelf)
Hugging Face       ← catalog of weights; not a runtime

Apple Silicon: both Ollama and LM Studio use the MLX path. NVIDIA: Ollama Docker or llama.cpp CUDA. AMD: HIP or ROCm llama.cpp, or Ollama’s ROCm tarball. CPU-only: works for everything; expect 4-bit quant and small models.

By job

JobPickWhy
Backend for an app, CI, DockerOllamaHeadless, MIT, official install + Docker
Claude Code / Codex against local weightsOllama (ollama launch claude)Documented on ollama.com
Will compile flags yourself, want every backendllama.cppllama-cli / llama-server; repo moved to ggml-org/llama.cpp
GUI agent for docs / slides / codeLM StudioBionic is the agent layer; Linux landed in 1.1.2
Pick a weight, check the card, downloadHugging Face Hubhf CLI + Python client

What “local AI” usually means here

A query for “local AI” in 2026 lands on local LLMs. We do not catalog local image gen, local TTS, or local video models on this site — those are separate niches and the cluster above is already the four big ones for inference. If your task is text generation, summarization, code, embeddings, or chat, one of the four above runs it.

For a side-by-side of runtimes, see best local LLM and Ollama vs LM Studio.

Gotchas (from the fetched pages)

  • API binds localhost with no auth by default on Ollama (:11434) and llama-server (:8080). Do not expose to the internet without a reverse proxy.
  • MLX is Apple Silicon only. CUDA is NVIDIA. ROCm/HIP is AMD. Vulkan works on most GPUs but is slower. Pick the backend that matches your hardware before tuning flags.
  • LM Studio is proprietary and closed-source. Use Ollama if that matters.
  • hf download still hits the network unless HF_HUB_OFFLINE=1. Gated models need hf auth login.
  • Cloud tier ≠ local. Both Ollama and LM Studio now have cloud products. Local is still free and never leaves the machine; cloud routes through their hosted capacity.
  • Stars on the website are not always the same number as the SDK repo. huggingface_hub is ~3.9k stars on GitHub — that is the CLI, not the Hub site. Don’t conflate them.

## embed this badge

cache ✓ in cache.directory
![cached](https://cache.directory/badge/local-ai.svg)