/local llm tools

The Hub is where open weights, datasets, and Spaces live. Search models by downloads, pull them with the hf CLI or huggingface_hub, then run them in Ollama, llama.cpp, or transformers.

Self-hosted AI platform with a chat UI on top of Ollama and any OpenAI-compatible API. Ships RBAC, RAG across 9 vector DBs, MCP/MCPO/OpenAPI tool servers, voice/video, channels, calendars, automations, and a native desktop client.

LLM and VLM inference in plain C/C++. The engine under Ollama, LM Studio's extension packs, and llama-server's OpenAI-compatible API. Targets everything from Apple Silicon to NVIDIA, AMD, Vulkan, SYCL, Hexagon, Ascend, and Moore Threads.

Open reasoning model family from DeepSeek. Base weights MIT-licensed, distilled into Qwen and Llama sizes from 1.5B up to 70B, plus the full 671B MoE. Current Ollama default `deepseek-r1` is the 0528-Qwen3-8B distill.

DeepSeek's hybrid reasoning base on Ollama: 671B MoE, supports both thinking and non-thinking modes from a single chat template, improved tool use, 160K context, text-only.

Google DeepMind's multimodal open family on Ollama: E2B / E4B for edge, 12B / 26B MoE / 31B dense for workstations. Text + image + audio (sizes vary), thinking mode, 128K–256K context.

OpenAI's open-weight reasoning MoE pair on Ollama: 20B and 120B, both quantized to MXFP4 with 128K context. Function calling, web search, configurable reasoning effort. Apache 2.0.

IBM's compact 2B vision-language model for visual document understanding. Extracts structured content from tables, charts, infographics, plots, and diagrams. Text + image, 16K context. License not stated on the Ollama page.

IBM's 2B and 8B text models with 128K context, fine-tuned for reasoning and instruction following. Apache 2.0, 12-language support, function calling, fill-in-the-middle for code. Released April 16, 2025.

Meta's open-weights LLM family in 8B, 70B, and 405B sizes. Multilingual, 128K context, tool use, strong reasoning for a first-generation release. 405B was the first openly available model competitive with GPT-4-class on Meta's evals.

Desktop app built around Bionic, an agent for work and code. Runs local models on MLX and llama.cpp, supports skills and MCP servers, and can reach frontier open models through a zero-data-retention cloud.

Mistral's 24B Apache 2.0 multimodal small model on Ollama: vision + tools + 128K context, ~150 tok/s on a single RTX 4090 or 32GB Mac.

Microsoft's 14B open-weight model on Ollama. Trained on synthetic + filtered public data with SFT and DPO. 16K context, text-only, 9.1GB quantized. Targets reasoning, math, and constrained-memory inference.

Alibaba's Qwen 3 family on Ollama: dense and MoE sizes from 0.6B to 235B. Tools and thinking modes on every size, 40K context on the dense 8B/14B/32B, 256K on 4B/30B/235B. 36.7M pulls, second only to Llama 3.1.

Microsoft's WizardLM-2 chat family on Ollama: 7B dense and 8x22B MoE. 32K context, text-only. README advertises complex chat, multilingual, reasoning, agent use cases. The 70B was listed as 'coming soon' on the page and never shipped to the library.

01.AI's bilingual (English/Chinese) Yi 1.5 family on Ollama. Three sizes: 6B, 9B, 34B. 4K context window, text-only, last updated on the library 2 years ago. Trained on 3T tokens.