Answers · a straight answer, sourced
What is the best small local model to run on my own machine?
There is no single best small local model — the right one depends on how much RAM you have and what you are doing (general chat, reasoning, coding, or embeddings) — so the useful answer is a dated, opinionated shortlist by job, checked against your machine, rather than a leaderboard that is stale the week you read it.
As a starting point in late 2026, by job: for everyday chat, Gemma 3 4B or Qwen3 8B if you have the RAM; for reasoning, Qwen3.8-27B is the strongest open dense model a 32 GB machine can run (tight there, comfortable from 48); for coding, Qwen2.5-Coder 7B; for embeddings (semantic search over your documents), nomic-embed-text. All run under Ollama on modest hardware. Treat any such list as a snapshot — verify the current version and its licence on Hugging Face before relying on it.
The pick that matters most is the one that fits: a model that needs more RAM than you have will swap to disk and crawl, or fail to load. Match the model to your memory first (see how much RAM do I need), then choose within what fits. A model that fits and is a little weaker beats a stronger one that thrashes.
SovereignAI keeps a curated starter shelf organised by job, each entry dated and carrying its licence, and sizes every one against the machine it is running on before you download anything — so "best" becomes "best that actually runs here", which is the only version of the question worth answering.
Own the answer, not just read it
SovereignAI is private AI you run yourself: local models, memory with receipts, disclosure at the moment anything leaves, and an exit you can verify. The trial is one Docker command, and deleting the volume deletes every trace of you.