Rule of thumb that holds up well: model file size + room for context + a bit for the process.

• 7-8B at 4-bit: ~4-5 GB file, so 8 GB total is comfortable
• 13B at 4-bit: plan ~12 GB
• 70B: not a CPU thing, don’t bother

The one people forget is context. Cranking num_ctx (or OLLAMA_CONTEXT_LENGTH) up because the model “supports 128k” eats memory fast, and the usual symptom is a model that loads fine and then dies halfway through a long answer.

On CPU, expect roughly 5-15 tok/s for a 4-bit 7-8B on decent cores. Memory bandwidth matters more than core count after a point.

ARM is a non-issue these days for the usual self-hosted stuff; most popular images ship arm64 builds (check the tags on Docker Hub before you commit to one). For a box at your mum’s, I’d honestly consider staying on macOS + Docker or OrbStack rather than Asahi, purely because remote fixes are easier when the OS is the boring one. Add Tailscale so you can reach it without opening ports. 8 GB is plenty for things like a photo backup, an ad blocker, or a small search engine like SearXNG; it’s tight only if you try to run local LLMs on it.

What’s stuck for me after trying too many things:

• Ollama (or llama.cpp server if you want control over flags) as the model server, bound to localhost only
• Open WebUI as the front-end; it also handles OpenAI-compatible APIs, so local and hosted models live in the same history
• LiteLLM in between once you have more than one backend: one endpoint, per-key budgets, and logs
• SearXNG as the search tool for anything agent-ish; keep concurrency low or upstream engines start blocking it

Biggest lesson: RAM decides what you can run, and document upload/embeddings in Open WebUI eat more memory than chat does. On small boxes, point embeddings at an API or give it 4 GB.