AI Lab · Free tool
Local AI hardware calculator: which machine for which model?
Pick an open model (Qwen, Gemma 4, Llama, gpt-oss, Mistral, DeepSeek), a quantisation and a context length: the tool works out the memory needed, says which machines are enough and estimates the speed.
🔒 Everything is computed in your browser: nothing is sent to or stored on a server.
1. The model
Without a known architecture, the KV cache is estimated conservatively (8 KV heads of 128, number of layers derived from the size).
2. The machine
3. The verdict
Every quantisation on your machine
Every machine for this setting
How the calculation works
All values are estimates: they give a reliable order of magnitude to choose, not a measurement.
- Weights = parameters × bits per weight ÷ 8
- KV cache = 2 × KV heads × head dimension × bytes × users × (full layers × context + windowed layers × min(context, window))
- Total = (weights + KV cache) × (1 + margin)
- Theoretical max speed ≈ memory bandwidth ÷ weights read per token (active weights for an MoE)
Understand it in 4 ideas
Quantisation
Lowering the precision of the weights (16 bits → 8, 5, 4 bits…) to divide the memory needed. Q4_K_M is the most used trade-off: about 3.3 times lighter than 16-bit, with a modest quality loss. Below Q3, quality drops sharply.
KV cache and context
To remember the conversation, the model keeps "keys" and "values" in memory for every token. This cache grows with the context length and the number of users: at 128,000 tokens it can exceed the size of the model itself.
MoE models (mixture of experts)
An MoE holds many parameters but activates only a small share per token (the "A" in 35B-A3B). You need memory for ALL the parameters, but speed depends mostly on the active ones: an MoE is fast for its size.
Hybrid attention
Recent models (Gemma 4, gpt-oss, Qwen3.5 and 3.8, Mistral Small 4) limit the cache: sliding-window layers, linear attention or a compressed cache (MLA). The tool reads these details from each model's official card, hence much lower needs at long context.
Honest limits
- The speed shown is a theoretical maximum for text generation. Real speed depends on the software (Ollama, LM Studio, llama.cpp, vLLM, MLX), drivers and prompt length; it is often 50 to 80% of that maximum.
- On a Mac, macOS gives the GPU only part of the unified memory (about two thirds up to 36 GB, three quarters above; observed values, adjustable via iogpu.wired_limit_mb).
- Partial offloading to the PC's RAM works but slows things down a lot: each token then reads part of the weights at RAM speed.
- Multimodal models (image, audio) need a little extra memory for their encoder.
- Gemma 4 E4B has 8B parameters including its per-layer embeddings; some software keeps them off the GPU and uses less.
Read the guide: local or cloud AI? → Compare with the cost of a cloud API →
Sources (checked October 2026)
- llama.cpp: bits per weight of each quantisation
- Apple: memory and bandwidth of M5 MacBook Pro
- NVIDIA: DGX Spark
- AMD: Ryzen AI Max+ 395
- Official model cards (config.json): Qwen3 4B, Gemma 4 E4B, Llama 3.1 8B, Qwen3 8B, DeepSeek-R1-0528-Qwen3-8B, Qwen3.5 9B, Gemma 4 12B, Qwen3 14B, gpt-oss-20b, Mistral Small 3.2 24B, Gemma 4 26B-A4B, Qwen3.8 27B, Gemma 4 31B, Qwen3 32B, Qwen3.5 35B-A3B, Llama 3.3 70B, Llama 4 Scout (109B-A17B), gpt-oss-120b, Mistral Small 4 (119B-A6.5B), Qwen3 235B-A22B
Why this tool
Real architectures
Layers, KV heads, sliding windows, linear attention and MoE are read from each model's official card: the cache is computed precisely, not with a rough rule of thumb.
A verdict for every machine
NVIDIA cards, M5 Macs and M3 Ultra, DGX Spark, Ryzen AI Max+ or a plain PC: fits in memory, fits with offloading (slow) or does not fit.
The right quantisation
The tool suggests the most precise quantisation that fits your machine, so you do not sacrifice quality for nothing.
Frequently asked questions
How much memory does an 8-billion-parameter model need?
In Q4_K_M, about 5 GB for the weights, plus the cache depending on context: an 8 GB card is enough for a conversation, 12 to 16 GB is comfortable for long contexts.
Can a 16 GB Mac run a local AI?
Yes, models up to 8 or 9 billion parameters in 4-bit. macOS gives the GPU only about two thirds of the memory: keep some headroom for the system.
Why is an MoE model faster than a dense model of the same size?
Because it reads only part of its weights for each token. It still needs enough memory for all its parameters.
Is the speed shown guaranteed?
No. It is a theoretical maximum computed from memory bandwidth; real speed is often 50 to 80% of that value depending on the software.
Going further
Local or cloud AI: how to choose for your needs and budget
Read the guide →Play Store listing generator
Open the tool →llms.txt and AI robots
Open the tool →Coding agent instructions file
Open the tool →Prompt library by profession
Open the tool →Automation ROI calculator
Open the tool →AI API cost calculator
Open the tool →Anonymiser before sending to an AI
Open the tool →Business AI assistant designer
Open the tool →Company AI usage policy
Open the tool →Bring AI into your business, for real?
Automations, assistants, multilingual content, apps: I design solutions that are useful, measurable and respectful of your data.