Weights
parameters × effective bytes per weight8.03B × 0.60 bytes
4.49 GiB
AI Wiki tools
LLM VRAM and RAM calculator
Estimate whether a local language model can fit your hardware. See model weights, KV cache, runtime reserve, ideal per-GPU split, and headroom—with every formula and assumption shown.
No API keyInputs stay in your browserCapacity estimate, not a benchmark
Private, browser-only calculation
Inputs are calculated on this device. Nothing is uploaded, saved, or connected to a model provider.
Configuration
Model and hardware
Use available memory, not the advertised total reserved for the operating system or display.
Planning estimate
Llama 3.1 8B at 4-bit
8,192 tokens · 1 concurrent sequence
Estimated total allocation
6.69 GiB
Weights + maximum KV cache + a visible runtime planning reserve
WeightsKV cacheRuntime reserve
- Model weights
- 4.49 GiB
- 0.60 effective bytes/parameter
- KV cache
- 1.00 GiB
- 2 bytes/cache element
- Runtime reserve
- 1.20 GiB
- Workspace and allocation cushion
- Ideal split per GPU
- 6.69 GiB
- Total divided evenly across 1 GPU
Capacity checks
“Tight” means less than 15% capacity remains. Real runtimes can need more.
Full GPU
Fits with reserve
6.69 GiB required per GPU
17.3 GiB headroom
RAM only
Fits with reserve
6.69 GiB required in system memory
25.3 GiB headroom
VRAM + RAM
Fits with reserve
Rough offload capacity, not a speed estimate
49.3 GiB headroom
Transparent methodology
How this estimate is calculated
All values use binary GiB (1,073,741,824 bytes). Quantized bytes per weight include a planning allowance for scales and metadata; actual file formats vary.
KV cache
2 × layers × KV heads × head size × context × concurrency × cache bytes2 × 32 × 8 × 128 × 8,192 × 1 × 2
1.00 GiB
Runtime reserve
max(1 GiB, 10% of weights + 0.25 GiB/sequence + 0.5 GiB/GPU)10% of weights + 1 sequence(s) + 1 GPU(s)
1.20 GiB
Important limitations
- This is a capacity estimate, not a performance benchmark or a guarantee that a particular runtime will load the checkpoint.
- KV cache assumes every concurrent sequence reaches the selected context length. Prefix caching, paged attention, sliding windows, or quantized KV can change it substantially.
- Presets are convenience approximations. Fine-tunes, conversions, multimodal adapters, and checkpoint revisions can have different tensors or architecture settings.
- RAM and VRAM are not always freely additive. CPU offload can be much slower, some tensors must stay together, and loaders may temporarily map or copy weights in both places.
- Leave extra capacity for the OS, display, runtime, drivers, and long-lived fragmentation. A result marked “tight” deserves a smaller context, lower concurrency, or more memory.
Reading the result
Memory capacity is only the first constraint
A model fitting in memory does not establish usable generation speed. Memory bandwidth, compute throughput, prompt processing, CPU offload, GPU interconnects, runtime kernels, and quantization format can all change performance. Treat this calculator as a first-pass capacity plan before testing the exact checkpoint and inference engine.
Weights are persistent
Every stored parameter consumes memory. Mixture-of-experts models still need all expert weights loaded even when each token activates only a subset.
KV cache grows with use
Longer contexts and more simultaneous sequences can turn a model that fits at idle into one that runs out of memory under load.
Headroom matters
Drivers, graph capture, temporary buffers, adapters, and fragmentation vary by runtime. The calculator flags less than 15% free capacity as tight.
Methodology source and scope
The weight-memory method follows the same core idea documented by the official Hugging Face Accelerate model memory estimator: estimate tensor storage from model configuration and dtype. Hugging Face explicitly notes that loading memory is not the full inference requirement, so this calculator separately adds KV cache and a visible runtime reserve. Exact runtime measurements remain authoritative.
Presets and planning factors last reviewed 2026-07-23. They are approximations, not vendor guarantees or hardware recommendations.
Common questions
LLM memory calculator FAQ
How much VRAM does an LLM need?
The largest component is usually model weights: total parameters multiplied by the effective bytes stored per parameter. KV cache then grows with layers, KV heads, attention head size, context length, cache precision, and the number of concurrent sequences. The runtime also needs workspace and allocation headroom.
Why is a 4-bit model estimated above 0.5 bytes per parameter?
Four raw bits equal 0.5 bytes, but quantized files also store scales, block metadata, alignment, and sometimes higher-precision tensors. This calculator uses 0.60 effective bytes per parameter as a planning approximation rather than pretending every 4-bit format has the same size.
Can system RAM make up for missing VRAM?
Some runtimes can offload model layers to system RAM, so combined capacity may be enough to load a model. That does not mean it will be fast: CPU offload, memory bandwidth, interconnect speed, tensor placement, and backend support can dominate performance.
How should Apple Silicon unified memory be entered?
Do not count the same shared pool as both VRAM and RAM. Set GPU count to zero and enter the memory actually available to the inference process as usable system RAM. Leave room for macOS, applications, and the display.