Local LLM VRAM Calculator
Estimate the video memory required to load and run a local large language model based on parameter count, quantization and context length.
Runs locally in your browser
Includes model weights, a rough KV-cache allowance for your context length, and runtime overhead. Leave 1–2 GB extra headroom for the OS and other apps.
Discuss a result
Write a short summary and review it before posting. Your tool inputs are never copied automatically.
What is the Local LLM VRAM Calculator?
This tool estimates how much GPU memory a local LLM needs to load its weights plus a working allowance for the KV cache at a given context length. It performs arithmetic only and does not run or call any model.
How to use it
- 1Enter the model size in billions of parameters.
- 2Choose a quantization (bits per weight).
- 3Set your context length.
- 4Read the estimated VRAM requirement.
How it works
Weight memory = parameters × bits / 8. A KV-cache allowance scales with context length and model size, and a fixed overhead is added for activations and the runtime.
Limitations & notes
- Real usage varies by runtime (llama.cpp, vLLM, Transformers) and settings.
- Estimates assume a single GPU; multi-GPU splitting changes the picture.
Frequently asked questions
How much VRAM do I need to run a 7B model?
At 4-bit quantization a 7 billion parameter model's weights take about 3.3 GiB. With an 8,000 token context and 0.8 GiB of runtime overhead, this calculator estimates about 5 GiB in total, so it fits on a 6 GB or 8 GB card. At FP16 the same model needs about 13 GiB for weights alone, which pushes it to a 16 GB card or partial CPU offload.
Which quantization option matches GGUF files like Q4_K_M?
GGUF K-quants store some tensors at higher precision, so their real average is above the nominal number. Q4_K_M averages roughly 4.8 bits per weight and Q5_K_M roughly 5.7. The tool offers fixed 4, 5, 6, 8, 16 and 32-bit choices, so pick the next option up for a conservative estimate, for example 5-bit for a Q4_K_M file. Checking the actual file size on disk is also a good guide to weight memory.
Why does longer context use so much more memory?
The KV cache stores attention keys and values for every token in the context, so it grows linearly with context length. This tool's allowance is about 0.12 GiB per 1,000 tokens for a 7B model, scaled by model size. For a 70B model at 8,000 tokens that is roughly 9.6 GiB on top of the weights. Models with grouped-query attention or a quantized KV cache can use considerably less than this rough allowance.
Can I run a model that is larger than my VRAM?
Yes, with trade-offs. Runtimes such as llama.cpp can offload some layers to system RAM and the CPU, but every offloaded layer is much slower, often turning tens of tokens per second into a few. Splitting across two GPUs pools their memory. A 70B model at 4-bit needs about 33 GiB for weights, so it fits on two 24 GB cards but not one.
What is the runtime overhead field for?
It covers memory the runtime reserves beyond weights and the KV cache: CUDA or Vulkan context, scratch buffers for activations and the inference engine itself. The default is 0.8 GiB. If your GPU also drives your monitors, the desktop and browser can take another 0.5 to 1.5 GiB, so add that here or leave free headroom. The estimate is arithmetic only; the tool does not detect your GPU or load a model.
No directly related discussion yet.
Start a focused thread and share the question this tool helped you explore.
Ask the community about Local LLM VRAM Calculator