How much VRAM do I need to run an abliterated model?

At the Q4_K_M quantisation most people use locally, budget roughly 0.6 GB per billion parameters plus about 1 GB for the KV cache and runtime. A 7B model needs around 5 GB, a 13B around 9 GB, a 32B around 20 GB and a 70B around 43 GB.

Does abliteration change the hardware requirements?

No. Refusal ablation edits the existing weights rather than adding any, so an abliterated model needs exactly as much memory as the base model it was derived from, at the same quantisation.

Can I run a model that is larger than my VRAM?

Yes, with CPU offload. llama.cpp keeps some layers in system RAM and the rest on the GPU, so anything fitting in VRAM plus RAM will run — but generation slows to single-digit tokens per second once a meaningful share of the layers sit on the CPU.

Do mixture-of-experts models need less memory?

No. Only the compute is sparse. Every expert must be resident in memory, so a 35B model with 3B active parameters still needs memory for all 35B.

Can I run it? Abliterated model hardware checker

Enter your GPU memory, system RAM and quantisation to see which abliterated models your machine can actually run, with estimated VRAM for each. Every estimate below is derived from the parameter count, not measured — Hugging Face publishes no file sizes through its listing API — so treat them as a planning figure with a little margin, not a guarantee.

Graphics cardVRAM (GB)System RAM (GB)QuantisationContext window

Apple Silicon (unified memory)GGUF only (Ollama, llama.cpp)

Budget: 12 GB usable · 348 of 751 models fit

Largest model your device can run

Qwen3-14B-abliterated

14B parameters · ~9.3 GB · 69.9K downloads

Most-used models that fit

Qwen3.5-9B-The-Defiant-Fable-Uncensored-HereticQwen·~6.2 GB·48% headroom·↓ 2M Ornith-1.5-9B-uncensoredOrnith·~6.2 GB·48% headroom·↓ 1.8M Cyber-Ornith-1.5-9B-OBLITERATEDOrnith·~6.2 GB·48% headroom·↓ 941.2K Qwen3-VL-8B-Instruct-abliteratedQwen·~5.6 GB·53% headroom·↓ 739K Qwen3.5-9B-Uncensored-HauhauCS-AggressiveQwen·~6.2 GB·48% headroom·↓ 702.4K Gemma4-12B-QAT-Uncensored-HauhauCS-BalancedGemma·~8.1 GB·33% headroom·↓ 696.7K Qwen3.5-9B-abliteratedQwen·~6.2 GB·48% headroom·↓ 546.2K Spark-x2.5-4B-Heretic-jpOther·~3.2 GB·73% headroom·↓ 452.1K Meta-Llama-3-8B-Instruct-abliterated-v3Llama·~5.6 GB·53% headroom·↓ 371.5K Qwen2.5-VL-7B-Instruct-abliteratedQwen·~5 GB·58% headroom·↓ 352.8K Ornith-1.5-9B-OBLITERATEDOrnith·~6.2 GB·48% headroom·↓ 332.3K Qwythos-9B-v2Other·~6.2 GB·48% headroom·↓ 315K Qwen3.5-4B-Uncensored-HauhauCS-AggressiveQwen·~3.2 GB·73% headroom·↓ 268.2K Qwythos-9B-Claude-Mythos-5-1MOther·~6.2 GB·48% headroom·↓ 235.5K gemma-4-12B-it-uncensoredGemma·~8.1 GB·33% headroom·↓ 129.8K

Runnable with CPU offload

These exceed your VRAM but fit in VRAM plus system RAM. llama.cpp will run them by keeping some layers on the CPU, at a large speed cost — expect single-digit tokens per second rather than tens.

Qwen3.8-27B-abliterated~17.2 GB needed·↓ 3.9M Qwen3.6-35B-A3B-abliterated~22.1 GB needed·↓ 3.6M Qwen3.8-27B-Uncensored~17.2 GB needed·↓ 3.3M Qwen3.8-27B-Uncensored-HauhauCS-Aggressive~17.2 GB needed·↓ 2.4M Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER~17.2 GB needed·↓ 1.4M Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive~22.1 GB needed·↓ 1.2M

What each graphics card can run

Model counts at Q4_K_M with a 4,096-token context, cumulative — a card that runs a 7B model also runs everything smaller.

GPU memory	Typical cards	Models that fit	Downloads
4 GB	GTX 1650, MX570	130	↓ 2.8M
6 GB	RTX 2060, 3050	213	↓ 6M
8 GB	RTX 3070, 4060	278	↓ 13.9M
12 GB	RTX 3060 12GB, 4070	348	↓ 16M
16 GB	RTX 4080, 4060 Ti 16GB	386	↓ 16.9M
24 GB	RTX 3090, 4090	712	↓ 49.2M
48 GB	RTX 6000 Ada, dual 3090	729	↓ 49.6M
80 GB	A100, H100	750	↓ 50.1M

How the estimate is calculated

Total memory is the weights, plus the KV cache, plus a fixed runtime overhead of about 0.6 GB for the CUDA context, activations and allocator slack. The weights dominate and scale directly with the quantisation:

Q4_K_M — 4-bit — 0.6 bytes per weight (4.2 GB for a 7B model) · the local default
Q5_K_M — 5-bit — 0.71 bytes per weight (5.0 GB for a 7B model) · slightly better fidelity
Q6_K — 6-bit — 0.82 bytes per weight (5.7 GB for a 7B model) · near-lossless
Q8_0 — 8-bit — 1.06 bytes per weight (7.4 GB for a 7B model) · effectively lossless
FP16 — unquantised — 2 bytes per weight (14.0 GB for a 7B model) · full precision

The KV cache grows with context length and, for the grouped-query attention every current model uses, with roughly the square root of the parameter count — about 0.5 GB for a 7B model at 8k tokens, 1.3 GB for a 70B. Long contexts are what turn a comfortable fit into an out-of-memory error, which is why the context selector matters as much as the quantisation.

Common questions

How much VRAM do I need to run an abliterated model?: At the Q4_K_M quantisation most people use locally, budget roughly 0.6 GB per billion parameters plus about 1 GB for the KV cache and runtime. A 7B model needs around 5 GB, a 13B around 9 GB, a 32B around 20 GB and a 70B around 43 GB.
Does abliteration change the hardware requirements?: No. Refusal ablation edits the existing weights rather than adding any, so an abliterated model needs exactly as much memory as the base model it was derived from, at the same quantisation.
Can I run a model that is larger than my VRAM?: Yes, with CPU offload. llama.cpp keeps some layers in system RAM and the rest on the GPU, so anything fitting in VRAM plus RAM will run — but generation slows to single-digit tokens per second once a meaningful share of the layers sit on the CPU.
Do mixture-of-experts models need less memory?: No. Only the compute is sparse. Every expert must be resident in memory, so a 35B model with 3B active parameters still needs memory for all 35B.

Browse the full catalogue on AbliteratedModels.org.