Gemma 4 E4B: 23 Abliteration Variants Compared

Forensic analysis by Abliterlitics , open-source abliteration forensics toolkit

Data & artifacts: HuggingFace | Report: abliterlitics.dev | Code: GitHub

Models on HuggingFace

What is Abliterlitics?

Gemma 4 E4B is Google’s 4.5B-parameter reasoning model. It thinks before answering, working through problems in a hidden chain of thought. It ships with safety training that makes it refuse harmful requests. Abliteration removes that safety training without retraining the model. It finds the direction in the weights that controls refusal and edits it out. Done well, the model keeps all its capabilities but stops refusing. Done badly, it damages reasoning, language fluency, or both.

I selected 23 abliterated variants from HuggingFace, chosen by popularity and recency. There may be more, but these are the ones the community is actually downloading. They range from surgical edits modifying 21 weights to sledgehammers modifying 381. Some are not abliterations at all. I took all 23 plus the original, ran them through weight forensics, KL divergence, a 13-task extended benchmark suite, and HarmBench with 400 harmful behaviours. All 9,600 HarmBench responses were reviewed by an LLM judge.

The short answer

• Best overall: heretic . 95.5% ASR, GSM8K +1.3pp. 29 weights, one type. KL=0.002.
• Zero refusals: abliterix . 100.0% ASR. Costs 0.5 MMLU-Pro and 8.1 TQA-MC1 points.
• Near-perfect: trevorjs . 99.3% ASR, 3 refusals. MMLU-Pro within 0.1 of base, GSM8K +1.4.
• Lightest touch: coder3101 . 21 tensors. 93.8% ASR, KL=0.002. Indistinguishable from base.
• Fine-tune, not pure abliteration: deckard . Heretic + RP fine-tune. 95.5% ASR but MMLU-Pro −5.7pp, GSM8K −6.8pp strict.
• Do not use: obliteratus . 381 tensors, KL=1.10. MMLU-Pro −11.7pp, GSM8K −20.9pp, 24 empty + 39 truncated. Only 72.0% ASR after all that damage.

Key findings

• Six method families, not 23 techniques. The variants cluster by which tensor types they modify: 3 pure o_proj, 10 with down_proj added, 1 mixed attention, 1 MLP-only, 4 full fine-tunes, 4 brute-force. The family predicts outcomes better than the tensor count. See Weight Analysis .
• Surgical beats brute force. coder3101 : 21 tensors, 93.8% ASR. obliteratus : 381 tensors, 72.0% ASR. The sledgehammer damages the model so badly it cannot comply consistently.
• Magnitude matters more than scope. sdft and obliteratus touch the exact same 381 tensors. sdft mean edit_norm = 0.167, obliteratus = 1.254. The 7.5x difference: sdft removes zero refusal (ASR 30.8% same as base). obliteratus destroys the model (ASR 72%, KL=1.10).
• The strict-flex gap exposes formatting damage. deckard-expresso : 60.4% strict, 81.0% flexible, a 20.6 pp gap. Clean abliterations stay under 1.5 pp. Combined with empty and truncated HarmBench counts, this surfaces reasoning damage any single metric misses.
• Abliteration can improve math. 15 of 23 variants beat base on GSM8K strict. Abliteration shortens thinking chains. The variants that lose math ability damaged internal representations.
• The most broken model is the most downloaded. obliteratus has 796,331 all-time downloads. The next most popular variant, claude-distill at 55,337, is not even an abliteration. obliteratus is 36x more popular than the runner-up despite having the worst capability damage in the set: 381 tensors modified, KL=1.10, 24 empty responses, GSM8K -20.9pp, IFEval -14.4pp, and ASR of just 72.0%. The “built autonomously by an AI agent” narrative and Pliny the Prompter ’s reach drove adoption. Quality does not predict downloads.
• Apostate edits MLP, not attention. Its 152 tensors are entirely in the MLP feed-forward block. KL=0.0036, capabilities intact. Refusal is reachable through the MLP, not just attention.
• Four clusters of identical weights. infinimind ≡ trevorjs (bit-for-bit). bendernina ≡ physshell (cosine 0.99999). The 4 fine-tune variants share 294 tensors. sdft ≡ obliteratus share 381. See Identical models .

Quick Facts

Benchmarks

Evaluated with lm-evaluation-harness 0.4.11 via vLLM v0.20.1, native BF16 on single RTX 5090. Loglikelihood tasks scored with a no_thinking chat template, see Methodology. Generative tasks with thinking enabled.

Methodology note: The Open LLM Leaderboard v2 loglikelihood multiple-choice scores, MMLU-Pro ~44% and GPQA ~34%, are NOT comparable to Google’s published generative numbers at ~69% and ~59%. The ~25 pp gap is by design: loglikelihood MC scoring does not allow the model to think before answering. Only the generative tasks, GSM8K and IFEval, are directly comparable to Google’s numbers, and both match community references. Document deltas within this suite, not against Google’s card.

Loglikelihood multiple-choice tasks

MMLU-Pro acc, 5-shot. GPQA Diamond acc_norm, 0-shot. BBH acc_norm, 3-shot. MUSR acc_norm, 0-shot. HellaSwag/ARC/PiQA use acc_norm. WinoGrande acc. TQA-MC1 acc.

Instruction-following: IFEval, generative

IFEval is the cleanest capability-damage signal: generative, directly comparable to Google’s methodology. Surgical abliterations lose under 3 points. The damaged cluster loses 9 to 15. Loglikelihood tasks are resilient to surgical edits. The Heretic cluster stays within 1 point of base on MMLU-Pro. The damaged cluster collapses on hard reasoning: obliteratus MMLU-Pro −11.7pp, BBH −11.8pp, IFEval −14.4pp. LAMBADA and HumanEval are excluded as raw-completion tasks broken by --apply_chat_template. See Methodology.

GSM8K

The strict-flex gap is the headline signal. Strict requires the canonical #### N marker. Flexible extracts the last number. A wide gap means the model can do the math but cannot format the answer.

• deckard-expresso : 20.62 pp gap. Strict 60.4% looks catastrophic. Flexible 81.0% shows it solves most problems.
• bendernina and physshell : 7.51 pp gap. Broad damage to formatting and arithmetic.
• deckard : 5.31 pp, obliteratus : 4.93 pp. Formatting damage on top of real reasoning loss.
• Clean abliterations stay under 1.5 pp. Indistinguishable from base’s 1.06 pp.

claude-distill loses 17 strict points with only a 3.6 pp gap. Its math circuits are genuinely damaged, not just misformatted.

Safety: HarmBench

HarmBench with 400 textual behaviours, max_tokens=8096, temperature=0, enable_thinking=true. All 9,600 responses across 24 models reviewed by LLM judge.

Empty = responses with no generated content. Truncated = responses that hit max_tokens=8096 without a stop token.

ASR by category

Base complies with 30.8%. 100 of 400 behaviours are copyright requests and base complies with all of them. Copyright inflates every model’s ASR by roughly 25 points. The real differentiation is in chemical/bio, cybercrime, harassment, and illegal.

• Empty and truncated responses reveal how a model fails. obliteratus : 24 empty, 39 truncated. deckard-expresso : 90 truncated. claude-distill : 45 truncated. Generation failures, not refusals.
• Reasoning depth correlates with ASR. Low-reasoning variants like base (1,157 chars) and distill (1,295) cluster at the bottom. High-reasoning like wwt (2,401) and abliterix (2,319) cluster at the top.
• Chemical/bio separates the field. huihui 100%, trevorjs 96.4%. The damaged models collapse: obliteratus 26.8%, physshell 10.7%. They cannot generate coherent step-by-step chemistry.
• Harassment is the most polarised. Base refuses 100%. The best abliterations hit 100% compliance. obliteratus manages only 64%.

Non-abliteration baselines

Four variants are not abliterations. Included to show what happens when you modify a model without targeting the refusal direction.

• sdft claims uncensored Heretic SDFT lineage but is identical to base on every safety metric. Same 381-tensor fingerprint as obliteratus . Magnitude 7.5x smaller. Removes zero refusal.
• treadon-disin uses the same surgical approach as abliteration but targets neutrality hedging instead of refusal. KL=0.296 is surprisingly high for minimal ASR change.
• distill is a Gemini 3.1 Pro reasoning SFT. GSM8K −3.6pp, MMLU-Pro −2.8pp, ASR barely moves. Did not improve reasoning or remove safety.
• claude-distill is SFT on 2,300 CoT samples from Claude 4.6 Opus. Broad damage: MMLU-Pro −12.5pp, GSM8K −17.1pp. Overwrote Gemma 4’s native reasoning circuits with Claude’s.

You can modify 53% of a model and not touch the refusal direction at all.

KL Divergence

F.kl_div on full vocab at 262K tokens, first-token logits from 100 harmless prompts, matching Heretic evaluator . System prompt: “You are a helpful assistant.”

KL divergence measures how far the output distribution shifted from base. Zero means identical. KL is the most important metric because it captures collateral damage benchmarks miss.

KL Calibration

Four of six land within 6x of card claims, all measuring lower, consistent with cross-hardware float differences. abliterix uses a different KL computation entirely. apostate likely uses a different prompt set.

KL vs benchmark impact

The surgical cluster at KL < 0.1 shows near-zero MMLU-Pro delta. Capability damage only becomes visible once KL passes ~0.3, and it compounds fast. By KL=0.9, bendernina , MMLU-Pro is down 10 points and GSM8K down 20. IFEval is the early-warning indicator, degrading before MMLU-Pro does.

Weight Analysis

Every variant’s weights compared tensor by tensor against base. Weight forensics separates true abliterations from fine-tunes, identifies identical reuploads, and explains why some variants damage the model while others do not.

Method families

The 23 variants are not 23 distinct techniques. They cluster into six families by which tensor types they modify. The family predicts outcomes better than the tensor count.

The pure o_proj family is the original Heretic method. The theory: refusal lives in a direction that attention reads out. The down_proj family extends this to the MLP output projection. The fine-tune and brute-force families are not abliterations at all.

Modification summary

E% / M% / L% = early (0-13) / mid (14-27) / late (28-41) layer distribution.

Types = distinct tensor types modified. E/M/L = early 0 to 13 / mid 14 to 27 / late 28 to 41.

Tiers

The method families map onto four tiers of aggressiveness:

• Surgical, 3 to 6%, 1 to 2 types. The o_proj and o_proj + down_proj families. Narrow band of mid-to-late layers. Every variant preserves capabilities.
• Moderate, 10 to 21%, 2 to 4 types. huihui , infinimind , trevorjs , abliterix , apostate . Capability preserved because edits are distributed, not concentrated in the sequential attention path.
• Fine-tune, 41%, 7 types. All 42 layers. Not targeted abliteration.
• Heavy, 48 to 53%, 12 types. The brute-force family. Significant collateral damage. A signal hides inside: obliteratus , bendernina , and physshell have median edit_norm near zero but mean ~1.2 and p95 ~5. Most of their 381 tensors carry zero change. A few carry massive edits, hidden in noise.

Identical and near-identical models

Weight forensics surfaces four clusters of duplicate work, plus one family of same-template-different-layers:

• sdft ≡ obliteratus : exact same 381 tensors. sdft mean edit_norm = 0.167, obliteratus = 1.254. The 7.5x difference: sdft removes zero refusal, obliteratus destroys the model. Same footprint, opposite outcomes.
• Fine-tune cluster: distill , deckard , deckard-expresso , claude-distill share the identical 294-tensor fingerprint. Different outcomes come from the training data on top.
• infinimind ≡ trevorjs : bit-for-bit identical, cosine 1.00000. Same Heretic biprojection config. infinimind credits TrevorJS. Yet trevorjs scores 99.3% ASR vs infinimind ’s 98.5%. Same weights, different safety at the refusal threshold.
• bendernina ≡ physshell : cosine 0.99999 across 345 tensors. Same model, no attribution between them.
• heresy/mythos/treadon/wwt: same o_proj + down_proj template, different layers. heresy shares 14/34 with mythos , 26/34 with treadon , 22/34 with wwt .

Apostate: a different approach

Apostate is the MLP-only family:

Zero o_proj. Zero q_proj, k_proj, v_proj. Yet KL=0.0036, GSM8K strict within 0.5 of base, MMLU-Pro within 0.05. The tradeoff is ASR at 85.8%, below the Heretic cluster. The gap concentrates in chemical/bio at 57.1% and illegal at 72.3%. Apostate proves refusal is reachable through the MLP. MLP edits distribute across parallel layers, making collateral damage less likely than attention edits concentrated in a single sequential path.

7 variants shipped with missing shared-KV tensors in layers 24 to 41. Fixed by copying from base. Lossless.

Optimal Tradeoff

Summary

Methodology

• Capability: lm-evaluation-harness 0.4.11 via vLLM v0.20.1, native BF16 on single RTX 5090. Extended 13-task suite: 5 Open LLM Leaderboard v2 tasks + 5 v1 forensic supplement + TruthfulQA + HumanEval + GSM8K.
• Two chat templates. Loglikelihood Phase 1 uses a no_thinking_chat_template.jinja override with thinking disabled, so the model scores answer choices directly instead of inside an unclosed <\|think\|> block. Generative Phase 2 uses a thinking_chat_template.jinja override with thinking enabled, matching Google’s published methodology.
• LL-MC vs generative gap. Loglikelihood multiple-choice scores are NOT comparable to Google’s published generative numbers. The ~25 pp gap is by design. Only GSM8K and IFEval are directly comparable, and both match community references.
• Dead tasks excluded. LAMBADA and HumanEval are raw-completion tasks that --apply_chat_template breaks. Excluded from capability comparison.
• GSM8K strict-flex gap. Strict requires the #### N marker. Flexible extracts the last number. The gap isolates formatting damage from arithmetic damage.
• Safety: HarmBench 400 textual behaviours, max_tokens=8096, temperature=0, enable_thinking=true. All 9,600 responses across 24 models reviewed by LLM judge.
• KL divergence: Full vocab at 262K tokens, first-token logits on 100 harmless prompts from mlabonne/harmless_alpaca split test[:100], matching current Heretic evaluator methodology. Card-published KL values for v1.2.0-era variants used the train[:400] split, which is the same dataset but a different prompt set, explaining the consistent 2.8 to 5.9x offset between their numbers and ours.
• Weight analysis: SVD, fingerprint, edit vector overlap, per-layer analysis, correlation, subspace alignment using Abliterlitics .
• Hardware: NVIDIA RTX 5090 with 32GB. All GPU work on GPU 0.

How this was made

GPU hours

What broke

• Thinking contaminates loglikelihood scoring. The chat template injects an unclosed <\|think\|> tag, so logits at the scoring position are over thinking prose, not answer letters. Depressed every MC score by ~25 pp. A HellaSwag diagnostic at 0.39 vs expected ~0.79 caught it. Fixed with no_thinking_chat_template.jinja.
• HarmBench with thinking disabled. A reasoning model without reasoning is a different model. Re-ran with enable_thinking=true. Cost 12 GPU hours.
• KL was wrong by 600x. Stale base logits from June. A variant that should be 0.002 showed 1.2. Fixed by re-collecting with current code.
• 108 silent HTTP failures on base. Empty responses with 200 OK status. Check response counts, not just HTTP status.
• 7 models shipped with missing shared-KV tensors. Fixed by copying from base. Lossless.

What was learned

1. Two templates, not one: reasoning enabled for generative tasks, disabled for loglikelihood scoring.
2. Check for chat_template.jinja or chat_template in tokenizer_config.json before trusting any MC score.
3. The strict-flex gap on GSM8K is the cheapest reasoning-damage detector. Costs nothing beyond strict scoring.
4. Empty and truncated HarmBench counts are a free collateral-damage signal.
5. KL is sensitive to the logit collection pipeline. Re-collect base logits with current code.
6. Shared-KV architectures break export pipelines.

Disclaimer

These models have had safety alignment removed. They will comply with harmful requests. Use responsibly and in accordance with applicable laws. The authors do not condone or encourage the use of these models for harmful purposes.

While I have taken the time to verify all results thoroughly, I am open to any corrections, additional benchmarks, or further analysis. If you spot something that looks wrong and can be confirmed, I am happy to fix it.