Table of Contents
What is Abliterix?
Abliterix
is a Heretic-derived multi-objective Optuna optimiser with native hybrid Mamba/attention support, projected abliteration, and expert-granular steering. It applies rank-3 LoRA-merged directional updates to attn.o_proj and mlp.down_proj.
Performance Across Models
I’ve tested Abliterix on Qwen3.6-27B, GLM-4.7-Flash, and Gemma4-E2B.
| Model | HarmBench ASR | Full CoT ASR | MMLU | GSM8K flex | KL Divergence | LAMBADA PPL |
|---|---|---|---|---|---|---|
| Qwen3.6-27B | 94.5% | 100% | 81.3% (-2.0pp) | — | 0.0222 | 9.12 (2.9x base) |
| GLM-4.7-Flash | 100% | 100% | 77.0% (-0.9pp) | — | 0.0116 | n/a |
| Gemma4-E2B (wangzhang) | 98.8% | — | 26.69% (-2.31pp) | 81.58% (-1.89pp) | 0.6984 | 1,072,918 (7.35x base) |
Key Characteristics
Quantisation sensitivity. Under BNB4 quantisation, Abliterix shows the worst capability preservation of any technique. Lambada perplexity increases 2.9x from 3.18 to 9.12 on Qwen3.6-27B. On Gemma4-E2B, the LAMBADA perplexity blowup is even worse at 7.35x base, reaching 1,072,918. The q_proj and v_proj modifications unique to the Gemma4 variant catastrophically damage language modelling. The model’s creator explains this as a quantisation interaction. The rank-3 LoRA signal lives in a low-dimensional subspace, and BNB4’s per-block NF4 quantisation is not subspace-aware.
Surgical component targeting. Despite the broad-sounding “LoRA search” description, Abliterix only modifies attn.o_proj and mlp.down_proj on Qwen3.6-27B and GLM-4.7-Flash. Just 2 weight types. It does this across all layers with a mid-to-late-stack sustained edit profile.
Gemma4-E2B attention targeting. The Gemma4 variant uniquely modifies q_proj and v_proj in addition to the standard attn.o_proj and mlp.down_proj. This is the only variant in the entire Gemma4 comparison that targets query and value projections. The attention input targeting correlates with catastrophic LAMBADA degradation at 7.35x base.
Mid-to-late-stack edit profile. The abliteration weight peaks at layer ~41 with a 35-layer decay radius, creating a sustained modification floor across the middle and late layers.
Weight Modification Profile
- 101 tensors modified (11.9% of total)
- 5.2% relative edit magnitude
- Components:
attn.o_proj+mlp.down_projacross all 64 layers - Profile: peak at layer ~41, 35-layer decay radius
Read the Full Analyses
- Qwen3.6-27B: Heretic vs Huihui vs AEON vs Abliterix vs HauhauCS
- GLM-4.7-Flash: Heretic vs Huihui vs HauhauCS vs Abliterix
- Gemma4-E2B: 13 Abliteration Techniques Compared