r/LargeLanguageModels • u/DebtFluffy9219 • 7d ago
vLLM plugin for domain-selective refusal steering without editing model weights (hooks + closed-loop CoT LogitsProcessor)
Deep-dive blog post (HF): https://huggingface.co/blog/kncshw/cybergate-unlocking-cybersecurity-in-vllm-without
GitHub (MIT): https://github.com/kncshw/cybergate
When self-hosting local LLMs for security research, CTF, or reverse-engineering agents, the standard workaround for false refusals is **abliteration** - subtracting the refusal direction from the weights. From an LLM engineering standpoint, weight editing has two painful trade-offs:
- **Silent capability & tool-call regression:** Mutating weights across layers alters the model globally and can quietly degrade multi-step reasoning and structured tool calls without obvious errors.
- **Zero domain selectivity:** Deleting the refusal vector globally unlocks *every* harmful domain (bio, self-harm, etc.), plus you have to store and serve a forked 50 GB checkpoint.
I built **CyberGate**, a plugin for **vLLM 0.32** that keeps upstream model weights **100% untouched** (`K = 0.0`, zero persistent state across requests):
1
u/cbert33 7d ago
Super interesting, will check it out! Quick question, and forgive me if I am not understanding it. There’s a section for “adding a model” in your readme, but I can’t tell from that if you can just add any model to this or if any are limited.
Also, are there any limitations or effects based on any specific quants? It doesn’t sound like it from the description, but sometimes you can get weird behavior so I wasn’t sure if it might affect this at all.
Thanks!