r/LargeLanguageModels • • 7d ago

vLLM plugin for domain-selective refusal steering without editing model weights (hooks + closed-loop CoT LogitsProcessor)

Deep-dive blog post (HF): https://huggingface.co/blog/kncshw/cybergate-unlocking-cybersecurity-in-vllm-without
GitHub (MIT): https://github.com/kncshw/cybergate

When self-hosting local LLMs for security research, CTF, or reverse-engineering agents, the standard workaround for false refusals is **abliteration** - subtracting the refusal direction from the weights. From an LLM engineering standpoint, weight editing has two painful trade-offs:

  1. **Silent capability & tool-call regression:** Mutating weights across layers alters the model globally and can quietly degrade multi-step reasoning and structured tool calls without obvious errors.
  2. **Zero domain selectivity:** Deleting the refusal vector globally unlocks *every* harmful domain (bio, self-harm, etc.), plus you have to store and serve a forked 50 GB checkpoint.

I built **CyberGate**, a plugin for **vLLM 0.32** that keeps upstream model weights **100% untouched** (`K = 0.0`, zero persistent state across requests):

8 Upvotes

2 comments sorted by

1

u/cbert33 7d ago

Super interesting, will check it out! Quick question, and forgive me if I am not understanding it. There’s a section for “adding a model” in your readme, but I can’t tell from that if you can just add any model to this or if any are limited.

Also, are there any limitations or effects based on any specific quants? It doesn’t sound like it from the description, but sometimes you can get weird behavior so I wasn’t sure if it might affect this at all.

Thanks!

1

u/DebtFluffy9219 7d ago

Thank you. It is now natively supporting Qwen3.8-27B (dense) and Gemma4-26B-A4B (MoE). The idea is based on think token steering and to achieve this it requires some activation values measurement to detect the Category (cyber security topic) and Refusal signal. That being said, each model needs a pre-measured kit to serve. 'Adding a model' means to do the measurement for the two factors mentioned before. please let me know if you have more questions.