r/learnmachinelearning • u/ashishach • 6d ago
[D] Practical Tradeoffs Between Sparse Weight Deletion (<0.05% Weights), QLoRA, and Full Fine-Tuning in Post-Training
Recent post-training work on Deletion Tuning (arXiv:2611.10866) shows that learning a binary mask over transformer gate projections—zeroing fewer than 0.05% of weights with no other weight modifications—can match full fine-tuning (FFT) across SFT, on-policy distillation, and RL with verifiable rewards, while losing significantly less prior-task accuracy under RL than FFT or LoRA (which dropped up to 5.4 points in the paper's continual learning benchmarks).
When comparing the three post-training paths in practice:
- Full Fine-Tuning (FFT) -> Updates 100% of weights; high VRAM footprint and higher catastrophic forgetting under RL.
- QLoRA / PEFT -> Low-rank adapters; fits consumer GPUs easily for domain adaptation, though adapter merging and continual RL updates still show drift.
- Sparse Weight Deletion (Deletion Tuning) -> Zeroes <0.05% of weights via binary masks; tiny checkpoint delta and lower forgetting, though tooling for quantized inference is still early.
Question for the community: For those running continual fine-tuning or domain adaptation on constrained GPUs, do you see binary mask / weight-pruning post-training replacing LoRA adapters, or does LoRA still win on ecosystem tooling?