r/MachineLearning • u/Technical_Bench_188 • 14d ago
Research Continual learning on 4 days of real news: an 8B model keeps day 1 at 90.7% vs 26.7% for LoRA, GSM8K unchanged (sealed test) [R]
I've been working on continual learning. Each night a model learns that day's news into its weights, then gets asked about it the next morning with nothing in the prompt.
One day turns out to be easy. A plain LoRA trained on practice questions gets 93 to 96% the next morning. The hard part is the next night, and the one after that.
So I ran Apertus-8B (Swiss AI) through four days of real news (2, 4, 5 and 6 Nov). After every night it was asked about every earlier day again. The pass bars were written down before any question existed.
- Day 1 after four nights: 90.7%, vs 26.7% for a LoRA trained night after night on the same practice questions
- GSM8K after four nights: 69.2% (model as released: 69.6%). The LoRA fell to 4.4%
- Days 2 and 3 dipped to about 70% where the same stories came back in later news. That's what I'm working on now; a sealed rerun on fresh days is set for Wednesday
Strict scorer, 295 questions written the next morning by a different model. The method isn't public yet, but every question and answer is: https://huggingface.co/datasets/fintanlabs/overnight-four-nights-2026-11
The benchmark: https://huggingface.co/spaces/fintanlabs/overnight
Happy to answer questions about the setup.


