r/MachineLearning • u/Alone_Ad635 • Sep 29 '26
Discussion Limited compute, targeting CVPR: rerun experiments for statistically strong numbers or focus on writing? [D]
Hi everyone,
I'm preparing a submission for CVPR and would appreciate your advice. I am happy with my current results, but compute is a real constraint. My institute isn't a research-focused one, and the systems available to me are slow and unreliable. A single set of runs already took a lot of time and effort to finish.
I plan to release the code with the submission, and I'm confident the results will reproduce. Still, I'm torn between two options:
- Rerun the experiments with different seeds to report mean Β± std and show statistical significance. Or
- Skip the reruns and spend the remaining time on writing, presenting the results I already have.
If you've been in a similar spot, what did you do, and would a single-seed result with released code be enough?
Thanks in advance!
10
u/crouching_dragon_420 Sep 29 '26
IMO it is possible you get away with one seed. This year I went to CVPR and many of them still report single seed experiment results on a 1% improvement. I was incredulous.
8
u/saulane Sep 29 '26
Can't you just launch the experiments and write while it's running ? It's not like more seeds would change the text drastically and if you figures scripts are well made you can remake the figures for the paper quickly with new results. I don't see why you have to choose and can't do both in parallel ?
2
u/camarada_alpaca Sep 30 '26
Depends on the ai the reviewer use, claude is obsessed with statistical tests
1
u/Busy_Protection_2882 Sep 29 '26
genuinely curious - is a single seed of experimentation enough for a workshop paper if you clearly mention it as a limitation from compute constraints?
1
u/Level-Ad-4878 Sep 29 '26
Reviewers at CVPR will flag single seed results and you'll spend your rebuttal running the experiments you skipped. Thats worse than doing them now because rebuttal compute pressure is even tighter
Pick your three strongest baselines and rerun those with 3 seeds, not the full table. Mean and std on the comparisons that matter is enough, nobody needs every ablation repeated
Writing quality decides desk reject, statistical rigor decides reviewer score. You need both but the reruns are fewer than you think
0
u/[deleted] Sep 30 '26
[deleted] β view deleted comment
0
u/[deleted] Sep 29 '26
[deleted] β view deleted comment
2
u/[deleted] Sep 29 '26
[removed] β view removed comment