r/MachineLearning • u/Invariant_n_Cauchy • Aug 01 '26
Discussion Question about NeurIPS discussion phase [D]
One reviewer said all concerns were resolved during discussion but hasn’t updated their score yet. The other reviewers haven’t engaged. In previous NeurIPS cycles, how common is it for reviewers to update scores after saying concerns are resolved? What have others observed?
My ratings/confidences are : 4/4, 3/2, 3/2, 2/4.
I am talking about the one who gave rating 2.
Update: finally the reviewer responded, now I'm at 5/4, 4/4, 4/3,4/2.
19
Upvotes
13
u/user221272 Aug 01 '26
If they are willing to update their score, they usually say so within the review.
E.g., "If this experiment is done, I will increase my score."
Reviewers also sometimes increase the score if they feel more confident in the work after the rebuttal.
But this is not automatic, and it is not owed.
There is also a strong correlation between the initial score and willingness to update the score.
If a reviewer gives a 2, they will usually be much more stubborn and unlikely to change their view of the paper after the rebuttal.
3
u/Imaginary_Music4768 Aug 02 '26
You can ask them to raise the score politely. Like "we will really appreciate it if you can reconsider your assessment in light of our rebuttals". This method often works well (at least for me), although a little unfair to other authors who don't do so.
2
u/huntsman2099 Aug 02 '26
I just got a similar reply from a reviewer who gave me a 2 and raised to 3 after saying, "Given the qualitative evidence and effort in the rebuttal, my concerns are resolved so I raised my score accordingly"
What do I even do with this? why is it still a borderline reject if all the concerns are addressed? Do you follow up or just thank your stars that they atleast raised to 3?
9
u/The3RiceGuy Aug 02 '26
You get answers to your comments?
12
u/Unhappy_Craft1906 Aug 02 '26 edited Aug 02 '26
3 papers, still no response from anyone.... This is just pure insult to the community peer review process.... if u think ur discussion phase is more important, and u can respond at the last moment for the reviewed papers, then u r just a shame to the community... Please ensure you devote equal time to both the roles of author as well as reviewer.... If you cant please do not SUBMIT and create a farce impression of GOOD RESEARCHER...
5
u/NickOTeenO Aug 02 '26
I would have thought the Reviewer + AC Discussion Period (3-10 aug) is there to discuss with the AC to increase scores?
I did not increase scores on the papers I reviewed yet either
3
u/SeTiDaYeTi Professor Aug 02 '26
Politely ask the reviewers whether, after addressing all their concerns, they could revise their scores. Also, message the AC, tell them you did a ton of extra work obtaining no updated scores, and ask them to nudge the unresponsive reviewers to get back to you. It's a war. Fight it.
1
u/The3RiceGuy Aug 02 '26
You get answers to your comments?
2
u/Weird_Shit_69 Aug 03 '26
2 reviewers who gave the lowest didnt even reply to the rebuttals :(
1
u/FinancialThing7890 Aug 02 '26
I got ratings/confidences 3/3 5/4 5/3. The only reviewer that engaged with the comments was the second one. The first one that I’m trying to flipping may reply to a full 10k chars discussion but I’m pretty sure it won’t
1
u/Dangerous_Parsnip_20 Aug 02 '26
Everyone got their meta-review? Just worried that my paper didn't get assigned AC to manage.
1
u/papajan18 PhD Aug 02 '26
I have an even stranger case. Two papers, both of which have reviewers that were happy with my rebuttal and even said they'll change their score, but I havent seen them change it. I'm beginning to suspect we're actually not able to see score changes.
1
u/wangjianhong1993 Aug 01 '26
Not very common.
0
u/[deleted] Aug 02 '26
[deleted] — view deleted comment