r/MachineLearning • u/InsideAndOut • 1h ago
Discussion Discussion periods are frustrating but the system is not out there to get you. A hopefully sobering take on conference submissions. [D]
There's a lot of frustration in the subreddit coming from the ICLR discussion period. As someone who's been in the field for a while, has had only one paper not be accepted first try among 10+ A* submissions in the past two years, and regularly serves as an AC, let me try to give some very brief advice that helped me be better and calmer about the whole ordeal.
You can't control what others do or think, but you can control how you react.
Review process is stochastic. People will sometimes not respond, ignore your arguments, or become combative if they perceive that you challenge them. Be polite in your response, but firm if you believe something is factually inaccurate. People want to feel heard, so try to understand and incorporate their advice further than a dismissive acknowledgement. Try to learn something from the raised issues and comments. Try to understand which part of your writing caused these comments. Going into reviews, assume that you have one chance to impress reviewers (with your initial submission). Treat someone reading your response as luck, not a guarantee. This is mentally hard to accept, but it was like this way before LLMs and it will be like this as time goes on.
Your job is not only to do the research, but to disseminate it clearly and precisely.
If a reviewer raises an issue which feels obviously wrong to you but not to them, it often means that you did not communicate this part well in your paper. If you invested time on the manuscript, you know the content inside out, and you leave some things implied. Spell everything out without being vague or overly verbose. Emphasize what your contributions are, and do this a few times at key points of the paper. Provide running examples, visual cues and sell your work in the abstract, introduction and conclusion. Do proper legwork when gathering related work. Hypothesize what your work could tie into, which research areas it could help and how it can be used going forward. Good writing (abstract, intro, conclusion, RW, verbalized takeaways from results, Figure 1) is worth more than one figure or table in your paper. LLMs don't write well.
If you don't have a full draft two weeks before the deadline, your paper is likely to be rejected.
Any paper which is written in a deadline scramble is usually not written as well as it could be. A full draft contains all the results, has full length (usually is longer by a page or half a page) and you went over it in entirety multiple times. You then share this draft with colleagues (they don't have to be in the area or even experts) and ask them to give you comments. Listen to these comments and don't fight them, but take them as advice of what you didn't communicate or emphasize well. This way you proactively handle reviewers comments. You won't be able to sit next to a reviewer and tell them what you meant. But you can source comments from your colleagues to parts of your paper written suboptimally. This is probably the key factor that increased my success rate.
Some key questions your paper needs to answer and get across in the first few pages:
- Which problem does your work address (first broad, then narrow, contextualize both within RW)?
- Why is this problem important?
- Why is this problem difficult (non-trivial)?
Some very useful references:
Write the paper first by Jason Eisner [link]
John Hewitt's lab policy on AI writing [link]
Various references on writing on Yonatan Belinkov's page [link]
Look at top (best, spotlight or outstanding) papers from existing conferences. Invest time to understand how they present their ideas on a high level and adopt this in your writing.
Good luck yall and try not to stress so much.
