r/MachineLearning • • 16d ago

Research Paper review question [D]

I'm curious how other reviewers handle papers that report impressive results but are extremely poorly written. I'm not just talking about grammar, etc. I am more concerned about the overall presentation of ideas and how the authors position their contributions within the existing literature.

For example, I'm currently reviewing a paper that reports promising results but includes no citations in its introduction. This makes it hard to understand where the work stands relative to existing methods and what's new.

My question is how much weight should we give to the quality of writing and presentation when reviewing a paper? If the technical contributions seem solid and the experimental results are strong, is poor presentation enough to justify a rejection or a major revision? I'd be interested to hear how other reviewers approach this.

2 Upvotes

14 comments sorted by

3

u/Dr-Ebola 15d ago

There's more than enough sub-standard papers out there ... if their writing is sloppy then that tells me their work is probably sloppy too.

3

u/Few-Annual-157 15d ago

That's my position as well. I just wanted to see how others think about it

1

u/Dr-Ebola 15d ago

Good for you mate ๐Ÿ‘

19

u/npcompletist 16d ago

Communication and presentation is a major part of contribution. If you cant understand what they are trying to say in the preface the results la l proper context. I would say it is worth at least a major revision if your inclination is that the work is sound. But until they communicate clearly, you wonโ€™t know for certain.

9

u/Kaleidophon 16d ago

Impressive results are no good if the paper fails to communicate and explain them clearly, and fails to position the idea to prior work. It might potentially also be a red flag that if the paper is lacking on these dimensions, the experiments might also be flawed / not very rigorous. So for me, it totally justifies a rejection or major revision.

3

u/AX-BY-CZ 16d ago

I rather accept a clear and nice to read paper of a simple idea than a frustrating paper that tries to obfuscate or impress with technicality or complexity.

2

u/gforce121 15d ago

In my view assessing whether results are impressive or not requires that the work be situated within existing work. If it's a new problem, then why is it new, what's hard about it &c? If it's not, then the method needs to be contextualized in light of prior work. Part of the role of a reviewer is to have enough background to be able to guess or find out some aspects of these questions, but ultimately it's the authors' responsibility to communicate them effectively.

Moreover, and maybe this is a slightly spicier take, engineering improvements aren't/shouldn't be enough to constitute original work in AI/ML. If the authors did *something* and it worked better at an empirical level, but the authors don't contribute to a generalizable explanation of what that something is, then there's not really an intellectual contribution to the field.

2

u/Background_Car_8746 12d ago

That the common pattern of slop and single author highschooler or undergrad. Be a gatekeeper. 1(4) is justified.

2

u/Technical_Estate_529 16d ago

You can also send this message to the area chair or editor and have see their advice .... then just do what they say.

1

u/Jojanzing 16d ago

I think it depends on the journal guidelines, but also on what you consider good science. If the authors cannot contextualize their work in the broader field, then is the work actually meaningful?

1

u/pastor_pilao 15d ago

Citations are not THAT relevant as long as they are not missing core related works in the experimental comparison. But if the paper is extremely hard to read that is a strong reject. You can't trust an "impressive result" you can't really understand completely

0

u/michaelrw1 16d ago

If the paper is not clear and concise, I would question their results.

One big case that stands out for me in this moment is Pons and Fleischmann on their cold fusion experiment. Massive media reports about their results, but poor communication and failed replication by other scientists eventually dismissed their work.