I’m reviewing for ICLR myself, so I know reading the revised papers and carefully reading all the lengthy rebuttals feels like a terrible time-sink, but to everyone else who’s also reviewing: please remember that most authors have spent an enormous time and effort in their submissions.

I’ve noticed that many reviews have already been updated after the rebuttal period but it seems that most if not all miss key points that are addressed in the rebuttal or in the revised paper. There’s an option to compare revisions which highlights the changes — please use this feature, as some authors address points in the revision but don’t mention it explicitly in the rebuttal (this actually happened for all papers I’m reviewing). I submitted a paper myself and my reviews were shorter than last year, and I also have the feeling that my rebuttal wasn’t carefully read by the reviewers who updated their reviews.

I also know that many reviewers this year are reviewing for the first time, but please do make an effort to spend some time going over rebuttals and revisions. You’re now part of the ML academic community — try to make it better, we need it especially now that the many of the highest-rated papers have extremely short reviews with low confidence scores, including reviews as short as 20 words.

TLDR: we all know there are not enough reviewers and way too many submissions. while reviewing for free can be frustrating, the community depends on us and the job includes being thoughtful and reading rebuttals/revisions carefully