What can we do to improve peer review in NLP?
A Rogers, I Augenstein - arXiv preprint arXiv:2010.03863, 2020 - arxiv.org
arXiv preprint arXiv:2010.03863, 2020•arxiv.org
Peer review is our best tool for judging the quality of conference submissions, but it is
becoming increasingly spurious. We argue that a part of the problem is that the reviewers
and area chairs face a poorly defined task forcing apples-to-oranges comparisons. There
are several potential ways forward, but the key difficulty is creating the incentives and
mechanisms for their consistent implementation in the NLP community.
becoming increasingly spurious. We argue that a part of the problem is that the reviewers
and area chairs face a poorly defined task forcing apples-to-oranges comparisons. There
are several potential ways forward, but the key difficulty is creating the incentives and
mechanisms for their consistent implementation in the NLP community.
Peer review is our best tool for judging the quality of conference submissions, but it is becoming increasingly spurious. We argue that a part of the problem is that the reviewers and area chairs face a poorly defined task forcing apples-to-oranges comparisons. There are several potential ways forward, but the key difficulty is creating the incentives and mechanisms for their consistent implementation in the NLP community.
arxiv.org