Top Reviewer Incentives by Michael Muthukrishna


The concept of peer review has come a long way since inception and is now widely used by popular online websites and forums, such as Reddit, StackOverflow, and Slashdot, and even websites like Amazon. The challenge these websites face is to incentivize reviews without any tangible rewards. One great innovation is the idea of reviewer “points” in the form of  badges or “karma”. These incentives seem to be sufficient for these forums to have a vibrant and active peer review process, ensuring quality contributions are more visibly “published”. These incentives rely on nothing more than reputation, which we know to be a powerful motivator of human behavior. At present, reputations in the review process are missing, and the rewards for being a reviewer are small and not visible. A line on a CV is an encouragement for young researchers, but fails to represent the level of contribution. I suggest a public reviewer point system, rewarding points for number of reviews, speed of reviewing papers, and other aspects of the review process that need to be incentivized. Long term and per-publication badges can also be rewarded for “top reviewer”, “fast reviewers”, etc., and can be journal specific. Issues of anonymity can be addressed by using pseudonyms for the display of reviewer points, using abstract badges instead of actual points, only providing a leaderboard, emailing reviewer points privately, or any combination of these.  I believe such a system would vastly improve the review process by incentivizing and better rewarding reviewers. In particular, young reviewers, already familiar with such systems, will be able to report not just that they are reviewers, but how much of a contribution their reviews have made.

Who would benefit

This idea would benefit editors, since it provides an added incentive to produce quality reviews more promptly and will encourage more researchers to more actively engage in the peer review process.  The idea will benefit reviews, since they now have a way of displaying their contributions as reviewers in the academic process, including on their CVs.  The idea will benefit authors, since they will be provided with better reviews, more promptly.