Elsevier Reviewer Badges and Rewards scheme by Simon Gosling

Idea

Reviews can take a long time to complete well. Too often, there is little incentive to academics with full schedules to complete reviews. Offers of 30-day free access to a journal are usually unhelpful because institutions provide access to staff and students anyway through license agreements. Moreover, the efforts that go into writing a review, do not usually present anything tangible that can be included on a CV, or put in a portfolio, or even on an office wall. This is especially frustrating for early career researchers, for whom the lengthy process of peer reviewing articles can go un-noticed. For instance, while one can state on their CV that they “have been a peer-reviewer  for Journal of…”, this does not reflect how many reviews they have completed. Essentially, the total amount of effort that has been employed is under-represented and does not help with career progression.     My suggestion for Elsevier is to introduce an “Elsevier Reviewer Badges and Rewards” scheme. This would provide a standardised tangible recognition system for the work invested by academics in the peer review process. This would be facilitated by the awarding of “Elsevier Badges” and subsequent “Elsevier Rewards”. Anytime someone conducts a review for an Elsevier journal, they will be awarded an “Elsevier Contributor Point”. These points can be accumulated and after accumulating a certain amount of points, a reviewer will receive an “Elsevier Badge”.     The first badge would be called an “Occasional Reviewer Badge”. As a reviewer accumulates more and more Elsevier Contributor Points, so they are awarded new badges. In the first instance, badges could be awarded for; up to 5 reviews (“Occasional Reviewer Badge”), up to 10 reviews (“Regular Reviewer Badge”), and 20 or more reviews (“Senior Reviewer Badge”). Other badges could be awarded at the journal Editor’s discretion. For instance, a badge could be awarded annually for the “Most Comprehensive Review”, another for the “Most Helpful Review”, and one for the “Most Contributions in a Year”, for instance. People at Elsevier may have ideas for other badges that could be introduced.     The badges would appear against a reviewer’s online profile. Also, a paper certificate could be posted to each reviewer every time they are awarded a new badge. This would be a tangible recognition from Elsevier of someone’s effort. These certificates would be especially helpful to early career researchers and they could be included on a CV and/or in a portfolio, and even put on your office wall, for instance. I am not aware of any scheme like this at present in academic publishing. Note, however, that Springer have started offering “posters” of papers that people publish, which they can frame or put on their office wall. There is clearly interest in such tangible products, and I think Elsevier could do well to join this trend.     A “one off” “Elsevier Reward” could be awarded to each reviewer for every badge they receive. For example, the following one-off discounts could be offered to people when they are awarded the following badges; 15% off any Elsevier product (“Occasional Reviewer Badge”), 20% off any Elsevier product (“Regular Reviewer Badge”), and 25% off any Elsevier product (“Senior Reviewer Badge”). Similar discounts could be awarded for the other badges I mentioned (e.g. the “Most Contributions in a Year” badge). The exact scale of discount will need to be decided by Elsevier. Importantly, the discount should apply to books and other products from Elsevier, since discounts against journals are not usually needed, due to institutional licenses.

Who will benefit?

The badges will be useful for Editors because they will be able to select “more experienced” reviewers for certain “high profile” papers, if necessary.     The “certificates” would be especially helpful to early career researchers and they could be included on a CV and/or in a portfolio, and even put on your office wall, for instance. I am not aware of any scheme like this at present in academic publishing. Note, however, that Springer have started offering “posters” of papers that people publish, which they can frame or put on their office wall. There is clearly interest in such tangible products, and I think Elsevier could do well to join this trend.    A “one off” “Elsevier Reward” could be awarded to each reviewer for every badge they receive. For example, the following one-off discounts could be offered to people when they are awarded the following badges; 15% off any Elsevier product (“Occasional Reviewer Badge”), 20% off any Elsevier product (“Regular Reviewer Badge”), and 25% off any Elsevier product (“Senior Reviewer Badge”). Similar discounts could be awarded for the other badges I mentioned (e.g. the “Most Contributions in a Year” badge). The exact scale of discount will need to be decided by Elsevier. Importantly, the discount should apply to books and other products from Elsevier, since discounts against journals are not usually needed, due to institutional licenses.

 

58 thoughts on “Elsevier Reviewer Badges and Rewards scheme by Simon Gosling

  1. The idea of badges and rewards seems to be very interesting and surely helpful to early career researchers. But i do feel that there would be significant chances of bias is this methodology. Especially when one says that editors can have the choice to select “experienced reviewers’ for high end articles and higher discount for regular reviewers.
    Bias in the sense that reviewers who are close to editors (friends, relatives, collegues,etc) would be given higher preference in intial phases of review and once one of the friend reaches the top ‘badge’ category the editor can focus on another one to make him/her reach the top end.
    In addition to the number of reviews performed by a reviewer the quality of review performed, the amount of information given for improvement of the article and similar contributions should be stressed upon. If number of reviews would be kept as a criteria to score higher badge, one will try to review many articles in less amount of time and ultimately the quality of revies is sure to get affected.

    Corrections/modification in the idea:
    1. Selection of reviewers by the editors should be on random basis from the pool of reviewers listed for that particular subject/topic on which the article is based, that would reduce reviewer selection/preference bias to large extent.
    2. Instead of giving 1 point for each review, the editorial team can score the reviewers quality of review in 5 points (1 point each for different criteria, eg: 1 point if work done in time, 1 point if decision justified correctly, 1 point for the contribution to improvement in article, etc). The better the article is reviewed the faster the reviewer will climb the ladder of success. Badges can then be decided based on ratio of number of articles reviewed and total score. One with more score will less reviews can be given “platinum reviewer badge’ (minimum ten reviews necessary to start receiving badges in all category), ratio of slightly more than 1:1 ratio can be given ‘Gold reviewer Badge” and those with 1:1 ratio “Silver reviewer Badge” (1:1 means reviewer is scoring 1 point every 1 review performed, that means he/she is regularly performing review but the quality of review needs to be improved to achieve a higher badge).
    3. Once the highest badge is received the reviewers interest in performing quality review may lose. To keep them active, the validity of badges can be kept for around1 to 3 yrs. AFter which depending upon last years performance the new badge can be awarded (which may be higher or lower or not at all if no reviews or less than 10 reviews performed in last year.

    • I support the idea and comments by Dr. Agrawal. The qualification for each reviewer and limitation of manuscripts for each reviewer per year are very important.

    • Great idea, but do incorporate the modifications proposed by Dr. Amit Arvind Agrawal.

  2. Good idea but I think the previous commenter has nailed some issues. I think the main hurdle would be how to ‘quantify’ these reviews into badges. The quality of the review should actually be valued more than the quantity.

    Also, the fact that there will be a ceiling (a highest score if you will) could be an issue. After that point in time the incentive to keep reviewing will be low. Addressing this issue by implementing a loss of the badge after time x will solve this, but once this happens this reviewer might 1. not return (losing out is not nice) 2. stop caring at all. As mentioned, for early career scientists this will for sure work.

    In short, as with all ‘gamification’ attempts the problem will be keeping momentum for established people / users.

  3. Great idea but I think the quality of the reviews should count more than the quantity. It takes little effort to write 10 bad review, but a single good, constructive review can be demanding. Don’t give credit to quantity give it to quality and added value.

    • I agree but speaking to quality vs. quantity, I would think that chronically bad reviewers would not be in the system very long anyway, right? Maybe that’s naive, but I always thought it was a self-correcting process on that end. My impression is that good reviewers naturally keep getting tapped, and the question is how can we better reward this set up, from a quantitative perspective, instead of the perverse incentive of burdening good reviewers with more (seemingly thankless) work while letting bad reviewers off the hook.

  4. While academic reviewers might not find the 30d on-line journal access offer attractive, for the legitimate reasons the proponent advances, there are reviewers not working in academia. I for one, would find an extended period of on-line access to Veterinary Parasitology (in lieu of the paper copies I currently receive) alone, a very attractive reward for my review efforts.

    The points made by the two commentators about reward for quality reviews are well made. Perhaps also a bit of reviewer guidance/feedback from the editor? I can sometimes not help myself from lapsing into proof-reading, but is this helpful?

  5. Having the quality of the review assessed could be helpful to better train reviewers if they are doing “the right thing”. Rewarding the efforts with access to online publications (perhaps even out of their field to avoid cutting into the profit margins) would be good.

  6. the greatest reward i believe is the possibility to get promoted to an associate editor. in order the process to be fair, i believe it is extremely useful that precise review performa including number of reviews, quality of reviews (as indicated by idea), etc. be available in public for a reviewer, so that he/she may include it in his/her application for the post of editor. a badge may be too restrictive.

  7. This discussion, particularly the emphasis in the comments on quality of reviews rather than just quantity, reminds me somewhat of the APS Outstanding Referees program (http://publish.aps.org/OutstandingReferees). The difference is that while the APS program is oriented towards recognizing long-term high-quality contributions, I think that what is proposed here would equally recognize meritorious contributions by younger faculty. I think it’s a great idea.

    One thing that I would like to see in such a program is integration of data across all of the Elsevier journals (or across journals from many publishers, if that is possible). Some of us referee for a number of different journals. If this is done on a journal-by-journal basis, some people who contribute quite a bit could be overlooked.

  8. I am proposer of this scheme and I would like to respond to the very helpful comments above.

    Firstly, thank you to all those above for taking the time to comment upon my idea. I am pleased to see acknowledgement of the value that implementation of the scheme could have for early career researchers, who comprise a large proportion of the peer-reviewer demographic in academic publishing.

    Moreover, this set of comments and suggestions is testament to the kind of discussion I would like to have with the team at Elsevier and journal Editors, with respect to what type of badges could be ‘included’ and/or how badges should be awarded, if this idea is taken forward. I expand upon my idea here, with reference to the above comments.

    Specifically, I am pleased to see that the issue of reviewer quality versus quantity has been raised because there was not room in the original application to describe this explicitly. I agree that review quality should be weighted more than quantity. Also, Marc DeHart’s point that this could actually help reviewers know that that they are ‘doing the right thing’ is a useful potential additional benefit of the scheme and further compliments my intention that the scheme should be useful to both reviewers and Editors.

    As my current role as a Guest Editor for a non-Elsevier published journal, I know that Editors and Field Editors do have the option for ‘grading’ or ‘scoring’ reviewers on a 0-100 scale for some journals. Such a scale could easily be implemented here to inform the badge awarding process, so that reviewer quality is considered. For example, reviews scoring less than, say 60/100, would not be eligible for a badge – therefore this would discourage people from writing very brief and unhelpful reviews. Badges could then be awarded at certain points thresholds (e.g. the first badge could be awarded at 200 points etc.), thus accounting for both review quality and quantity. I would be happy to put together a formula that weights such scores based upon a set of Likert scale questions or similar that the Editor completes, if the proposal is accepted – this would be relatively trivial to compute and could be based upon a similar weighting scheme to those employed by Universities for weighting modules/degrees, for instance.

    When the Editor submits a reviewer score they would also be required to include a line or two that justifies their score – this would address Johann Schröder’s comment that some reviewer feedback from the Editor would be useful, and it would not take much time at all for the Editor to do this. Editors also have access to statistics such as; number of reviews completed on time, for instance, and these could be implemented in the weighted badge scheme too, relatively easily (Dr Agrawal alluded to this above). I would be cautious with laying additional requirements upon the Editor than those stated here, however, because Editors are already required to work under tight timescales and this scheme is meant to be useful to them, not a hindrance.

    With respect, I do not agree with Dr Agrawal that there are ‘significant chances of bias with this methodology’ due to the possibility of Editors only selecting friends, relatives and colleagues to perform reviews. All journal Editors and Field Editors I know have always selected reviewers based upon their academic fit to the paper, not because they are friends or family. This objective approach has always been crucial to the integrity of the academic peer review model and I do not believe that there is a chance of bias occurring with journals published by a name so prominent as Elsevier.

    Thank you to Dr Agrawal and Koen Hufkens for noting the potential for momentum to ‘drop off’ when reviewers hit the ‘ceiling of top badge’. However, I do not believe that this poses a problem for the following reasons: 1) The ‘Senior Contributor Badge’ would require in excess of 20 reviews – it would take several years for a reviewer to reach this number of reviews for journals published by Elsevier; and 2) similar to the loyalty schemes offered by some airline companies (e.g. air-miles), reviewers could indeed be required to complete a number of good quality reviews each year to maintain their current ‘status’, so as to provide an incentive for them to stick with the scheme.

  9. I like the idea of having something for people who do reviewing as it can be a thankless task yet a very important one. I agree with the comment that once a certain level of badges has been obtained the incentive could run low… but this is not how I read the idea. There were to be other badges available and introduced by the editor etc… the badges could also have a 12 month/ 3 year/ 5 year, and so on status so that there is a renewed interest, I read it that this is how it would probably continue. So it would be obvious who was a ‘newbie’ and was working up their first 12 months and what their short term contributions were like but also that more senior academics could show their pro-active participation through a series of ‘lifetime’ achievements. All in all nice idea, I think it would help new researchers such as myself.

  10. Rewards are the right direction and I think it is best if publishers as Elsevier reduce the price of subscriptions for every review they receive by a certain percentage for the reviewer or the institution the reviewer works for. This will help libraries to keep and maybe even extend the amount of Elsevier journals their staff can access. One can also combined this with additional bonus points for fast and thorough review as judged e.g. by the editor. At least I would have some extra motivation for performing reviews and a good argument to review at all using the resources of my institution. At present review work is usually performed for free while the publishers significantly rise their prices and especially their profit. Why should we pay twice (free work + institution resources for the review and high subscription prices) and wouldn’t this be only fair to give something back apart from fantasy badges etc. (I would’t care about them at all and it would not impress me at all if someone has a lot of them)?!

  11. great concept. just fine tune this. make a table and call it a day. too lengthy

  12. There are a few different ideas here.
    – Giving credits to the reviewers: I prefer the option of an impact impact for reviewers (see different finalist). Some kind of metrics based on the quality and number of reviews would be better than a badge to print in my office.
    – Compensation: some journals give reviewers a credit to get books, cds, etc based on the number of reviews submitted on time during the last 12 months. That’s better than a discount on a product that would probably not buy.

  13. I partially agree with the basic idea, but strongly support some of the replies to the comments. I am not sure what is the use of badges, but it is obvious that there is a lack of recognition for the reviewer’s contributions. I am not in academia anymore and I still do at least two or three reviews a year for two separate journals that do not know of each other. So one of the suggestions I have to improve the proposal is to give reviewers identity numbers, similar to frequent flyer numbers, a number that you use whenever you perform a review for Elsevier. After that a point system can be applied. The 30 days access to Scopus is too short. I am now in the private sector a don’t have library access to journals. I review about 5 or 6 papers a year, and have done that continuosly for the last 20 years or more with no recognition. But I do this because I believe that this is essential for the system to work. So yes, any system that recognizes the continuous effort of the reviewers should be commended.
    Whether to allocate 1 or 5 points, that’s a different question. The ‘quality of the review’ can also be assessed, but it would put an enormous stress on editorial offices who would need to set criteria for reviewer assessment and this can become a nightmare.
    Yes, I agree with a point system, I don’t see much point in badges, but certificates of recognition that one can hang on the wall are an excelelnt idea.

  14. With the boycott Elsevier movement growing and with articles written about this in the New York Times, etc. and with more names being added to the cost of knowledge dot com website, I have to wonder whether “Elsevier Badges” would be viewed positively or negatively. Just a thought.

    I am serious about this though. Even if you don’t agree with the movement (and many of you may not) do you really want an “Elsevier Badge” listed on your vitae given how strongly some researchers feel about boycotting these journals?

    (I wonder how long this post will remain visible….)

  15. If this idea is selected, it may be smart to involve collaborators with expertise in game design / gamification. Folks like Jane McGonigal,, Scott Rigby, and others have written extensively about many of these concepts. In addition to possible consulting pay, you might also offer them an invited article / special issue describing the process in one of Elsevier’s journals. A publicly visible profile page with “reviewer stats” of this kind over various periods of time may help younger investigators receive more informational feedback and recognition.

  16. Among all finalists, this is the most useful and easy to install tool for analyzing and recognizing reviewers !! I support this idea !!

  17. vERY GOOD! The reviewers need more convenent way to give proffesional suggests.

  18. The immediate notion of “badges” struck me as childish; bt the ideas suggested by the original submitter are sensible. It’s too complicated and some respondents sugested open to abuse (as is anything that gets complicated with too many ‘rules’).

    I like the idea of a simpler version: maybe Bronze Silver and Gold “stars” (ie, decal-like star-shaped sheriff badges) that reviewers have a PW for and can insert into their webpage. Any badge should be for one year only (Gold Reviewer 2011); the decal should be over-signed by the Publisher of the specific Journal.

  19. Recognition is essential in any kind of voluntary work, and when you talk about a specialized effort like peer-reviewing, recognition should certainly become public. Part of the peer-reviewing job is a manual, hard-work job: it takes time to read and to verify if all journal’s rules (including reporting guidelines such as Strobe, Consort and so on) were followed. The other part requires a deep, technical and specialized point of view that only experienced professionals and academics do have: this is based on their accumulated knowledge. A “quantity-based” system of scoring for reviewers would do well for the first part, which can be done by anyone who has been involved in any scientific research in life and who can analyze if certain rules were followed: a young scientist or consultant can do this. However, this system would not fit into a well-experienced, specialized scientist who reviews one paper a year or so, but who does that with skills that no one else has. These two parts of the peer-reviewing job should be acknowledged in a scoring recognition system.

  20. Fab idea, Simon!! Linked with a ‘quality control’ mechanism, like some of the other proposed ideas have outlined (e.g., the ElseReview idea), this would be a really good way to ensure there are mutual benefits to authors, reviewers and publishers.

  21. The concept is great. However, to evaluate the review itself will bring more work to editors. It might be good to have a mechanism to evaluate instead of just depending on editors.

  22. It’s really an interesting idea to link reviewers efforts with some sort of incentive. The real challenge is how to strike a balance between quantity and quality of reviews contributed by a reviewer. Certificates of some sort of rewards/awards along with possibility of promoting active reviewers to editorial board membership could work better than presently followed mechanical “Thank You!” practice.

  23. I agree that a system to track (and award) reviewers performance must be implemented. I strongly agree that the quality and God NOT the amount of reviews must be the criterion!! I think it is time to discourage the reviewers from rejecting innovative papers based on a quick going-through the abstract… only…
    Responsability for the written review is also an issue. Why not disclose the reviewers names to the authors?
    I am not sure about the badges, but selection as an interim associate editor, as well as offering a credit or discounts on the purchase of Elsevier products, both sound beneficial.

  24. I also strongly agree with the idea of reward. However, the optimiztion of the mechanism is a tough process.
    I have the following suggestions.
    Those who opt for reviewing should be first given a standardized manuscript for review which should be evaluated before INDUCTION as
    reviewer. It should aim at the following:
    1. The quality
    2. Time taken
    etc.

    If due awards are given, the reviewing could occupy a center stage which could pave way for better quality articles and an effective tool for the academic progression of the reviewers also.

    various levels could be achieved depending upon the number of reviews done, time taken and quality of the review done. All these should be monitored. ( A mechanism needs to be deviced)

    After attaining certain levels, should make the reviewers eligible for more senior roles and ultimately help in getting due recognition for
    their time and efforts.

  25. agree that a system to track (and award) reviewers performance must be implemented. I strongly agree that the quality and God NOT the amount of reviews must be the criterion!! I think it is time to discourage the reviewers from rejecting innovative papers based on a quick going-through the abstract… only…
    Responsability for the written review is also an issue. Why not disclose the reviewers names to the authors?
    I am not sure about the badges, but selection as an interim associate editor, as well as offering a credit or discounts on the purchase of Elsevier products, both sound beneficial

  26. I like to core of the idea, which is to provide an incentivization to reviewers to be a part of the intellectual community associated with a journal. Every journal develops such a community where previously-published authors serve as reviewers of newly-submitted manuscripts. I also like the fact that the reviewer’s freedom to accept or reject an article is not compromised by this suggestion. So let’s try and improve on it.

    First, Elsevier is a businesss and businesses don’t like to give 15%, 20% or 25% off on their products. So it might not spring for this idea, good as it is.

    Here’s a suggestion for building a tangible reward that protects Elsevier’s interests. An author does not pay to publish in most Elsevier journals. Since the article is not fiscally sponsored by the author, it does not get publicly released by Elsevier. However, previous authors are also future reviewers. For every four to six reviews that an person does for Elsevier, s/he is given the chance to sponsor one of her/his own articles for free. Since the price of sponsoring is money that Elsevier would not have seen otherwise, they are not losing money. The helpful reviewer gets a tangible reward because one of his/her articles becomes publicly downloadabe. (Multiple authors on a paper can pool their points.)

  27. I support the idea of overall recognition of the reviewers based on the quality of the review. Incentives should include the badges, certificates which can be showcased on CV. In addition, incentives in the form of discounts on Elsevier books and/on coupons for books from (say) Amazon are also recommended.

  28. Good idea in principle, but as stated by others: quality is more important than quantitiy and should be reflectedadequately in the badge/reward system!

  29. I agree that free access to journals is not useful or an incentive. However, I don’t think having badges or certificates mean much either–no one would know how to read that from a vita either. I do like the idea of credits and discounts!

  30. I agree with the principle of the idea, but not with the proposed solution for implementation. We are not in a primary school to use the rating/notation system. The reviewer should be rewarded based on the quality of his work. In addition, there is the risk of favoritism, as mentioned earlier by Dr. Agrawal.

  31. It is good idea. The responsible reviewers should be rewarded. But at present it is hard to be fair for every reviewer. Paper is very important for some author. if it is published, it will change many things about author. So many author begin to look for the familar reviwer . It will effect the quality of paper . The responsible reviewers is very important for development of the journal .

  32. The original idea has great merit – together with the excellent contributions particularly from Dr Amit – as an early career researcher who appreciated quality detailed reviews of my own papers (even when they were highly critical!) I became a long time reviewer who has taken the role very seriously and was then offered a position on the Editorial Board in recognition of my contribution. Editors know who do serious quality reviews among their readers and contributors and I would hope that they would NOT invite reviewers who submit poor quality reviews to review again.

  33. I appreciate to all learned reviewers and commentators for commenting the ideas given by the Prof.Simon Gosling.Why you think about awards/rewards?Please be honest to review the papers.How to control and justify the quality and quantity of manuscript reviewed by the referees.

  34. James Oluwagbamigbe Fajemiroye (student)
    As much as incentive can be a catalyst for reviewing the submitted manuscripts in good spirit, qualitative and quantitative input requires, getting the right expertise to do the review job. A situation where a reviewer is given an assignment outside his scope, two things are bound to happen; 1. Accept the assignment, take longer time to review and make questionable comment 2. Outright turn down of the assignment. In case 1. Reviewer may end up achieving quantitatively which could earn him more incentive at expense of quality job. This in some instance can cause outright rebuttal of the reviewers comment by the author. I support incentive that rewards service and excellence. I will suggest that reviewer be awarded certificate or any written note acknowledging their intellectual contributions by taking into consideration the following:
    1. Impact of paper reviewed in terms of citation
    2. Quality of review in terms of rebuttal and corrections made by authors in their revision letter to editor. In case of outright rejection, reviewers comment should remain documented for future purpose
    3. Quality of the journal
    4. Nature of the paper – highly advanced work definitely needs to be graded separately
    5. Bias – in case there is need to develop ranking system for reviewer, the system must engender confidence

  35. “Who will benefit?” is almost entirely a repetition of last third of the Idea section. Says it would be useful for Editors “because they will be able to select “more experienced” reviewers for certain “high profile” papers”. I’m sure they already do that “in house”.

    Rewards for reviewers ? Probably a small fee per review, or perhaps a cumulative research fund (for purchase of software or small equipment ?) or travel/conference fee fund would be better than certificates and badges (Gold Star at kindergarten ?). Note that rewarding reviewers does not necessarily make them better reviewers. Otherwise, you would have to find some way to make employers, research grant bodies and governments value the review process. This would be difficult. It would have to be something that makes them look good or makes their lives easier. Listing reviews/certificates in your CV ? – you must be joking. Research funding bodies, especially, are looking for reasons to eliminate applicants. Fault-finding rather than point scoring. Non-academic employers usually hate it – you shouldn’t be wasting your (employer’s) time with non-profitable things like reviews.

    One (Elsevier) journal did once offer me a certificate in appreciation of my reviewing efforts. While it amused me to accept this certificate, it in no way enhances my promotion chances or research grants. I would be a “Senior Reviewer Badge” by the way.

  36. Idea of giving credit to the young reviewers is great. Generally people who review do it with a sense of responsibility and regard for the subject area.

    It will be difficult to rank the reviews. Also if the reviews are ranked with lots of qualification rules it may be useless for many reviewers to mention the reviewership in their CVs!

    People who do not write meningfull reviews may not even be sent articles for reviews!

    Regards,

    Gautam

  37. Several comments regarding the proposal:
    1 – as I am not working in University but in a professional public body, I have a very limited access to Elsevier journal. Nevertheless I am doing a lot of review and the 30d access is for me very valuable. May I recall that in research centres (this was the case in a previous job position), there is not an extensive list of journals and subscription is generally usually to the most important ones. Consequently it is not true to say that 30d access is unhelpful.
    2 – I am reviewing several papers from the ASCE journals, and recognition is made for outstanding reviewers in the form of a diploma. I do not know what is the basis for that (number of reviews, quality…) but a close look may be interesting.
    3 – the idea with the several proposed improvements looks very good. There is no perfect system and bias will always exist. Nevertheless a simple and automatic system has to be imagined to make the proposal efficient.

  38. Most contributors (commenter) do not know how the editing-refereeing process works.
    Generally, frequent reviewers provide the most comprehensive reports.
    “Big” names frequently reject to write reviews and write short &useless ones.
    A “big” name will be scored high by editors and colleagues, independently from the usefulness of their review.

    Similar systems exist (existed earlier) to this badge system. Top referee for a special journal (at Elsevier, not taking into account the dispersion of themes in thousands of journals), “most valuable reviewer” (each year), list of referees printed each year, etc.

    As the scientists definitely have some “vanity” and “envy” such budge-system, silver, gold platinum, (and renewable) is welcomed together with the random selection of reviewers (within a given topic).

  39. I support your idea, but you must harmonize the different reviewers’ work.

  40. The badges and rewards proposed system is a good idea but its implementation could be time consuming. In other journals i have seen a nice way to motive reviewers: the is a record of the numbers of papers reviewed by someone in a year, and they give to the reviewer 1 year free access to a journal (from a list to choose). This may not be so attractive to people in academia but to others like me working in the industry

  41. Good idea ! Indeed rewards, recognition and incentives are the back bone for reviewers and this idea shall encourage enthusiastic reviewers.

  42. I think it would be very nice to be able to have some form of official acknowledgement of my efforts to review papers. I recognize that the details of this type of system could be tricky but there are several scenarios suggested above that I would find workable.

  43. Just a note that Springer sends an official letter of appreciation to the reviewers that they consider have done an excellent reviewing job and places them in the top 5% of its reviews.

  44. Very good idea, especially with the modifications proposed by Dr. Agrawal. However, as pointed out previously, evaluation of reviewers would add work for the editors and an automatic method is difficult to implement (the longer the review, the better?) Authors could be asked to evaluate the review.
    Badges, Diploma,.. any paper from elsevier showing that you are a reviewer, would be much appreciated in many countries.
    Regarding rewards, anything reviewers find interesting. I particularly like the idea of sponsored papers from Dinshaw Balsara.
    In any case, this is a very good idea and should be implemented.

  45. The people who review do it with patience and with full responsibility as a reviewer the young ones do with lot of patience
    and with great energy the idea of of giving credit to the young reviewers is simply good this gives a great feeling to the reivewer. some sort of citation should be given in the form of certificate.

  46. This idea is very good and will be helpful for the researchers to maintain the review time and quality. Such recognition will create interest in the reviewers, to review ..

Comments are closed.