Dear All, Very interesting debate indeed. We are debating two different questions here. First, whether the review process has to be blind, double blind or total transparent, and, second, what do to with the content of the reviews we all get from our submitted manuscripts. I don't have a definitive answer to the first, but share most of what has been said here.
In fact, I would like to come back to the first one at some point, but now I am very most interested in the 2nd one, which initiated this debate. Nobody teaches us how to review an article. Jonathan's initial intention was to use some review comments he got for pedagogical purposes. We all know that learning how to review correctly is very important. Journals try to guarantee a confidential reviewing process. That is why most journals say: "Please keep all reviews, including your own, confidential.", when they contact referees. However, the confidentially of the most important official highly secret documents of any country in the world prescribe at some point. My feeling is that journals mean that you should keep all materials, from the ms to the reviewer comments and author replies and responses confidential as long as the reviewing process takes place. Once it is over and a final decision is made, that confidentiality should prescribe. So as a reviewer, if you want to publish your own review reports once the original paper has been published, since, as Hal said, the reviews are copy-right materials by their own authors by default, you should be allowed to do it. Otherwise, if you want to publish reviews you got, you could probably write about them with quotations, and cite them as Anonymous and the year in your own work, teaching, writings and papers. A related question is that, as a referee, I have always found very frustrating that all referee's work and scientific communication back and forth with authors until final acceptance is mostly lost. This is a shame. In the old times, when Philosophical Transactions of the Royal Society was the only journal, papers and reviewers' comments and criticisms where available to the readers. My frustration comes from the fact that, after writing a long review that took me time and work, the final accepted paper is usually unable to reflect my contrasting points of views from those of the authors, even I may think that authors' contribution is somehow worth to be published. Some referees are very conservative, they tend to reject a paper, not because it is basically and technically wrong, but only because it takes a different perspective they don't _find_ "interesting" or "useful" enough or "appropriate" enough for the journal, etc. I underscore here "_find_" precisely to remark that there is a lot of subjectivity in this attitude. To tell the truth I don't find this "defensive attitude" totally right. I think that a possible reason for that it is that the referee does not have any way to communicate to the final reader his/her (sometimes slightly) different perspective about the matter. If the paper is finally accepted, only authors' views and the way they have filtered all referees' comments will be ALL that will reach the final reader. Since referees are very aware of that, they usually take a that conservative approach: "If I don't like something, I reject". To avoid that, my suggestion is precisely to give some publicity to referee's comments. The journal should give the option to the reviewer of writing a one-paragraph short final evaluation of the paper under review. This evaluation should be published online. So, when the paper appears online, readers could go and click on a bottom "referee opinion", where final referees' assessments are there to be read. If a paper is finally accepted against the comments of a single referee, the reader has the right to read and learn from and access to these comments and judge the paper accordingly by him/herself. If the paper was accepted with an unanimous excellent from all the reviewers, this is a very important information readers would value. Obviously, if such a thing happens when the paper was reviewed by only two referees, this has a different meaning that when it was reviewed by 5 referees, etc. For all journals I know of, this information is lost and I am convinced the reader would like to know about it. I really think that readers would be extraordinarily interested in knowing the opinions of scientists that acted as reviewers of any given published paper because reviewers are world experts on that matter. All journals should implement my suggestion. It is simple and doable. Cheers, David. On Wed, Mar 3, 2010 at 2:46 AM, Wayne Tyson <landr...@cox.net> wrote: > "Consider the source!" That's what my grandma used to counsel when I whined > about gossip and unfounded criticism. But it seems to me that reviews should > be considered part of the paper. Largely because of its transparency and > directness, I value Ecolog more than tedious, pretentious papers as a source > of good bottom-line thinking on ecology. > > And open criticism is the other; at Ecolog, you get lots of feedback when > you err, and that is the most welcomest contribution of all. I sometimes get > some "interesting" comments via off-list email too, which, if I want to post > so the rest of you are aware of it, I must get the contributor's permission > to post my rebuttal along with the off-list review. This, of course, never > happens, because those who expose themselves in rude ways want to confine > their flashing to their victims. I suspect that this practice of off-list > sniping (to put it politely) is not limited to me, so we "Ecologgers" are > being deprived of some pretty creative comment. My own policy is to nearly > always post to the list (except when I figure it might not get past David > for some good reason), and to always expect that the recipient of the email > has every right to post the review and rebuttal onto Ecolog. I do fully > understand that such posts put David on the spot, as he doesn't want "flame > wars" on Ecolog, but, then, as grandma said, we should be able to "consider > the source." Anyone has my permission to publish anything I send them that > is in the slightest of relevance to ecology or otherwise of interest to > Ecolog subscribers. > > Where I grew up, rudeness was often rewarded by a punch in the nose. > Punches were rare, and people were polite but frank. Courtesy was a social > institution. I miss the plain-speaking evenness of those people. Perpetual > archival of pixel petulance on Ecolog might keep the discourse even more > civil and certainly more informative than it is. I suspect that published, > signed reviews would similarly shape-up the quality of the academic > discourse much as it did the social discourse of my boyhood. > > As the beer ad says in England, "Take Courage!" > > WT > > > ----- Original Message ----- From: "Jonathan Greenberg" < > greenb...@ucdavis.edu> > To: <ECOLOG-L@LISTSERV.UMD.EDU> > Sent: Tuesday, March 02, 2010 1:40 PM > Subject: Re: [ECOLOG-L] Are reviews anonymous? > > > Wow, I clearly hit on a painful topic here! I've been getting asked > on-list and off- to clarify, a bit, why I might want to do what my > original question was about: make the reviews (but not the identity of > the reviewers) I've received of my OWN papers public to other people. > I would hazard to guess 99% of us have had a least one paper rejected > due to a reviewer who did a less-than-stellar job at reviewing a paper > -- whether laziness, unfamiliarity with the subject matter, politics, > competition on a subject that the reviewer may be trying to get > published themself, etc (how often have you thought "Did this person > even read my paper?"). There don't seem to be a lot of forums out > there to shed light on "good reviews" vs. "bad reviews" -- we receive > them, we complain to our co-authors and other people we are close to, > but, as my OP indicated, I wanted to know if we could turn around and > show the world these types of reviews in an attempt, not as revenge > (since the reviewer remains anonymous), but to let people know these > types of reviews are unhelpful and can really damage the scientific > process, hopefully to get across to the people producing these bad > reviews they need to think about how they treat their role in the > peer-review process, and to teach new reviewers how to write good > reviews. > > I want to clarify a point: I don't mean that a "bad review" is one in > which your paper is rejected, a "bad review" I define a bad review as > one that does little to help improve a paper that may have potential. > > --j > > > On Tue, Mar 2, 2010 at 12:02 PM, Murray Efford > <murray.eff...@otago.ac.nz> wrote: > >> James Crants has it right. I recently wrote a harsh review of a poor paper >> by a high-profile author, pointing out numerical and conceptual errors and >> disregard of the literature. I did what I thought was the decent thing and >> signed the review. The paper was published with a less-than-gracious >> acknowledgment of my contribution. This should not have got past the >> editors, but it did, and I will not sign reviews for them again. Anonymity >> serves to depersonalise the review process and dilute the pernicious effects >> of status and reputation. >> >> Murray Efford >> >> ________________________________________ >> From: Ecological Society of America: grants, jobs, news [ >> ecolo...@listserv.umd.edu] On Behalf Of James Crants [jcra...@gmail.com] >> Sent: Wednesday, 3 March 2010 5:39 a.m. >> To: ECOLOG-L@LISTSERV.UMD.EDU >> Subject: Re: [ECOLOG-L] Are reviews anonymous? >> >> On the other hand, if the reviewers are anonymous, the authors should be, >> too. I think transparency is a bad thing, in this case; I think reviews >> should be double-blind. >> >> While reviewer anonymity allows reviewers to be impolite and harsh, it >> also >> protects them from retribution for simply being honest when a paper is >> bad. >> Yes, scientists should all be mature enough to accept a negative review >> without trying to punish the reviewer, but some just aren't up to that >> challenge (if you can't think of a way for one scientist to punish >> another, >> or if you can't think of a scientist who would do such a thing, you can't >> be >> trying that hard). Even if you can't imagine deliberately punishing >> someone >> for their review, you must be able to imagine being miffed at a colleague >> who gives your paper a bad review, or having them be miffed at you, even >> if >> the bad review is merited. That's an incentive to be polite, sure, but >> also >> an incentive to let things slide that shouldn't be allowed to slide. I >> think the benefits of reviewer anonymity outweigh the costs. >> >> Author anonymity would have a similar advantage: it would make it >> harder for reviewers to pan someone's work just because they don't >> personally like the author, or to reward their friends with favorable >> reviews. Obviously, if the reviewer is quite familiar with the author's >> other work, it is possible to identify the author by writing style, study >> system, and hypotheses raised, but any uncertainty about the authorship of >> a paper under review should go one step toward dissuading reviewers from >> letting personal feelings hold too much sway over their judgement. >> >> Author anonymity could also prevent reviewers from judging authors and >> their >> works harshly based on their earlier submission of an unpublishable paper. >> If you thought someone's submitted paper was a real dog, you might not >> think >> much of their intelligence, and you'd give less credence to anything else >> they said subsequently. If the author were genuinely a poor scientist, >> you'd be ahead of the game by learning to doubt them early on, but if they >> were a solid researcher, and their name were on a bad paper for any of a >> hundred other possible reasons, you'd be cheating yourself by selling them >> short. >> >> (Sorry for taking another step down this tangental path. Also, I wish our >> language had a singular pronoun for a person of unspecified sex.) >> >> Jim Crants >> >> On Tue, Mar 2, 2010 at 3:44 AM, Marc Kochzius <kochz...@uni-bremen.de >> >wrote: >> >> Dear All, >>> >>> I agree completely with Kevin that reviewers should sign their review. >>> That's what I started to do and I will not make any reviews for journals >>> that insist that I stay anonymous. From my point of view the problem is >>> that >>> some colleagues hide in anonymity and provide reviews that are not >>> adequate >>> (e.g. impolite, unsubstantiated criticism). Another problem in this >>> context >>> are the editors. I think it is their responsibility to check if a review >>> is >>> adequate. However, my experience is rather that most editors just pass >>> the >>> review to me and I just wonder what kind of reviews I receive. In many >>> cases >>> there is absolutely no quality control regarding the reviews. From many >>> journals I also never get a feedback about my review, nor do I receive >>> the >>> reports of the other reviewers. This makes it impossible for me to >>> evaluate >>> if my review was in concordance with the other reviewers. >>> >>> Regarding the anonymity of the author, I think both sides (author and >>> reviewer) should be named, the system should be as transparent as >>> possible. >>> Unfortunately, it is currently not transparent at all. >>> >>> Cheers, >>> >>> Marc >>> >>> >>> Kevin Murray wrote: >>> >>> Off the point here, but I think that the anonymity should be reversed. >>>> Authors should be anonymous and reviewers should be named. >>>> >>>> Start a peer review revolution...sign all of your reviews!!! >>>> >>>> Regarding YOUR own reviews. It seems that, if they are anonymous, then >>>> posting should be ok. If the reviewer is named, however, you should not >>>> post. No laws or moral values were consulted in regards to this email. >>>> >>>> KLM >>>> >>>> >>>> >>>> On Mon, Mar 1, 2010 at 5:09 PM, Jonathan Greenberg < >>>> greenb...@ucdavis.edu >>>> >wrote: >>>> >>>> >>>> >>>> Interesting -- I'm primarily interested in reviews YOU receive on your >>>>> own submitted manuscript (which, 99% of the time, you don't know who >>>>> they are from) -- are you allowed to post these in any public forum? >>>>> Since the reviews cannot be linked back to an individual (unless that >>>>> individual steps forward and takes credit for it), and it is a >>>>> criticism of your own work, it seems like one should feel free to post >>>>> these if you want. I was interested in compiling the types of reviews >>>>> people get on manuscripts for teaching purposes, so I'm trying to find >>>>> out if its legit for people to share these reviews with me if they end >>>>> up going out into the public (e.g. on a website)? >>>>> >>>>> --j >>>>> >>>>> On Mon, Mar 1, 2010 at 3:07 PM, Jonathan Greenberg <jgrn...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> Interesting -- I'm primarily interested in reviews YOU receive on your >>>>>> own submitted manuscript (which, 99% of the time, you don't know who >>>>>> they are from) -- are you allowed to post these in any public forum? >>>>>> Since the reviews cannot be linked back to an individual (unless that >>>>>> individual steps forward and takes credit for it), and it is a >>>>>> criticism of your own work, it seems like one should feel free to post >>>>>> these if you want. I was interested in compiling the types of reviews >>>>>> people get on manuscripts for teaching purposes, so I'm trying to find >>>>>> out if its legit for people to share these reviews with me if they end >>>>>> up going out into the public (e.g. on a website)? >>>>>> >>>>>> --j >>>>>> >>>>>> >>>>>> On Mon, Mar 1, 2010 at 2:16 PM, Christopher Brown <cabr...@tntech.edu >>>>>> > >>>>>> >>>>>> >>>>>> wrote: >>>>> >>>>> >>>>> Jonathan, >>>>>> >>>>>>> >>>>>>> As it so happens, a message close to yours in my email folder was >>>>>>> from >>>>>>> a >>>>>>> review I did for American Naturalist. As part of the message from the >>>>>>> editor is the line "Please keep all reviews, including your own, >>>>>>> confidential." Thus, at least for Am Nat, it appears that the reviews >>>>>>> should remain unpublished in any form. >>>>>>> >>>>>>> CAB >>>>>>> ******************************************** >>>>>>> Chris Brown >>>>>>> Associate Professor >>>>>>> Dept. of Biology, Box 5063 >>>>>>> Tennessee Tech University >>>>>>> Cookeville, TN 38505 >>>>>>> email: cabr...@tntech.edu >>>>>>> website: iweb.tntech.edu/cabrown >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Ecological Society of America: grants, jobs, news >>>>>>> [mailto:ecolo...@listserv.umd.edu] On Behalf Of Jonathan Greenberg >>>>>>> Sent: Monday, March 01, 2010 12:48 PM >>>>>>> To: ECOLOG-L@LISTSERV.UMD.EDU >>>>>>> Subject: [ECOLOG-L] Are reviews anonymous? >>>>>>> >>>>>>> Quick question that came up recently that I was curious about -- I >>>>>>> know >>>>>>> REVIEWERS are anonymous, but are the reviews you get supposed to be >>>>>>> anonymous, or can they be posted in a public forum? >>>>>>> >>>>>>> --j >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >>>> >>> >> > > > -------------------------------------------------------------------------------- > > > > No virus found in this incoming message. > Checked by AVG - www.avg.com > Version: 8.5.435 / Virus Database: 271.1.1/2718 - Release Date: 03/02/10 > 07:34:00 >
