I am puzzled by writer Stan Moore's query, "These are questions that
enter my mind after perceiving many failures in peer review while
reading the scientific literature over the years."
I have several key questions:
1. Which scientific literature has been read, and was being referred
to? Is this the scientific discipline in which the writer was trained?
2. What was the nature of the perceived failures of review? Was he
concerned by verifiable errors of fact? Clear errors in data
analysis, statistical or otherwise? Clear errors in interpretation
of the data? Were the paper's conclusions suspect, and if so, why?
3. I have no idea what is meant by the writer's query about "ultimate
and subultimate goals." Has the writer himself served as a referee?
After almost three decades of serving either as an Associate Editor
(AE) or as reviewer for more than a dozen different journals, both in
ecology as well as in general science (Science, Nature, PNAS), I have
never once considered the review process to be a superficial
exercise, and I certainly never considered it to be primarily a
spell-checking and grammar review exercise.
On the contrary, I view the role of reviewer as a critical component
of the scientific process: he or she is expected by the journal in
question to accept or reject manuscripts based upon a number of key
criteria, including (but not limited to) suitability for the journal
in question; the methodologies used; scientific soundness;
consistency with existing knowledge; the quality of exposition; and
the degree to which the authors' conclusions are supported by the
data which were presented and analyzed in the manuscript being
reviewed. As an AE, I always chose reviewers who were considered to
be established, knowledgeable experts in the research topic covered
by the manuscript being reviewed. As a reviewer, I will not accept
an invitation to review any manuscript written by a close personal
friend or close collaborator.
Please remember, this is objective science we are talking about, and
not subjective politics. In dealing with the general public, for
example, I try to emphasize that scientific papers are not like
articles in Time magazine. Opinions don't count: hypothesis
testing, the use of rigorous and up-to-date scientific methodologies,
and the collection of independently verifiable data are essential
currencies of legitimate scientific inquiry. Fraud and deliberate
manipulations or distortions of the data will ultimately be
identified and revealed as falsehoods, and will be rejected by the
scientific community.
I urge the writer to view the Instructions for Reviewers of ESA
journals at http://esapubs.org/esapubs/reviewers.htm, which I hope
will help address many of his questions and concerns; similar
guidelines for reviewers are available on line for many other
scientific journals. As emphasized by the ESA guidelines, objective,
high quality peer reviews are essential for insuring the quality of
scholarly journals -- and thus for insuring the advancement of
scientific knowledge.
Val H. Smith
Professor
Department of Ecology and Evolutionary Biology
University of Kansas
Lawrence, KS 66045
At 03:53 PM 7/9/2007, Ben Bond-Lamberty wrote:
>On 7/9/07 9:53 AM, "stan moore" <[EMAIL PROTECTED]> wrote:
>
> > What is the minimal obligation of any peer reviewer for any scientific
> > publication? For instance -- must the peer reviewer read all the reference
> > materials?
>No: the time commitment could be enormous. But presumably the reviewer is
>someone in the field, and so will be familiar with most of the references.
>I would say the "minimal obligation" is to read and professionally evaluate
>the ms, while disclosing any conflicts of interest, within the time period
>requested by the journal.
>
> > How much time and effort should go into any peer review?
>I expect, on average, to spend 2-5 hours on a ms, including writing the
>evaluation.
> >
> > What should be the ultimate and subultimate goals of the reviewer?
>??
> >
> > For instance, is the overall objective of the paper under
> review subject
> > to question?
>Sure. Perhaps it isn't appropriate for the journal, for instance.
> >
> > Is peer review more superficial and mostly a spell-checking and grammar
> > review excercise?
>Not in my experience.
> >
> > On what basis should peer reviewers be chosen -- species expertise,
> > technical application of scientific method?
>Either or both. Ideally a ms is read by many reviewers who bring multiple
>points of view to bear.
> >
> > Does reviewer acquaintance, friendship, collaboration with author(s)
> > constitute bias?
>Yes, and thus it should be disclosed. But it doesn't necessarily disqualify
>the reviewer.
> >
> > How should obvious errors in peer review after publication be dealt with?
>Can you give a more concrete example?
>
> > These are questions that enter my mind after perceiving many failures in
> > peer review while
