We continue on the interesting (but alas evidence-poor) question of
discipline-differences in the preprint/postprint difference ("DIFF"),
and, in particular, the question of what percentage of submitted papers
never gets published by any journal in any form.
On Wed, 6 Dec 2000, George Lundberg wrote:
> large numbers of papers submitted to biomedical journals are of
> insufficient quality to appear (either at all or in the form in which
> they were originally submitted/rejected) in any "good" journal
Helene was asking about the percentage in the "not at all" category,
rather than the revise-and-resubmit category, although both would be
of interest (if only anyone had actual data!).
> i am not at all sure that Stephen Lock's frequently quoted 1984 number
> bears any relation to current experiences
Lock reported that in biomedical research just about everything
eventually appears somewhere, in some form. So in the end the function
of peer review is to determine where (and, equally important, in what
form, with what content) a paper should appear: Peer review is not a
passive red-light/green-light filter, it is a dynamic, interactive,
iterative, corrective filter that actively changes the contents and
form of preprints.
Lock, Stephen. A difficult balance : editorial peer review in
medicine / Stephen Lock. Philadelphia : ISI Press, 1986.
So, as a dynamic quality-shaper and certifier, peer review sign-posts
the level of quality of a paper at the locus where it eventually
appears -- a hierarchy of journals, from those with the highest
quality, rigour of refereeing, rejection rate, and impact factor at
the top, grading all the way down to journals so unrigorously reviewed
as to be little more than a vanity press.
(I am describing the standard lore here: I do not have data either.)
The function of this sign-posted hierarchy is to guide the reader and
the user, who have finite reading time and research resources, and need
to make sure they are reading reliable work, worth taking the risk of
building upon and worth citing. Researchers can pick their own level,
depending on their time, resources, and the aspired quality level of
their own work. They can decide for themselves how low in the
hierarchy they wish to go.
> At JAMA for my 17 years we rejected roughly 85% of all articles
> received. Many did appear in other journals, but a huge number seemed to
> simply disappear. We believed that was a good thing. i do not know of
> any recent study that hangs credible numbers on those observations.
Nor do I know of recent studies on this. (Does anyone?) But note that
apart from JAMA's 85% rejection rate (which attests to its being one of
the journals at the top of the clinical-medical hierarchy, along with
NEJM, Lancet and BMJ), George is not in a position to provide objective
data on what proportion of JAMA's rejected papers never went on to
appear anywhere, in any form. That would require a systematic follow-up
study (taking into account, among other things, title changes, and
possibly stretching across several years after the original rejection).
It would be splendid if someone gathered (or already had) such data.
I think we can all agree that in clinical medicine, where erroneous
reports can be hazardous to human health, it would be a good thing if
they never appeared anywhere, in any form. But in the online age
especially (what with child porn and hate literature proving so
difficult to suppress), this problem is well beyond the powers of
journals and journal editors.
Harnad, S. (2000) Ingelfinger Over-Ruled: The Role of the Web in
the Future of Refereed Medical Journal Publishing. Lancet (in
press)
http://www.ecs.soton.ac.uk/~harnad/Papers/Harnad/harnad00.lancet.htm
In the vast majority of research that has no bearing on human health
and welfare, however, it is not clear how strongly we should be
believing that it would be "a good thing" if a a "huge number" of
preprints rejected at one level of the hierarchy "simply disappeared"
rather than moved downward till they found their own level (including,
at the very bottom, permanent unrefereed status in the preprint sector
of the eprint corpus -- the eprint archives' "vanity press").
Who is to say what would be "a good thing" here for research, across
disciplines, a priori? This is the problem of the "wheat/chaff" ratio
that inevitably dogs every area of human endeavour: We would like to
have only the cream, and not the milk, but alas not only does human
performance invariably take the shape of a bell curve, but there is no
known way of ensuring that one can filter out the top 15% of that curve
without letting it all flow. (Not to mention that, peer review, being
human too, often misfilters, mistaking [to mix metaphors] wheat
for chaff and vice versa. The only protection against this is time,
and a retrospective record, for possible second thoughts about a piece
of work.)
Harnad, S. (1986) Policing the Paper Chase. (Review of S.
