On Sat, 17 Feb 2001, Greg Kuperberg wrote: > sh> Although it is probably a better idea not to draw still further > sh> attention to this thesis (that self-archiving is a SUBSTITUTE > sh> for refereed publication, rather than just a SUPPLEMENT), > sh> because of the confusion and opposition it understandably elicits,... > > gk> The claim that e-print archives might have no direct effect > gk> on peer review is a hard sell for BOTH believers and skeptics
It does and will have an effect, but mostly on the cost and efficiency of implementing peer review (online is cheaper and faster, especially if the submission can just be downloaded by the journal from an Eprint Archive URL, as we are doing with the paper journal I edit: http://www.bbsonline.org/). I believe online processing will also shore up some of the weaknesses of peer review, by putting referee selection on a much broader, hence potentially more equitable basis, allowing editors to search for and select qualified referees from a much larger potential pool. And shortening turnaround times can only help too, in many ways. The pre-refereeing feedback on the preprints will also be on a larger scale and faster than it was in paper days, so that will improve papers too, and help speed the process of convergence on an acceptable draft. But that is all part of the normal evolution of peer review (implementation) anyway. It has always taken advantage of new media (xerox, fax) to try to do things faster and more efficiently. That is not at all the kind of fundamental and speculative change that is being advocated without supporting evidence by the would-be "reformers" (including the outright eradicators) of peer review.. > With the benefit of experience, the objection to your claim can already be > read from your own terminology. How can you call peer review "a priori" > if it comes after contribution to the arXiv, which is so much better at > distributing papers than journals? What is it prior to? The answer is very simple: (1) There is, and always has been, "a priori" feedback from colleagues, prior to peer review. This is informal, nonbinding, and self-selected feedback. It was solicited by authors on-paper, it continues to be so on-line, on a wider scale, as noted above. THIS IS NOT PEER REVIEW! (2) In peer review, it is not the author who self-selects his referees, nor the referees who self-appoint themselves. The system is a BINDING and ANSWERABLE one, implemented by a (supposedly) competent, qualified, even-handed, expert Editor (or Associate Editor, or Board). It is the Editor who selects the referees and which of their recommendations will be binding; and the author is answerable to that editor and the selected referees. Of course the Editor is and always has been the weak link in peer review. If an incompetent or biassed Editor is in charge, things can go badly wrong. But fortunately, at least in science, the system is self-corrective, and incompetence and bias are eventually ferreted out (by authors, readers, referees) -- or at least so one hopes (and no one has demonstrated a better way so far). Journals' current (hierarchical) level of quality (such as it is) is a result of the degree to which this system is indeed working. If there is a better way, it first has to be tested and shown to work. Peer review is neither self-policing nor vigilantism. It is quality control of expert work, answerable to qualified expert editors, who select qualified expert referees. The editors themselves are in turn answerable to the peer community, for the quality of the contents of their journals. > When I write a new paper, I first arXiv it and then I submit it for > peer review. Empirically peer review for me is already "post hoc", > whether it extends an "invisible hand" into my research or not. We are right now doing a survey so we can weigh this kind of anecdotal observation on a population rather than an individual basis, and can compare what Simon says, and why, with what Simon actually does. Please ask your colleagues participate in the survey: http://www.eprints.org/survey/ But the short answer is: There has always been nonbinding a-priori informal feedback AS WELL AS binding formal feedback. How much a particular paper changes as a result of the one or the other is no doubt an individual matter; the survey will try to ascertain what authors FEEL is actually the case. How much the overall quality of the outcome is a consequence of the ubiquitous constraint of answerability (the "invisible hand" effect) will only be known when someone tests it empirically. NO ONE HAS DONE SO. (And, as I repeat endlessly, the archive status quo is NOT a test of the alternative hypothesis; everything is still submitted for peer review and 100% answerable to it, just as it always was. No new causal inferences can be made from that. [I should think that a little reflection would make this clear to mathematicians, even though theirs is not an empirical but an a-priori science; in any case, some self-archiving physicists have been making the same unwarranted inference, and theirs is!]) > Even if you don't like me giving myself as an example, it is certainly > not true that "everyone keeps submitting everything for refereeing". > I gave an example before: The arXiv article q-alg/9709040, by Maxim > Kontsevich, isn't published. This is hardly a forgotten, rejected draft > either. Kontsevich is famous, and the result in that article is one of > his crowning achievements. Indeed, q-alg/9709040 has earned a far more > prestigious form of peer review than journal publication, since it was > read and recognized by the Fields Medal committee. Prestigious though > it may be, it is also "post hoc" from any viewpoint. Alas, it is hard to draw empirical conclusions from an N of 1... > I have never interviewed Kontsevich to get his opinions on scholarly > communication. I think that he isn't interested in that topic and > that he harbors no particular skepticism about journals. He does still > publish most of his papers. But I also believe that Kontsevich hasn't > gotten around to publishing q-alg/9709040 because he has been from the > yoke of promotion. My theory is that he might publish it somewhere if > he had some free time, but he's busy with new research, and anyone who > wants it can get it from the arXiv. I hope the survey will give us a bigger N to base conclusions on (though of course it will have the sampling bias that always comes form self-selection!) -------------------------------------------------------------------- Stevan Harnad har...@cogsci.soton.ac.uk Professor of Cognitive Science har...@princeton.edu Department of Electronics and phone: +44 23-80 592-582 Computer Science fax: +44 23-80 592-865 University of Southampton http://www.ecs.soton.ac.uk/~harnad/ Highfield, Southampton http://www.princeton.edu/~harnad/ SO17 1BJ UNITED KINGDOM NOTE: A complete archive of the ongoing discussion of providing free access to the refereed journal literature online is available at the American Scientist September Forum (98 & 99 & 00 & 01): http://amsci-forum.amsci.org/archives/American-Scientist-Open-Access-Forum.html You may join the list at the site above. Discussion can be posted to: american-scientist-open-access-fo...@amsci.org