Stevan Harnad, Professor of Cognitive Science Southampton University, UK ---------------------------
Dear All, This contribution is in response to Richard Poynder's posting of 28 September 2004: [See, http://groups.undp.org/read/messages?id=97854#97854] > Mark McCabe argues that peer review is not endangered by a shortage of > eyeballs. > [See, http://groups.undp.org/read/messages?id=97687 ] Peer-reviewers (and their eye-balls) are a scarce, over-harvested resource. But Mark McCabe is right that there are probably enough qualified reviewers to keep up with the growth in the number of research articles. (The reviewers, after all, are just the authors -- the peers -- wearing other hats.) The Web, and online article indices and databases (including Open Access full-texts and reference lists) will help distribute the reviewing load more evenly, however. See: "Classical peer review will not change in its essentials in the online/open-access age, but it will become faster, cheaper and more efficient to implement. Papers will be submitted by depositing them in the journal's website or the author's institutional website. Many authors will elect to make their preprints publicly accessible, either on the journal's website or on their institutional website. Referee selection will be aided by online searches of the open-access literature and sometimes by calls-for-referees to targetted specialist lists. Referees will access the manuscripts online and submit their reports online, and editorial dispositions will be done online. Self-selected commentary may sometimes supplement (but not substitute for) the editor-appointed refereed. After publication, a similar system can be used for open peer commentary, and authors' responses. Article download and citation impact will be tracked from the preprint onward by online scientometric engines." http://www.ecs.soton.ac.uk/~harnad/Temp/peerevabs-amchem.html "PostGutenberg Peer Review: The Invariant Essentials and the Newfound Efficiencies" http://www.jpgmonline.com/documents/editor/25/10_Harnad_peerreview_3.pdf http://www.ecs.soton.ac.uk/~harnad/Temp/peerev.ppt > If, however, there were > effective alternative quality-control mechanisms available to do the > first-level > work of assessing scholarly papers might not those eyeballs be better > occupied on > something other than reviewing every paper written? Qualified experts provide corrective and evaluative feedback on specialized work in advance as a service to their busy peer user community -- so researchers know in advance which papers (1) justify the investment of their scarce reading time, and (2) which papers justify the further investment and risk of their even scarcer and more precious research time and effort, the risk of trying to use, apply and build upon them. This is peer-to-peer service provided for free by referees when they are wearing their referees' hats. Essential to the service is that it is (1) done in advance (i.e. *before* their fellow-researchers invest their time and effort in the paper), that it is (2) dynamic and interactive (i.e., it is not just a green-light/red-light tagging of papers as "publishable" or "unpublishable" [in a journal with a certain known track-record for quality] but a process of revision in which the quality of the paper is meant to improve] and it is (3) answerable (the referees are selected for their expertise by a qualified expert editor, the author is answerable to the referees and editor, the referees are answerable to the editor, and the editor and journal are answerable to the journal's users). There are potential ways to make this answerable system of human evaluation and feedback more efficient and equitable (see URLs above), but as far as I know, no *alternative* to it has yet been tested and shown to be successful, sustainable and scaleable, and to yield articles that are of a quality and reliability level at least equal to the one yielded by peer review. Alternatives, in other words, are merely in the mind of the speculator, until and unless they have been tested and demonstrated to succeed, sustain and scale. No such alternatives exist yet -- and open peer commentary in particular (post-hoc vetting by self-appointed "peers," with time on their hands and answerable only to the anarchic court of online opinion) is certainly no such alternative. See: Harnad, Stevan (1998/2000) The invisible hand of peer review. Nature [online] (5 Nov. 1998) http://helix.nature.com/webmatters/invisible/invisible.html Longer version in Exploit Interactive 5 (2000): http://www.exploit-lib.org/issue5/peer-review/ "Self-Selected Vetting vs. Peer Review: Supplement or Substitute?" (2002) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/2340.html Harnad, Stevan (1997) Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary and Copyright. Learned Publishing 11(4) 283-292. Short version appeared in 1997 in Antiquity 71: 1042-1048. Excerpts also appeared in the University of Toronto Bulletin: 51(6) P. 12. http://citd.scar.utoronto.ca/EPub/talks/Harnad_Snider.html http://cogprints.soton.ac.uk/documents/disk0/00/00/16/94/index.html > Concerning the larger question of whether peer review is relevant to a > discussion of OA: it might be easier to agree that it was not relevant if > all that was being archived along the green road were the "final, > peer-reviewed, revised, corrected, accepted" papers that Stevan Harnad > recommends. The reality, however, is somewhat different. As Stevan has > himself elsewhere pointed out > (http://www.infotoday.com/it/apr04/poynder.shtml) > what is currently being archived is "a good deal of the target content - > peer-reviewed > journal articles - but also preprints, unpublished papers, non-papers, and > metadata without the full-text papers". Anyone sifting for gold through this > plethora of material will surely be seriously challenged? While the > traditional > peer review process may be good enough when it comes to assessing what is and > is > not of relevance/good quality in the print environment, it may not be good > enough > in a self-archiving universe. Richard forgets that the journal-name -- the certificate that the paper has met that journal's established quality standards -- is still there to serve as the gold standard, as it always did, whether on-paper or on-line, whether for toll-based access or open access. Researchers are well aware of the difference between an unrefereed preprint and a refereed journal article. They have been dealing with it for years. But one *unwelcome* filter they have also had to deal with for years -- namely, whether or not their institution happens to be able to afford to pay for access to the journal in which a particular paper appears -- is no longer a constraint once all peer-reviewed articles are self-archived and made Open Access. As to whether the minuscule additional element of uncertainty as to whether an OA article is indeed what it purports to be (by the self-archiving author), please see the relevant entries on authentication, corruption and version-control in the BOAI self-archiving FAQ: http://www.eprints.org/self-faq/#2.Authentication http://www.eprints.org/self-faq/#3.Corruption http://www.eprints.org/self-faq/#23.Version None of these call for an alternative to peer review. The relevant role of peer-vetting in all this is not as a substitute for peer review, but as a detector of occasional scholarly lapses in self-archiving practises! http://www.eprints.org/self-faq/#7.Peer Yes, peer-review is relevant to a discussion of OA -- but only inasmuch as it is the *peer-reviewed* literature we are seeking Open Access *to*: This is done by providing a version of each article that is free of access-tolls, not one that is free of peer review! > Indeed, if one talks to researchers in the developing world it is clear that > many > view self-archiving not as a supplement to peer review, but as an > alternative. You'll see the same untested view in the developed world too (and even among some of the physicists and mathematicians who have been self-archiving pre-refereeing preprints for years). Peer Review Reform Hypothesis-Testing (started 1999) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#480 A Note of Caution About "Reforming the System" (2001) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#1170 Self-Selected Vetting vs. Peer Review: Supplement or Substitute? (2002) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#2341 The important thing in OA is to look at what "Simon" is *doing*, not what "Simon Says" he is doing: Peer review proceeds apace, and all the literature (and its quality) remains answerable to it; the self-archiving of pre-refereeing preprints has not changed that in the least. Pre-refereeing preprints are not so much a matter of Open Access (OA) as of "Early Access" (EA), as Michael Kurtz has noted. Preprints are to be used with caution, but they are certainly a useful supplement to Open Access to the later peer-reviewed versions in many fields and many cases: "OA advantage = EA + AA + QB + OA + UA" http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/3977.html But for most of the not yet self-archiving world, even OA is EA, and will continue to be until all peer-reviewed journal articles are 100% OA! > Unable to obtain a place at the peer-review table they believe self-archiving > offers the best chance they have of getting their work before the eyeballs of > their peers. Open Access is about Open Access to the 2.5 million articles published in the 24,000 peer-reviewed journals worldwide. Anything more (be it early access or books) is merely a bonus. If it is work that cannot meet the standards of peer review, it is not even that. Access to that is certainly not OA (for the same reason that access to anything else someone may decide to put on the Web -- be it an advert, a polemic, or a piece of porn -- is not OA). Please let's (continue to) distinguish web access in general from OA and the OA movement in particular! "The literature that should be freely accessible online is that which scholars give to the world without expectation of payment. Primarily, this category encompasses their peer-reviewed journal articles, but it also includes any unreviewed preprints that they might wish to put online for comment or to alert colleagues to important research findings." http://www.soros.org/openaccess/read.shtml > Perhaps the greatest mistake the OA movement could make is to assume that the > only > issue is one of access, without considering that in the process other things > may > have to change as well. Access is certainly not the only issue. There are many other worthy issues (poverty, illness, injustice) that have nothing to do with OA. But the only OA issue is access, and only access to the target literature (which is, to a first approximation, one and the same as the 2.5 million articles published in the planet's c. 24,000 peer-reviewed journals). The OA movement is not the "Open Information" movement. Human beings, with mouths to feed, may still want to sell their books, magazine articles, music, software, rather than give it away, even if it is all digital. But every single author of every one of the 2.5 million yearly articles in the 24,000 peer-reviewed journals wants to give them away so as to maximize their uptake, usage and impact. That is how (indirectly) the researcher manages to feed the mouths to feed: http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.2 http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.1 Napster http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#9.1 But OA and self-archiving are not about *self-publishing* either: http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm#1.4 They are about providing access to peer-reviewed research articles, both before and after peer-review. "What should be self-archived?" http://www.eprints.org/self-faq/#What-self-archive Nor are they about reforming or replacing peer review: Peer Review Reform Hypothesis-Testing (started 1999) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#480 A Note of Caution About "Reforming the System" (2001) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#1170 Self-Selected Vetting vs. Peer Review: Supplement or Substitute? (2002) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/subject.html#2341 Stevan Harnad Moderator, American Scientist Open Access Forum Professor of Cognitive Science Department of Electronics and Computer Science University of Southampton, UK URL: http://www.ecs.soton.ac.uk/~harnad/ --- 20 September- 4 October 2004: gpgNet Forum on "Open Access to Scholarly Publications: A Model for Enhanced Knowledge Management?" Co-hosted with the Open Society Institute (OSI). Read background paper to the discussion at http://www.gpgnet.net/topic08.php View messages posted to this forum at http://groups.undp.org/read/?forum=gpgnet-oa To post your comments on the issue, send them to: gpgnet...@groups.undp.org