On Wed, 14 Apr 2004, Garfield, Eugene wrote: > The results obtained for computer science by analysis of CiteSeer are > distorted for a variety of reasons. They cannot be compared with the > literature of e.g. life sciences. Computer science is heavily dependent upon > conference literature. I cannot comment upon the physics literature, but > there are other studies which seem to indicate that readership increases > will not necessarily be followed by increased citation impact.
Gene Garfield, the father of citation analysis as well as ISI, is quite right that the Lawrence (2001) study on the impact enhancing effects of open access in computer science needed to be replicated in other fields to check whether it was merely an artifact of the fact that computer science is conference- rather than journal-based. Lawrence, S. (2001) Online or Invisible? Nature 411 (6837): 521. http://www.neci.nec.com/~lawrence/papers/online-nature01/ But, thanks to the ISI database licensed to OST and a special contract generously provided by ISI to conduct the study, we are in the process of testing the Lawrence effect across all disciplines in a 10-year ISI sample of 14 million articles. The physics analyses up to 2001 are already done, and they reveal even larger effects than those reported by Lawrence, with OA/non-OA citation ratios of 2.5 - 5.8. All indications are that 2002 will raise them even further, as the biggest effects occur within the first 3 years of publication in scientific disciplines (and both OA and the awareness and visibility of OA articles are also increasing yearly). Brody, T., Stamerjohanns, H., Harnad, S. Gingras, Y. & Oppenheim, C. (2004) The effect of Open Access on Citation Impact. Presented at: National Policies on Open Access (OA) Provision for University Research Output: an International meeting, Southampton, 19 February 2004. http://opcit.eprints.org/feb19prog.html http://www.ecs.soton.ac.uk/~harnad/Temp/OATAnew.pdf > In one study of a single chemical journal that I refereed there were about > 100 readerships for each citation of that journal, but there did not seem to > be any perceptible increase of citation by the research literature. The ratio of "reads" to "cites" will no doubt vary by field. Kurtz and co-workers report it as 17:1 and even 12:1 in astrophysics. Kurtz, Michael J.; Eichhorn, Guenther; Accomazzi, Alberto; Grant, Carolyn S.; Demleitner, Markus; Murray, Stephen S.; Martimbeau, Nathalie; Elwell, Barbara. (2003) The NASA Astrophysics Data System: Sociology, Bibliometrics, and Impact. Journal of the American Society for Information Science and Technology http://cfa-www.harvard.edu/~kurtz/jasis-abstract.html Kurtz, M.J. (2004) Restrictive access policies cut readership of electronic research journal articles by a factor of two, Michael J. Kurtz, Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA http://opcit.eprints.org/feb19oa/kurtz.pdf Tim Brody's remarkable download/citation correlator/predictor gives the size of the correlation by field, and can be used to predict citation 6-24 months later from downloads today (with an adjustable time-window): http://citebase.eprints.org/analysis/correlation.php > Undoubtedly the web will increase apparent readership of literature, but > that will not necessarily change the population of relevant researchers who > are in a position to cite particular studies. Not necessarily, but very probably! And also actually, in the fields tested so far. After all, access is a necessary if not a sufficient precondition for citation. And since Open Access (OA) dramatically increases the number of would-be users who would otherwise have been denied access to the article (maximises it, in fact, for all who have access to the web) it stands to reason that it can only increase both usage and impact. The way to test this, however, is not just to compare apples and oranges (i.e., OA and non-OA journals). The right way is to compare OA and non-OA articles in the *same* journals (and years). That is what our study with the ISI data is doing. (As Gene himself has often stressed, it is the article [and author] citation counts that should be weighed, and not just the average citation counts of the journals in which the article appears!) > I do not think the ISI study is definitive but it is not irrelevant. It is certainly not irrelevant to have shown "that there was no discernible difference in terms of citation impact or frequency" between the 191 OA journals and the 8509 non-OA journals indexed by ISI, equating for comparable journals as closely as possible: http://www.isinet.com/oaj But obviously there is a certain risk of circularity in this! It does show that the skeptics are wrong (for these OA journals): OA journals *are* indexed by ISI, and they *do* have comparable citation impacts. But the real test of the effect of OA on citation (and download) impact is on an article basis, where the journals can be equated exactly (by being the *same* journal and year!). And there the impact-maximising effects of OA are proving to be very dramatic indeed. Stevan Harnad > > Stevan Harnad wrote: > > > > The ISI press release says: > > > > "Today, Thomson ISI... announced that journals published in the > > new Open Access (OA) model are beginning to register impact in > > the world of scholarly research... Of the 8,700 selected journals > > currently covered in Web of Science, 191 are OA journals... [A > > study on] whether OA journals perform differently from other > > journals in their respective fields [found] that there was no > > discernible difference in terms of citation impact or frequency > > with which the journal is cited." http://www.isinet.com/oaj > > > > But if you want to get a better idea of the effect of OA on impact, > > don't just compare the 2% of ISI journals that are OA journals > > with the 98% that are not, to find that they are equal in impact > > (for this may well be comparing apples with oranges). Compare the > > much higher percentage of *articles* from the 98% non-OA journals > > that have been made OA by their authors -- by self-archiving > > them -- with articles (from the very same journals and volumes) > > that have *not* been made OA by their authors: You will find that > > there is indeed a discernible difference in terms of frequency > > with which the *article* is cited, and that that difference > > is from 250%-550% in favor of the articles that their authors > > have made OA! That is what an ongoing series of comparisons > > based on a 10-year sample of the same ISI database across all > > disciplines is revealing (in computer science and physics so far): > > http://www.ecs.soton.ac.uk/~harnad/Temp/OA-TAadvantage.pdf