If one assesses an institute's productivity by the papers from its researchers, and one rates those papers with the help of journal impact factors, is it not the case that one should expect the results to be in line with the citation counts for those papers? Is it me or is there a circular argument here?
Jan Velterop -----Original Message----- From: Peter Suber [mailto:[email protected]] Sent: 25 November 2002 01:59 To: [email protected] Subject: Re: UK Research Assessment Exercise (RAE) review In the recent postings on RAE ratings and scientometrics, I don't believe I've seen anyone cite this piece of research: Andy Smith and Mike Eysenck, "The correlation between RAE ratings and citation counts in psychology" (June 2002) http://psyserver.pc.rhbnc.ac.uk/citations.pdf The authors' summary: We counted the citations received in one year (1998) by each staff member in each of 38 university psychology departments in the United Kingdom. We then averaged these counts across individuals within each department and correlated the averages with the Research Assessment Exercise (RAE) grades awarded to the same departments in 1996 and 2001. The correlations were extremely high (up to +0.91). This suggests that whatever the merits and demerits of the RAE process and citation counting as methods of evaluating research quality, the two approaches measure broadly the same thing. Since citation counting is both more costeffective and more transparent than the present system and gives similar results, there is a prima facie case for incorporating citation counts into the process, either alone or in conjunction with other measures. Some of the limitations of citation counting are discussed and some methods for minimising these are proposed. Many of the factors that dictate caution in judging individuals by their citations tend to average out when whole departments are compared. Peter ---------- Peter Suber, Professor of Philosophy Earlham College, Richmond, Indiana, 47374 Email [email protected] Web http://www.earlham.edu/~peters Editor, Free Online Scholarship Newsletter http://www.earlham.edu/~peters/fos/ Editor, FOS News blog http://www.earlham.edu/~peters/fos/fosblog.html
