Update on the Australian ERA (research evaluation)
Below is the Minister's recent release on ERA. To help members of this list to interpret it, · Consultation on the submission drafts sent out has meant that the time window for the submission for the PCE and HCA clusters has been pushed out from 20 April - 8 May 2009 to June (PCE) and August 2009 (HCA). · The PCE and HCA cluster evaluations have morphed into a "trial". · Esteem indicators will not be included in this trial, and the ARC will do more work on what esteem metrics it can create/find in the full ERA. Personal analysis My guess is that universities (mainly) complained about preparing submissions in the original timeframe, resulting in the push-out. It was certainly tight. Conveniently for the Government in these financial times it is now in the next financial year. Nevertheless universities will still be in full preparation mode as 4 or 5 months still does not give them much time. The omission of esteem also effectively causes the morphing into a trial, probably better called a dummy run. The real ERA has thus been delayed a year, and it is unclear whether the PCE and HCA clusters will still be the first to be done properly in 2010. That would seem sensible to capitalize on the work already done by both universities and the ARC's committees. We await the finalized Submission and Technical guidelines due in March. Note that we also still await the selected supplier of the citation data for the HCA cluster. The PCE cluster data will be supplied by Scopus as I previously posted. It is not known if the PCE evaluation will use Scopus' SJR metrics (the PageRank algorithm applied to cites instead of hyperlinks). It would be nice if it did, to make the metric mix richer and because the SJR seems to be a better measure of the scientific value of a paper than raw cite counts. Arthur Sale Emeritus Professor of Computer Science University of Tasmania PS Roughly PCE = Physical, Chemical & Earth Sciences Cluster; HCA = Humanities and Creative Arts Cluster (including Fine Art, Music, Drama and Architecture), but they are both defined by ANZSRC codes. http://minister.innovation.gov.au/Carr/Pages/ERATIMEFRAME.aspx ERA TIMEFRAME The Minister for Innovation, Industry, Science and Research, Senator Kim Carr, today announced the timeframe for the Excellence in Research for Australia (ERA) trial, which will evaluate the Physical, Chemical and Earth Sciences (PCE) and Humanities and Creative Arts (HCA) clusters. "The Australian Government is committed to the development of a world- class research quality and evaluation system that has the confidence of the sector," Senator Carr said. "Recent feedback on the ERA Submission Guidelines and ERA-SEER Technical Specifications raised concerns about the proposed timing of the ERA trial. "To ensure universities are adequately prepared for the trial, submissions for the PCE cluster will now commence in June 2009, and submissions for the HCA cluster will commence in August 2009. "The outcomes of these trials will inform the full ERA process in 2010. "This timeframe will ensure that universities are able to collect and submit quality and accurate data for ERA. "Recent feedback also raised concerns about the scope of some of the data to be collected. "Following this feedback, esteem indicators will not be included in the ERA trial. I have asked the ARC to further investigate the collection of esteem indicators, which will be included in future evaluations. "The expert review and feedback that the ARC has received so far has been crucial in shaping ERA. "I trust universities will be pleased with the timeframe I have announced today and I ask that they continue to provide constructive feedback on ERA to the ARC." The ARC will release the final ERA Submission Guidelines and ERA-SEER Technical Specifications for the trial, in early March. You can find out more about ERA at www.arc.gov.au Media contacts: Patrick Pantano, Minister's office, 0417 181 936 Sheena Ireland, ARC, 0412 623 056