>I forgot the link >http://www.timesonline.co.uk/tol/news/world/iraq/article1469636.ece > > >http://www.sciencemag.org/cgi/content/summary/293/5538/2187a > >SCIENTIFIC PUBLISHING: >Peer Review and Quality: A Dubious Connection? >Martin Enserink >BARCELONA, SPAIN--Despite its flaws, letting scientists anonymously >judge each other's work is widely considered the "least bad way" to >weed out weak manuscripts or research proposals and improve promising >ones. But that common wisdom was questioned last weekend at a meeting >attended by hundreds of editors of medical journals and academics. In >a meta-analysis that surprised many--and that some doubt--researchers >found little evidence that peer review actually improves the quality >of research papers. >
this is all very cute but it doesn't say anything. I think I know a bit more about this topic than anyone here (wife works on a medical journal and knows the peer review process inside and out, and I know something about meta-analysis, doing consulting work in the area on a regular basis, for more info see http://www.lyonsmorris.com/MetaA/). first off the methodology the authors used was flawed. Vote counting approaches really does not say anything about the relationship they studied. Moreover the authors did not make any with-in group comparisons, its another comparing apples and oranges problem. In other words they lumped all sorts of studies together and compared the summed result with the summed result from the non-peer reviewed studies. A much better approach would have been to have done within study type comparisons - i.e., compare all peer reviewed articles that have looked at heart attacks with non peer reviewed studies. Generally this meta-analysis is not all that good. Simple tripe would be a good descriptor. A much better approach would have been to look at methodological quality, ie was it a well designed an run study. In the work I've done, I've found this to be a much larger determinant of effect size than almost anything else. Unfortunately the peer review process cannot protect against most crap. Typically what happens is that the editor has a very limited pool of reviewers. They may know how to read a radiograph or NMR but have no clue on study quality. larry ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Upgrade to Adobe ColdFusion MX7 The most significant release in over 10 years. Upgrade & see new features. http://www.adobe.com/products/coldfusion?sdid=RVJR Archive: http://www.houseoffusion.com/groups/CF-Community/message.cfm/messageid:230499 Subscription: http://www.houseoffusion.com/groups/CF-Community/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.5
