On Wed, 5 Mar 2003, Arthur P. Smith wrote: > >>There seem to be a few cases of this in bio-medicine in recent years > > > >Which? > > The placebo-effect for one. A recent review seems to have seriously > questioned the original "placebo-effect" research.
But placebo effects were always fuzzy, and hardly medical. Medically, their only role has been as a control for the effects of (putative) treatments: I give a drug to N patients, a sugar-pill (placebo) to N patients (double-blind), and I compare the effects. If significantly more patients get better with the drug than with the placebo, then maybe the drug really works. If I had just compared the drug patients to N patients getting no treatment at all, and there was a significant improvement, then that might have just been the "placebo" effect: The inevitable proportion of patients who improve either spontaneously or because of suggestive effects from belief in the drug. (Placebo research borders on another fuzzy area, hypnosis research, which in turn grades into a still fuzzier area, namely, parapsychology: telepathy, telekinesis!) But none of this is biomedicine! No one ever started selling "placebo" as such in drug stores, as a medicine (though many medicines sold as something else are probably just placebos). Placebo, in brief, is *non-effect* in medicine. The only thing that keeps coming and going cyclically is the tendency to overinterpret placebo as being something *more* than just that. Placebo effects are the perfect example of the kind of effect on which nothing can be built! > Almost any "twin" study on heritability seems to be met with great > skepticism now since the abuses of the original researchers in the early > 20th century were uncovered. I think this is dead-wrong (as stated), and more in the nature of what the Swedish journalist might say, one way or the other. Heritability studies in twins are robust for all kinds of heritable traits (e.g., tendency toward heart disease, obesity, alzheimer's). You probably mean the twin studies on the heritability of IQ, in which some data were inflated by Cyril Burt. But there the paradoxical result was that although his data-fudging was discovered -- and for a while discredited research on the heritability of IQ, and even on the reality of IG itself -- it turns out that he need not have bothered inflating his data, because the effect was real! Yes, a significant portion of the variance in IQ is heritable too, along with heart disease, obesity, and alzheimer's. (Mendel did similar fudging, yet his effects turned out to be real too.) So this is not an example of persisting undetected error either: It is rather an example of the opposite: That the self-correctiveness prevails after all, even when *real* errors, even deliberate ones, are suspected and detected -- if the errors happened to be in the direction of the true effects anyway! > And there seem to be frequent reports of researchers only publishing > "positive" results about drugs, under the encouragement of the companies > funding their research. That's certainly true. But that's not science; that is the drug business. In that gray area between hypothesis and hype, nolo contendere. > Andrew Odlyzko mentions an interesting phenomenon that has arisen in > response to the flood of information, much of it of questionable quality > - the "authority site". You could think of these as another level of > "peer review", although they tend to be run by just a single individual > (and consequently somewhat narrowly focused - but that's not that > different from a highly specialized journal). Should we be moving to > fund this sort of thing as a replacement for peer review, or in > addition? Is there some way these should be formalized and included > within the recognized knowledge infrastructure of our fields? How do you > ensure the trust-worthiness of those responsible for these efforts (and > how exactly is that different from trusting peer review at a regular > journal?) What aspects of the current information infrastructure for > scholarly communication help, or hinder, the work of the people creating > these sites? As I have pointed out repeatedly to Andrew and Greg Kuperberg in this Forum, these are all merely *supplements* to peer review, not *substitutes* for it, because the self-archived preprints are virtually all submitted for peer review, as they always were. At best, these authorities provide a faster clearing-house for some papers than peer-review; and they also apply further levels of validation, *after* peer review. These valuable new features of an open-access literature will no doubt grow and become more systematized and even institutionalized with time, but they will not replace the basic filter of peer review. They will merely enhance it. No "authority" has the time to sample and filter everything, or even most things. If they really did have nothing better to do with their time than to process all those raw, unfiltered manuscripts, one would have to begin to wonder about their competence! Never happens, though, as the "authorities" only vet a selected subset of the research glut. But that is neither here nor there for this Forum, whose main objective is freeing access to peer-reviewed research, not freeing research from peer review. Stevan Harnad