--- On Fri, 1/7/11, Peter Langfelder <peter.langfel...@gmail.com> wrote:
> From: Peter Langfelder <peter.langfel...@gmail.com> > Subject: Re: [R] Waaaayy off topic...Statistical methods, pub bias, > scientific validity > To: "r-help@r-project.org" <r-help@r-project.org> > Received: Friday, January 7, 2011, 2:06 AM > >From a purely statistical and > maybe somewhat naive point of view, > published p-values should be corrected for the multiple > testing that > is effectively happening because of the large number of > published > studies. My experience is also that people will often try > several > statistical methods to get the most significant p-value but > neglect to > share that fact with the audience and/or at least attempt > to correct > the p-values for the selection bias. > > That being said, it would seem that biomedical sciences do > make > progress, so some of the published results are presumably > correct :) > Totally a placebo effect :) > Peter > > On Thu, Jan 6, 2011 at 9:13 PM, Spencer Graves > <spencer.gra...@structuremonitoring.com> > wrote: > > Part of the phenomenon can be explained by the > natural censorship in > > what is accepted for publication: Stronger results > tend to have less > > difficulty getting published. Therefore, given that > a result is published, > > it is evident that the estimated magnitude of the > effect is in average > > larger than it is in reality, just by the fact that > weaker results are less > > likely to be published. A study of the literature on > this subject might > > yield an interesting and valuable estimate of the > magnitude of this > > selection bias. > > > > > > A more insidious problem, that may not affect > the work of Jonah Lehrer, > > is political corruption in the way research is funded, > with less public and > > more private funding of research > > (http://portal.unesco.org/education/en/ev.php-URL_ID=21052&URL_DO=DO_TOPIC&URL_SECTION=201.html). > > For example, I've heard claims (which I cannot > substantiate right now) that > > cell phone companies allegedly lobbied successfully to > block funding for > > researchers they thought were likely to document > health problems with their > > products. Related claims have been made by > scientists in the US Food and > > Drug Administration that certain therapies were > approved on political > > grounds in spite of substantive questions about the > validity of the research > > backing the request for approval (e.g., > > www.naturalnews.com/025298_the_FDA_scientists.html). > Some of these > > accusations of political corruption may be groundless. > However, as private > > funding replaces tax money for basic science, we must > expect an increase in > > research results that match the needs of the funding > agency while degrading > > the quality of published research. This produces > more research that can not > > be replicated -- effects that get smaller upon > replication. (My wife and I > > routinely avoid certain therapies recommended by > physicians, because the > > physicians get much of their information on recent > drugs from the > > pharmaceuticals, who have a vested interest in > presenting their products in > > the most positive light.) > > > > > > Spencer > > > > > > On 1/6/2011 2:39 PM, Carl Witthoft wrote: > >> > >> The next week's New Yorker has some decent > rebuttal letters. The case is > >> hardly as clear-cut as the author would like to > believe. > >> > >> Carl > > ______________________________________________ > R-help@r-project.org > mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, > reproducible code. > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.