Dear Colleagues,
Since I converted to Bayesian statistics in the late 1980's, I have not
looked at most frequentist methods. However, every time I look at them
again, I notice how apparently preposterous many of them are.
First that was the Bonferroni correction, which makes me update my
belief about the results of an experiment based on how many other
experiments I happen to conduct with it (and which of course implicitly
assigns a low prior probability). One researcher even told me once that
he has students first conduct fewer experiments so a finding has a
better chance of being significant. I just walked away scratching my head.
Now, in the process of designing a small test for a student, I noticed
that two-tailed hypothesis testing is completely unreasonable. Along
with the one-tailed test, it gives me decision rules which enable me to
reject the hypothesis that the mean is less than or equal to 0, but not
reject the hypothesis that it equals 0. The explanation is wrapped up in
a story about the question asked and long run behavior with other
similar experiments, that are not even run. So two people can walk away
from the same experiment with different updated beliefs about whether
the mean is 0, not based on their prior beliefs, but based on the
question they happened to ask. In general, hypothesis testing does not
seem to be the way to go. We should simply compute confidence intervals
or posterior probability intervals.
The Bayesian's world is so much simpler. She updates her belief solely
on her prior beliefs and the data. There is no story that leads to
strange results.
All this matters, especially in medical applications, because so many
studies are deemed significant or not significant based on the enigmatic
p-value and the Bonferroni correction. I like to say that in medicine
for every study there is an equal and opposite study.
I am writing this because I wonder who else has noticed these oddities?
I never read about them. I simply observed them independently. I find it
curious that they have persisted for so long, and more is not said about
them.
Best,
Rich
--
Richard E. Neapolitan, Ph.D., Professor
Division of Health and Biomedical Informatics
Department of Preventive Medicine
Northwestern University Feinberg School of Medicine
750 N. Lake Shore Drive, 11th floor
Chicago IL 60611
_______________________________________________
uai mailing list
uai@ENGR.ORST.EDU
https://secure.engr.oregonstate.edu/mailman/listinfo/uai
