*** For details on how to be removed from this list visit the ***
*** CCP4 home page http://www.ccp4.ac.uk ***
You guys are starting to make me feel funny.
Kendall
On Aug 31, 2005, at 9:06 AM, Douglas Theobald wrote:
*** For details on how to be removed from this list visit the ***
*** CCP4 home page http://www.ccp4.ac.uk ***
On 8/31/05 4:45 AM, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> a
écrit :
You are also using a non-traditional definition of 'statistical
bias'. The
Gauss-Markov theorem guarantees that if condition (1) above holds
[even if
(2) and (3) are violated], then the least-squares solution is
unbiased. Here
I am using the standard def of 'bias' in statistics:
I'm using the same definition - more or less ;o) Only from a Bayesian
viewpoint, what corresponds to a 'point estimate' would be the
expectation
value of the posterior probability for which you, of course, need the
distribution.
Well, I can see how there is a Bayesian analog of 'bias' (though I
have
never seen a precise def), but it is still a very different concept
from the
frequentist usage of 'bias', which is meaningless for a Bayesian.
If you integrate over a different distribution, you'll often get a
different estimate. If you only _use_ the mean and the variance,
you are
in effect committing yourself to a normal distribution.
Ok. We have to separate two issues. One is the set of assumptions
laid out
by a traditional LS derivation and the other is the 'biased view'
that
Bayesian Theory is the only reasonable scientific inference
machine. You
are right that LS does not explicitly make any assumptions about
the error
distribution, however if you analyse standard ad hoc traditional
techniques from the Information Theory and Bayesian point of view
(which
can be done with some care) you often reveal the real hidden
underlying
assumptions on which you base your inference.
OK -- I was not intending to defend the frequentist position (I
actually
care little about bias), but rather simply explain the traditional
justification for LS in terms of the G-M Theorem, which does not
depend on a
Gaussian error distribution.
However, and this is really a subtle point, I still disagree that a
Bayesian
analysis reveals a hidden normality assumption for least squares.
From a
Bayesian and/or Info Theoretic POV, if you have only the first two
moments,
you are best off acting as if the distribution is normal. That is
different
from *assuming* that it is normal. If your analysis is based on a
normality
assumption, it is valid only to the extent that the normality
assumption is
valid. In contrast, a Bayesian is saying that it doesn't matter
what the
real distribution is, normal or not, you are best off using a normal
distribution in your calculations, because a normal distribution
assumes the
least structure (or information) in your data given only the first two
moments. Use of any other distribution would involve extra unwarranted
assumptions.
