*** For details on how to be removed from this list visit the *** *** CCP4 home page http://www.ccp4.ac.uk ***
On 9/2/05 1:43 PM, "Ian Tickle" <[EMAIL PROTECTED]> a écrit : >> -----Original Message----- From: [EMAIL PROTECTED] >> [mailto:[EMAIL PROTECTED] On Behalf Of >> [EMAIL PROTECTED] Sent: 30 August 2005 22:58 To: Douglas >> Theobald Cc: [email protected] Subject: Re: [ccp4bb]: maximum >> likelihood question > >> Yes, but given an expectation value and a variance, Maximum >> Entropy methods say that the only unbiased error distribution >> is Gaussian. So, although it's never explicitly stated in >> Least-Squares, the fact that you consider only an expectation >> and a variance (and the lack of correlation), in a sense, >> actually already nails down the error model (if you want to >> remain unbiased, that is). If the errors were non-Normal then >> you can still apply the method and get good results, but I'm >> not sure that they are optimal, they don't have the highest >> likelihood and are certainly not the most probable. > > I couldn't let this pass without commenting that the last > sentence needs to be qualified: it's certainly true if the > Gauss-Markov conditions (i.e. fixed weight matrix proportional > to inverse of var-covar matrix of observations) are satisfied > (and we're assuming throughout that the errors are > non-normal). However there's nothing in the G-M theorem which > says that the G-M conditions have to apply! It just means > that if they don't then the resultant parameters don't have > the properties of unbiasedness and minimum variance (that is, > minimum w.r.t. variations in the weights) that are guaranteed > by the theorem if the conditions do apply. There seems to be > a widespread belief that you can't vary the weights in LS: you > can, it's just that you don't get unbiased minimum variance > results, I'm not sure what you mean here -- if you weight by the precision (i.e. the inverse of the variance), then you certainly *do* get unbiased, minimum variance results. This is pretty standard practice for heteroskedastic data. > but then these properties are not necessarily desirable anyway > (e.g. see article on bias in Wikipedia: > http://en.wikipedia.org/wiki/Bias_%28statistics%29# > The_sometimes-good_ki nd ). > > Under specified conditions in which the weights used in a LS > iteration are functionally dependent on the parameters from > the previous iteration (unless the errors are normal, in which > case there is no dependence and the weights remain fixed), and > under fairly general conditions on the form of the error > distribution (which you have to know), the LS iterations in > fact converge to the ML solution. Again, I'm not sure exactly what you mean. If the errors are normal, but heteroskedastic, and you weight by the precision, then you get the ML solution. But *only* if the errors are normal. When doing IRLS, especially assuming normal errors, one usually calculates the variance based on the residuals calculated from the model from the previous iteration. > So in that case LS certainly does give the highest likelihood > (i.e. identical to ML). Only for normal errors. > This is the basis of the IRLS (Iteratively Reweighted Least > Squares) algorithm widely used in GLM (Generalised Linear > Modelling). Actually this is really ML because you have to > know the form of the error distribution, it's just that the > mechanics for forming and solving the equations for the > parameter updates is pure LS (or Newton-Raphson to be precise). This is not really true. Weighting by the inverse of the variance with arbitrary, non-normal errors is *not* ML but it *will* satisfy the G-M conditions (assuming the expected error is 0, of course). > Also it seems to me that nowhere here is there any assumption > (implicit or otherwise) of normal errors, in fact quite the > contrary: there's an explicit assumption of non-normal errors! Again, I disagree -- you don't have to assume any form for the error distribution to justify the use of LS. The main thing you need is have zero expectation for the errors. > I raised this point originally because I was trying to answer > the question: what happens if you know for sure that the > errors are non-normal but you don't know the form of the PDF > sufficiently accurately to do ML? It seemed to me that to say > that LS assumes a normal distribution is not helpful in such a > situation. This is not a theoretical question: e.g. I know > for sure that B factors don't have a normal distribution (if > the measured B is 10, the probability that the true B is <=0 > is precisely zero, but Pr(B>=20) is clearly finite, so the > distribution is strongly skewed & can't be normal). However I > don't have the faintest idea that the error distribution is! A few good candidates would be the log-normal, the inverse gaussian, and the inverse gamma. I would go with the latter -- it is the standard Bayesian proper prior for a variance (so should immediately apply to B-factors). You could easily do a ML fit and see what the chi-squared is like. If you would like to send me a file (any text format) containing the B-factors you're interested in I can do a very quick analysis to see which distribution they fit to best. Cheers, Douglas
