Rolf Turner wrote:
I have been building an R function to calculate the ***observed***
(as opposed to expected) Fisher information matrix for parameter
estimates in a rather complicated setting. I thought I had it
working, but I am getting a result which is not positive definite.
(One negative eigenvalue. Out of 10.)
Is it the case that the observed Fisher information must be positive
definite --- thereby indicating for certain that there are errors in
my code --- or is it possible for such a matrix not to be pos. def.?
If you are at the maximum, it should be at least positive indefinite (or
nonnegative definite). Numerical errors could make zero (or small
positive) eigenvalues look negative. It's also possible that your
optimization has missed the maximum by a bit, and then it could have
truly negative eigenvalues.
In either case I'd expect the negative eigenvalues to be small.
It seems to me that if the log likelihood surface is ***not*** well
approximated by a quadratic in a neighbourhood of the maximum, then
it might well be that case that the observed information could fail
to be positive definite. Is this known/understood? Can anyone point
me to appropriate places in the literature?
If there is a true negative eigenvalue, then moving along that
eigenvector should increase the likelihood, so I don't think even
irregular problems could have true negative eigenvalues at the MLE. The
problem there would be that a zero score and a positive definite
observed information matrix don't necessarily imply you're at even a
local maximum.
Duncan Murdoch
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
