Herman Rubin <[EMAIL PROTECTED]> wrote:
> Dave J. <[EMAIL PROTECTED]> wrote:
>> Could anyone recommend a good starting point for learning more about mixture
>> models? I've had several stats classes, none of which has addressed this
>> topic, and I need to start somewhere. Thanks.
>
> A mixture model is just that; in the simplest case, a
> distribution is the mixture of distributions. The
> problems come in the very difficult job of estimating
> the components; without lots of assumptions, there
> are typically an infinite number of solutions, even
> if the exact distribution is known. It is often the
> case that even assuming that one has a mixture of a
> few distributions of a given type, the estimation
> problem is quite difficult.
I believe this is overstating the difficulty of the problem.
It is true that there may be a multitude (even an infinity) of
local maxima of the log-likelihood or other goodness of fit,
and there may be saddle points, and there may also be broad
plateaus or basins.
However, if the purpose of the exercise is to obtain a model
which fits some data well, the non-identifiability of the
parameters is beside the point. With some heuristics, it is
not too difficult to obtain a workable solution.
Permit me to wander off on a tangent for a moment. A proper
Bayesian approach to the estimation problem would average
over all values of parameters. A particularly troublesome
problem is choosing the number of components in the fitted
mixture; typically there is a broad range of the number of
components which fit more or less well. A scheme has been
invented by Geoffrey Hinton and one of his students, called
"splitting & merging expectation-maximization" (SMEM) in which
changes of the number of components is accepted or rejected by
a rule of the Metropolis type. The readers of the newsgroup
may wish to take a look.
For what it's worth,
Robert Dodier
--
``Socrates used to meditate all day in the snow, but Descartes'
mind worked only when he was warm.'' -- Bertrand Russell
=================================================================
Instructions for joining and leaving this list and remarks about
the problem of INAPPROPRIATE MESSAGES are available at
http://jse.stat.ncsu.edu/
=================================================================
