Re: Dirchlet

[Ted Dunning]

On Thu, Nov 3, 2011 at 2:12 PM, Grant Ingersoll <gsing...@apache.org> wrote:

> ...
> So, to deploy a Combiner, we need to understand what type of distribution
> we are dealing with (which we do know, but may need a marker interface or
> something).  Then, if is an "Combinable" distribution, it can do above
> (which I admittedly need to work through a bit more)?  Do we have Welford
> implemented in our math package?
>

Yes.  A marker would be required.

The key method would be add(Model).

We do have a Welford implementation for one dimensional estimates
in org.apache.mahout.math.stats.OnlineSummarizer

Rough pseudo code would be really helpful as to what the Combiner might
> look like.  I'll worry about the math later.
>

Here is a sketch:

   public class ModelCombiner extends Reducer<Integer, Model> {
      void reduce(Integer key, Iterable<Model> values, Context ctx) {
          Model m = null;
          for (Model x : values) {
               if (m == null) {
                  m = x;
               else {
                  m.add(x);
               }
          }
          ctx.write(key, m);
       }
    }