Let me try to clarify:
Computational Linear Algebra is an incredibly
well established field.
Now we have M/R as a real option for day-to-day
work.
But I could imagine that the best way to use M/R
to do computational Linear Algebra might not
consist of "naive" adaptions of classical algorithms
to M/R, but in the development of different algos,
that take advantage of M/R, or in the rediscovery
of existing algos that are of little importance in
classical applications, but that all of a sudden
make a lot of sense in a M/R context.
So, I am wondering whether people in the numerical
analysis community have started to think in this
direction and what they have come up with so far
(if anything).
This is not a question about Mahout, and I admit that
it is therefore a little OT for this list, on the other hand,
I would assume that the members of this list would be
the first people to know about progress in this area...
Best,
Ph.
On Thursday 14 January 2010 10:26:17 pm Ted Dunning wrote:
> On Thu, Jan 14, 2010 at 10:09 PM, Philipp K. Janert <[email protected]> wrote:
> > > If you mean matrix factorization, take a look at this:
> > > http://arxiv.org/abs/0909.4061v1
> >
> > That seems to support my earlier hunch that
> > efficient implementations of such factorizations
> > on M/R would likely be approximate only or
> > partial (ie yielding the largest of the eigenvalues,
> > not necessarily the entire spectrum).
>
> For very large sparse problems, approximate decompositions are generally
> preferred. Due to limited accuracy in the input, only the first several
> eigenvectors can be extracted at all. Moreover, many important problems
> have very large apparent dimensionality, but limited actual rank. Neither
> of these characteristics is a characteristic of map-reduce in the
> slightest.
>
> > It is the common sense of those on this mailing list that these kinds of
> >
> > > algorithms could be done using map-reduce.
> >
> > I am not sure what you are trying to tell me here.
>
> I am trying to say that we don't yet have working implementations. There
> should be a k-means implementation that use these techniques before long.
> You would be very welcome to try your hand at some other of the algorithms
> and I am sure that you would have quite a lot of support from the mailing
> list.
>
> You comments puzzle me, though. Do you have an application in mind? Was
> there something you were particularly looking for?
