Tamas, See https://issues.apache.org/jira/browse/MAHOUT-376 as well. I have been using these techniques in R and want to push them into Mahout over the next few months for more scalability.
The only issues with the current distributed Lanczos solver is storage for the auxiliary matrices as they are produced. Jake intimated that the had a solution for that that wasn't prime-time yet. On Thu, May 6, 2010 at 12:20 PM, Jake Mannix <[email protected]> wrote: > Tamas, > > MAHOUT-371 will be able to leverage the existing DistributedLanczosSolver > and DistributedRowMatrix (in o.a.m.math.decomposer.hadoop package in core) > to do full sparse truncated SVD on the entire user-item matrix already, so > that part is taken care of. > > -jake > > On Thu, May 6, 2010 at 11:38 AM, Tamas Jambor <[email protected] > >wrote: > > > that looks interesting, but quite general. I'd be interested to know how > he > > plans to divide the task that will be distributed. I mean SVD in general > > takes the whole user-item matrix, so it will be challenging to find a > good > > way to divide the task. Papers written on SVD do not discuss this aspect, > as > > far as I know. > > > > > > On 06/05/2010 18:32, Sean Owen wrote: > > > >> We're lucky to have a GSoC student implementing this over the summer: > >> https://issues.apache.org/jira/browse/MAHOUT-371 > >> > >> On Thu, May 6, 2010 at 6:28 PM, Tamas Jambor<[email protected]> > >> wrote: > >> > >> > >>> I am looking into the problem of distributed SVD for recommender > systems. > >>> does anyone know whether someone else tried to tackle this problem > >>> before? > >>> > >>> > >>> > >> >
