Re: mllib.recommendation Design

2015-03-30 Thread Debasish Das
For alm I have started experimenting with the following: 1. rmse and map improvement from loglikelihood loss over least square loss. 2. Factorization for datasets that are not ratings (basically improvement over implicit ratings) 3. Sparse topic generation using plsa. We are directly optimizing

Re: mllib.recommendation Design

2015-03-30 Thread Xiangrui Meng
On Wed, Mar 25, 2015 at 7:59 AM, Debasish Das wrote: > Hi Xiangrui, > > I am facing some minor issues in implementing Alternating Nonlinear > Minimization as documented in this JIRA due to the ALS code being in ml > package: https://issues.apache.org/jira/browse/SPARK-6323 > > I need to use Vector

Re: mllib.recommendation Design

2015-03-25 Thread Debasish Das
Hi Xiangrui, I am facing some minor issues in implementing Alternating Nonlinear Minimization as documented in this JIRA due to the ALS code being in ml package: https://issues.apache.org/jira/browse/SPARK-6323 I need to use Vectors.fromBreeze / Vectors.toBreeze but they are package private on ml

Re: mllib.recommendation Design

2015-02-17 Thread Debasish Das
There is a usability difference...I am not sure if recommendation.ALS would like to add both userConstraint and productConstraint ? GraphLab CF for example has it and we are ready to support all the features for modest ranks where gram matrices can be made... For large ranks I am still working on

Re: mllib.recommendation Design

2015-02-17 Thread Xiangrui Meng
The current ALS implementation allow pluggable solvers for NormalEquation, where we put CholeskeySolver and NNLS solver. Please check the current implementation and let us know how your constraint solver would fit. For a general matrix factorization package, let's make a JIRA and move our discussio

mllib.recommendation Design

2015-02-13 Thread Debasish Das
Hi, I am bit confused on the mllib design in the master. I thought that core algorithms will stay in mllib and ml will define the pipelines over the core algorithm but looks like in master ALS is moved from mllib to ml... I am refactoring my PR to a factorization package and I want to build it on