[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095232#comment-14095232 ]
Debasish Das commented on SPARK-2426: ------------------------------------- Hi Xiangrui, The branch is ready for an initial review. I will do lot of clean-up this week. https://github.com/debasish83/spark/commits/qp-als optimization/QuadraticMinimizer.scala is the placeholder for all QuadraticMinimization. Right now we support 5 features: 1. Least square 2. Least square with positivity 3. Least square with bounds : generalization of positivity 4. Least square with equality and positivity/bounds for LDA/PLSA 5. Least square + L1 constraint for sparse NMF There are lot many regularization in Proximal.scala which can be re-used in mllib updater...L1Updater is an example of Proximal algorithm. I feel we should move NNLS into QuadraticMinimizer as well and clean ALS.scala as you have suggested before... QuadraticMinimizer is optimized for direct solve right now (cholesky / lu based on problem we are solving) The CG core from NNLS should be used for iterative solve when ranks are high...I need a different variant of CG for Formulation 4 so NNLS CG is not sufficient for all the formulations. Right now I am experimenting with ADMM rho and lambda values so that the NNLS iterations are at par with Least square with positivity. I will publish results from the comparisons. I will also publish comparisons with PDCO, ECOS (IPM) and MOSEK with ADMM variants used in this branch... For recommendation use-case, I expect to produce Jellylish L1 ball projection results on netflix/movielens dataset using Formulation 5. Thanks. Deb > Quadratic Minimization for MLlib ALS > ------------------------------------ > > Key: SPARK-2426 > URL: https://issues.apache.org/jira/browse/SPARK-2426 > Project: Spark > Issue Type: New Feature > Components: MLlib > Affects Versions: 1.0.0 > Reporter: Debasish Das > Assignee: Debasish Das > Original Estimate: 504h > Remaining Estimate: 504h > > Current ALS supports least squares and nonnegative least squares. > I presented ADMM and IPM based Quadratic Minimization solvers to be used for > the following ALS problems: > 1. ALS with bounds > 2. ALS with L1 regularization > 3. ALS with Equality constraint and bounds > Initial runtime comparisons are presented at Spark Summit. > http://spark-summit.org/2014/talk/quadratic-programing-solver-for-non-negative-matrix-factorization-with-spark > Based on Xiangrui's feedback I am currently comparing the ADMM based > Quadratic Minimization solvers with IPM based QpSolvers and the default > ALS/NNLS. I will keep updating the runtime comparison results. > For integration the detailed plan is as follows: > 1. Add ADMM and IPM based QuadraticMinimization solvers to > breeze.optimize.quadratic package. > 2. Add a QpSolver object in spark mllib optimization which calls breeze > 3. Add the QpSolver object in spark mllib ALS -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org