[
https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14174824#comment-14174824
]
Xiangrui Meng commented on SPARK-2426:
--------------------------------------
[~debasish83] Thanks for working on this feature! This is definitely lots of
work. We need to figure out couple high-level questions before looking into the
code:
1. License. There are two files that requires special license: proximal, which
ports cvxgrp/proximal (BSD) and QPMinimizer:
{code}
... distributed with Copyright (c) 2014, Debasish Das (Verizon), all rights
reserved.
{code}
Code contribution to Apache follows ICLA:
http://www.apache.org/licenses/icla.txt . I'm not familiar with the terms. I saw
{code}
Except for the license granted herein to the Foundation and recipients of
software distributed by the Foundation, You reserve all right, title,
and interest in and to Your Contributions.
{code}
My understand is that if you want your code distributed with Apache License, we
don't need special notice about your rights. Please check with Verizon's legal
team to make sure they are okay with it. It would be really helpful If someone
can explain in more details.
2. Interface. I'm doing a refactoring of ALS (SPARK-3541). I hope we can
decouple the solvers (LS, QP) from ALS. In
https://github.com/mengxr/spark-als/blob/master/src/main/scala/org/apache/spark/ml/SimpleALS.scala
The subproblem is wrapped in a NormalEquation, which stores AtA, Atb, and n. A
Cholesky solver takes a NormalEquation instance, solves it, and returns the
solution. We can plug-in other solvers as long as NormalEquation provides all
information we need. Does it apply to your use cases?
For public APIs, we should restrict parameters to simple types. For example,
constraint = "none" | "nonnegative" | "box". This is good for adding Python
APIs. Those options should be sufficient for normal use cases. We can provide a
developer API that allows advanced users to plug-in their own solvers. You can
check the current proposal of parameters at SPARK-3530.
3. Where to put the implementation? Including MLlib's NNLS, those solvers are
for local problems. What sounds ideal to me is breeze.optimize, which already
contains several optimization solvers and we use LBFGS implemented there and
maybe OWLQN soon.
4. This PR definitely needs some time to testing. The feature freeze deadline
for v1.2 is Oct 31. I cannot promise time for code review given my current
bandwidth. It would be great if you can share your MATLAB code (hopefully
Octave compatible) and some performance results. So more developers can help
test.
> Quadratic Minimization for MLlib ALS
> ------------------------------------
>
> Key: SPARK-2426
> URL: https://issues.apache.org/jira/browse/SPARK-2426
> Project: Spark
> Issue Type: New Feature
> Components: MLlib
> Affects Versions: 1.0.0
> Reporter: Debasish Das
> Assignee: Debasish Das
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> Current ALS supports least squares and nonnegative least squares.
> I presented ADMM and IPM based Quadratic Minimization solvers to be used for
> the following ALS problems:
> 1. ALS with bounds
> 2. ALS with L1 regularization
> 3. ALS with Equality constraint and bounds
> Initial runtime comparisons are presented at Spark Summit.
> http://spark-summit.org/2014/talk/quadratic-programing-solver-for-non-negative-matrix-factorization-with-spark
> Based on Xiangrui's feedback I am currently comparing the ADMM based
> Quadratic Minimization solvers with IPM based QpSolvers and the default
> ALS/NNLS. I will keep updating the runtime comparison results.
> For integration the detailed plan is as follows:
> 1. Add QuadraticMinimizer and Proximal algorithms in mllib.optimization
> 2. Integrate QuadraticMinimizer in mllib ALS
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
