I asked Tim whether he would change the license of SuiteSparse to an Apache-friendly license couple months ago, but the answer was no. So I don't think we can use SuiteSparse in MLlib through JNI. Please feel free to create JIRAs for distributed linear programming and SOCP solvers and run the discussion there. I'm very interested since I don't really know how to do linear programming in a distributed way. -Xiangrui
On Mon, Sep 8, 2014 at 7:12 AM, Debasish Das <debasish.da...@gmail.com> wrote: > Xiangrui, > > Should I open up a JIRA for this ? > > Distributed lp/socp solver through ecos/ldl/amd ? > > I can open source it with gpl license in spark code as that's what our legal > cleared (apache + gpl becomes gpl) and figure out the right way to call > it...ecos is gpl but we can definitely use the jni version of ldl and amd > which are lgpl... > > Let me know. > > Thanks. > Deb > > On Sep 8, 2014 7:04 AM, "Debasish Das" <debasish.da...@gmail.com> wrote: >> >> Durin, >> >> I have integrated ecos with spark which uses suitesparse under the hood >> for linear equation solves....I have exposed only the qp solver api in spark >> since I was comparing ip with proximal algorithms but we can expose >> suitesparse api as well...jni is used to load up ldl amd and ecos libraries. >> >> Please follow ecos section of my spark summit talk. We can discuss more >> but we can formulate interesting things like google's ceres solver's trust >> region formulation. >> >> >> http://spark-summit.org/2014/talk/quadratic-programing-solver-for-non-negative-matrix-factorization-with-spark >> >> Let me point you to the code so that you can take a look at it. >> Suitesparse (ldl and amd) is lgpl but ecos is gpl and therefore I was not >> sure how straightforward it will be to add the solver to mllib. Our legal >> was not convinced to add lgpl/gpl code in apache project. >> >> Could you also detail the usecases you are looking for ? You want a >> distributed lp / socp solver where each worker solves a partition of the >> constraint and the full objective...and you want to converge to a global >> solution using consensus ? Or your problem has more structure to partition >> the problem cleanly and don't need consensus step (which is what I >> implemented in the code) >> >> Thanks >> Deb >> >> On Sep 7, 2014 11:35 PM, "Xiangrui Meng" <men...@gmail.com> wrote: >>> >>> You can try LinearRegression with sparse input. It converges the least >>> squares solution if the linear system is over-determined, while the >>> convergence rate depends on the condition number. Applying standard >>> scaling is popular heuristic to reduce the condition number. >>> >>> If you are interested in sparse direct methods as in SuiteSparse. I'm >>> not aware of any effort to do so. >>> >>> -Xiangrui >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
