Re: Solving Systems of Linear Equations Using Spark?

2014-09-08 Thread Xiangrui Meng
You can try LinearRegression with sparse input. It converges the least squares solution if the linear system is over-determined, while the convergence rate depends on the condition number. Applying standard scaling is popular heuristic to reduce the condition number. If you are interested in

Re: Solving Systems of Linear Equations Using Spark?

2014-09-08 Thread Debasish Das
Durin, I have integrated ecos with spark which uses suitesparse under the hood for linear equation solvesI have exposed only the qp solver api in spark since I was comparing ip with proximal algorithms but we can expose suitesparse api as well...jni is used to load up ldl amd and ecos

Re: Solving Systems of Linear Equations Using Spark?

2014-09-08 Thread Debasish Das
Xiangrui, Should I open up a JIRA for this ? Distributed lp/socp solver through ecos/ldl/amd ? I can open source it with gpl license in spark code as that's what our legal cleared (apache + gpl becomes gpl) and figure out the right way to call it...ecos is gpl but we can definitely use the jni

Re: Solving Systems of Linear Equations Using Spark?

2014-09-08 Thread Xiangrui Meng
I asked Tim whether he would change the license of SuiteSparse to an Apache-friendly license couple months ago, but the answer was no. So I don't think we can use SuiteSparse in MLlib through JNI. Please feel free to create JIRAs for distributed linear programming and SOCP solvers and run the

Re: Solving Systems of Linear Equations Using Spark?

2014-09-08 Thread Debasish Das
Yup...this can be a spark community project...I saw a PR for that...interested users fine with lgpl/gpl code can make use of it... On Mon, Sep 8, 2014 at 12:37 PM, Xiangrui Meng men...@gmail.com wrote: I asked Tim whether he would change the license of SuiteSparse to an Apache-friendly license

Solving Systems of Linear Equations Using Spark?

2014-09-07 Thread durin
such an algorithm for Spark? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Solving-Systems-of-Linear-Equations-Using-Spark-tp13674.html Sent from the Apache Spark User List mailing list archive at Nabble.com