Re: Which linear algebra interface to use within Spark MLlib?

Hi Burak, For local linear algebra package why are we not extending breeze ? Breeze is a mllib dependency...Also that way the local linear algebra package will be used by other scala based frontend APIs as well that do not necessarily pull in Spark dependencies... Thanks. Deb On Fri, Mar 20, 2

Re: Which linear algebra interface to use within Spark MLlib?

Hi, We plan to add a more comprehensive local linear algebra package for MLlib 1.4. This local linear algebra package can then easily be extended to BlockMatrix to support the same operations in a distributed fashion. You may find the JIRA to track this here: SPARK-6442

Re: Which linear algebra interface to use within Spark MLlib?

Yeah it will be better if we consolidate the development on one of them...either Breeze or mllib.BLAS... On Thu, Mar 19, 2015 at 2:25 PM, Ulanov, Alexander wrote: > Thanks for quick response. > > I can use linealg.BLAS.gemm, and this means that I have to use MLlib > Matrix. The latter does not

Re: Which linear algebra interface to use within Spark MLlib?

Thanks for quick response. I can use linealg.BLAS.gemm, and this means that I have to use MLlib Matrix. The latter does not support some useful functionality needed for optimization. For example, creation of Matrix given matrix size, array and offset in this array. This means that I will need t

Re: Which linear algebra interface to use within Spark MLlib?

I think for Breeze we are focused on dot and dgemv right now (along with several other matrix vector style operations)... For dgemm it is tricky since you need to do add dgemm for both DenseMatrix and CSCMatrix...and for CSCMatrix you need to get something like SuiteSparse which is under lgpl...so

Re: Which linear algebra interface to use within Spark MLlib?

Thank you! When do you expect to have gemm in Breeze and that version of Breeze to ship with MLlib? Also, could someone please elaborate on the linalg.BLAS and Matrix? Are they going to be developed further, should in long term all developers use them? Best regards, Alexander 18.03.2015, в 23:

Re: Which linear algebra interface to use within Spark MLlib?

dgemm dgemv and dot come to Breeze and Spark through netlib-java Right now both in dot and dgemv Breeze does a extra memory allocate but we already found the issue and we are working on adding a common trait that will provide a sink operation (basically memory will be allocated by user)...addi

Which linear algebra interface to use within Spark MLlib?

Hi, Currently I am using Breeze within Spark MLlib for linear algebra. I would like to reuse previously allocated matrices for storing the result of matrices multiplication, i.e. I need to use "gemm" function C:=q*A*B+p*C, which is missing in Breeze (Breeze automatically allocates a new matrix