Yes, you can only use RowMatrix.multiply() within the driver. We are working on distributed block matrices and linear algebra operations on top of it, which would fit your use cases well. It may take several PRs to finish. You can find the first one here: https://github.com/apache/spark/pull/3200 -Xiangrui
On Wed, Jan 14, 2015 at 2:06 PM, Alex Minnaar <aminn...@verticalscope.com> wrote: > I am working with a RowMatrix and I noticed in the multiply() method that > the local matrix with which it is being multiplied is being distributed to > all of the rows of the RowMatrix. If this is the case, then is it > impossible to multiply a row matrix within a map operation? Because this > would essentially be creating RDDs within RDDs. For example, If you had an > RDD of local matrices and you wanted to perform a map operation where each > local matrix is multiplied with a distributed matrix. This does not seem > possible since it would require distributing each local matrix in the map > when multiplication occurs (i.e. creating an RDD in each element of the > original RDD). If this is true then does it mean you can only multiply a > RowMatrix within the driver i.e. you cannot parallelize RowMatrix > multiplications? > > > Thanks, > > > Alex --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
