?Good idea! Join each element of c with the corresponding row of A, multiply through, then reduce. I'll give this a try.
Thanks, Alex ________________________________ From: Reza Zadeh <r...@databricks.com> Sent: Monday, January 12, 2015 3:05 PM To: Alex Minnaar Cc: u...@spark.incubator.apache.org Subject: Re: RowMatrix multiplication Yes you are correct, to do it with existing operations you would need a transpose on rowmatrix. However, you can fairly easily perform the operation manually by doing a join (if the c vector is an RDD) or broadcasting c (if the c vector is small enough to fit in memory on a single machine). On Mon, Jan 12, 2015 at 11:45 AM, Alex Minnaar <aminn...@verticalscope.com<mailto:aminn...@verticalscope.com>> wrote: That's not quite what I'm looking for. Let me provide an example. I have a rowmatrix A that is nxm and I have two local matrices b and c. b is mx1 and c is nx1. In my spark job I wish to perform the following two computations A*b and A^T*c I don't think this is possible without being able to transpose a rowmatrix. Am I correct? Thanks, Alex ________________________________ From: Reza Zadeh <r...@databricks.com<mailto:r...@databricks.com>> Sent: Monday, January 12, 2015 1:58 PM To: Alex Minnaar Cc: u...@spark.incubator.apache.org<mailto:u...@spark.incubator.apache.org> Subject: Re: RowMatrix multiplication As you mentioned, you can perform A * b, where A is a rowmatrix and b is a local matrix. >From your email, I figure you want to compute b * A^T. To do this, you can >compute C = A b^T, whose result is the transpose of what you were looking for, >i.e. C^T = b * A^T. To undo the transpose, you would have transpose C manually >yourself. Be careful though, because the result might not have each Row fit in >memory on a single machine, which is what RowMatrix requires. This danger is >why we didn't provide a transpose operation in RowMatrix natively. To address this and more, there is an effort to provide more comprehensive linear algebra through block matrices, which will likely make it to 1.3: https://issues.apache.org/jira/browse/SPARK-3434 Best, Reza On Mon, Jan 12, 2015 at 6:33 AM, Alex Minnaar <aminn...@verticalscope.com<mailto:aminn...@verticalscope.com>> wrote: I have a rowMatrix on which I want to perform two multiplications. The first is a right multiplication with a local matrix which is fine. But after that I also wish to right multiply the transpose of my rowMatrix with a different local matrix. I understand that there is no functionality to transpose a rowMatrix at this time but I was wondering if anyone could suggest a any kind of work-around for this. I had thought that I might be able to initially create two rowMatrices - a normal version and a transposed version - and use either when appropriate. Can anyone think of another alternative? Thanks, Alex
