[
https://issues.apache.org/jira/browse/SPARK-21049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045536#comment-16045536
]
Vincent commented on SPARK-21049:
---------------------------------
[~srowen]thanks. that's right. But we found it quite often that, the matrix is
not skinny, and it spent quite a lot of time computing gramian matrix.
Actually, we found that in such case, if we compute the svd on the original
matrix, we could at least have 5x+ speedup. So, I wonder, whether it's possible
to add an option here, to offer the user a choice to choose whether go with
gramian or the original matrix. After all, user knows their data better, what
do u think?
> why do we need computeGramianMatrix when computing SVD
> ------------------------------------------------------
>
> Key: SPARK-21049
> URL: https://issues.apache.org/jira/browse/SPARK-21049
> Project: Spark
> Issue Type: Improvement
> Components: ML, MLlib
> Affects Versions: 2.1.1
> Reporter: Vincent
>
> computeSVD will compute SVD for matrix A by computing AT*A first and svd on
> the Gramian matrix, we found that the gramian matrix computation is the hot
> spot of the overall SVD computation, but, per my understanding, we can simply
> do svd on the original matrix. The singular vector of the gramian matrix
> should be the same as the right singular vector of the original matrix A,
> while the singular value of the gramian matrix is double as that of the
> original matrix. why do we svd on the gramian matrix then?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
