[
https://issues.apache.org/jira/browse/SPARK-12869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15107533#comment-15107533
]
Fokko Driesprong commented on SPARK-12869:
------------------------------------------
Hi guys,
I've implemented an improved version of the toIndexedRowMatrix function on the
BlockMatrix. I needed this for a project, but would like to share it with the
rest of the community. In the case of dense matrices, it can increase
performance up to 19 times:
https://github.com/Fokko/BlockMatrixToIndexedRowMatrix
The pull-request on Github:
https://github.com/apache/spark/pull/10839
> Optimize conversion from BlockMatrix to IndexedRowMatrix
> --------------------------------------------------------
>
> Key: SPARK-12869
> URL: https://issues.apache.org/jira/browse/SPARK-12869
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: Fokko Driesprong
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> In the current implementation of the BlockMatrix, the conversion to the
> IndexedRowMatrix is done by converting it to a CoordinateMatrix first. This
> is somewhat ok when the matrix is very sparse, but for dense matrices this is
> very inefficient.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
