[
https://issues.apache.org/jira/browse/SPARK-13196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133548#comment-15133548
]
yuhao yang commented on SPARK-13196:
------------------------------------
https://github.com/apache/spark/pull/11078
> Optimize the option and flatten in Word2Vec to reduce the max memory
> consumption
> --------------------------------------------------------------------------------
>
> Key: SPARK-13196
> URL: https://issues.apache.org/jira/browse/SPARK-13196
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Reporter: yuhao yang
> Priority: Minor
>
> code optimization.
> 1. remove the unnecessary Option and flatten.
> 2. avoid the extra space caused by the ++ operation.
> This would reduce the maximum memory word2vec requires, thus to support
> larger dataset or vocabulary for limited memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
