[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995889#comment-16995889
]
Samuel Shepard edited comment on SPARK-6235 at 12/16/19 4:39 PM:
-----------------------------------------------------------------
[~tgraves] , [~irashid]
One use case could be fetching large results to the driver when computing PCA
on large square matrices (e.g., distance matrices, similar to Classical MDS).
This is very helpful in bioinformatics. Sorry if this already fixed past
2.4.0...
was (Author: sammysheep):
One use case could be fetching large results to the driver when computing PCA
on large square matrices (e.g., distance matrices, similar to Classical MDS).
This is very helpful in bioinformatics. Sorry if this already fixed past
2.4.0...
> Address various 2G limits
> -------------------------
>
> Key: SPARK-6235
> URL: https://issues.apache.org/jira/browse/SPARK-6235
> Project: Spark
> Issue Type: Umbrella
> Components: Shuffle, Spark Core
> Reporter: Reynold Xin
> Priority: Major
> Fix For: 2.4.0
>
> Attachments: SPARK-6235_Design_V0.02.pdf
>
>
> An umbrella ticket to track the various 2G limit we have in Spark, due to the
> use of byte arrays and ByteBuffers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
