[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973531#comment-14973531
]
Jerry Lam commented on SPARK-8597:
--
FYI ... The solution described here solves the proble
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599977#comment-14599977
]
Matt Cheah commented on SPARK-8597:
---
I've attached the CSV file used in the test.
> Dat
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601687#comment-14601687
]
Matt Cheah commented on SPARK-8597:
---
I did some more digging. The memory space is taken
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602009#comment-14602009
]
Michael Armbrust commented on SPARK-8597:
-
Parquet allocates fairly large buffers
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14603816#comment-14603816
]
Matt Cheah commented on SPARK-8597:
---
Cool, a coworker and I think we have something simi
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606199#comment-14606199
]
Vlad Ionescu commented on SPARK-8597:
-
Hi,
Regarding the third suggestion, the functi
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606248#comment-14606248
]
Michael Armbrust commented on SPARK-8597:
-
We could use Spark's external sort whic
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606554#comment-14606554
]
Vlad Ionescu commented on SPARK-8597:
-
Actually I've used an ExternalAppendOnlyMap to
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606561#comment-14606561
]
Matt Cheah commented on SPARK-8597:
---
I'm also concerned about the possibility that using
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608791#comment-14608791
]
Vlad Ionescu commented on SPARK-8597:
-
I did some stress tests, the main purpose was t
[
https://issues.apache.org/jira/browse/SPARK-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608806#comment-14608806
]
Reynold Xin commented on SPARK-8597:
We are implementing a Tungsten version of externa
11 matches
Mail list logo
