[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017438#comment-17017438
]
Samuel Shepard commented on SPARK-6235:
---
[~irashid] I followed your suggestion of looking in the
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16998577#comment-16998577
]
Imran Rashid commented on SPARK-6235:
-
[~sammysheep] Spark's ML library uses the same jira project,
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997798#comment-16997798
]
Samuel Shepard commented on SPARK-6235:
---
[~irashid] I meant the former (task result > 2G) as best I
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997631#comment-16997631
]
Imran Rashid commented on SPARK-6235:
-
[~sammysheep] are you discussing the use case for task results
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995889#comment-16995889
]
Samuel Shepard commented on SPARK-6235:
---
One use case could be fetching large results to the driver
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820309#comment-16820309
]
Imran Rashid commented on SPARK-6235:
-
[~glal14] actually this was fixed in 2.4. There was one open
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820229#comment-16820229
]
Gowtam Lal commented on SPARK-6235:
---
It would be great to see this go out. Any updates?
> Address
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484069#comment-16484069
]
Imran Rashid commented on SPARK-6235:
-
Would be nice to find a better home for this, but for now I
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482901#comment-16482901
]
Imran Rashid commented on SPARK-6235:
-
[~tgraves]
WAL -- write-ahead-log for receiver-based streaming.
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482746#comment-16482746
]
Thomas Graves commented on SPARK-6235:
--
>> Still unsupported:
* large task results
* large blocks
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482699#comment-16482699
]
Imran Rashid commented on SPARK-6235:
-
Linked a [design
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477986#comment-16477986
]
Imran Rashid commented on SPARK-6235:
-
derp, I missed a pretty basic case -- if you cache a large
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477827#comment-16477827
]
Imran Rashid commented on SPARK-6235:
-
I've been testing the current state of the 2GB block limit, and
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469991#comment-16469991
]
Cyanny commented on SPARK-6235:
---
Hi, when will this Jira feature included in a spark releas? [~rxin]
>
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009808#comment-16009808
]
Jamie Hutton commented on SPARK-6235:
-
Hi there, Is there any update on when this will be included in
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15515042#comment-15515042
]
Guoqiang Li commented on SPARK-6235:
ping [~rxin]
> Address various 2G limits
>
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508670#comment-15508670
]
Guoqiang Li commented on SPARK-6235:
[~rxin] Any comments?
> Address various 2G limits
>
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470410#comment-15470410
]
Apache Spark commented on SPARK-6235:
-
User 'witgo' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15467563#comment-15467563
]
Apache Spark commented on SPARK-6235:
-
User 'witgo' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421048#comment-15421048
]
Guoqiang Li commented on SPARK-6235:
Yes, it contains a lot of minor changes, eg: Replace ByteBuffer
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421018#comment-15421018
]
Apache Spark commented on SPARK-6235:
-
User 'witgo' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420746#comment-15420746
]
Sean Owen commented on SPARK-6235:
--
How does this relate to the existing subtasks and their
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15420554#comment-15420554
]
Guoqiang Li commented on SPARK-6235:
[~hvanhovell]
The main changes.
1. Replace DiskStore method
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15419151#comment-15419151
]
Herman van Hovell commented on SPARK-6235:
--
[~gq] it might be a good idea to share some design
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418225#comment-15418225
]
Guoqiang Li commented on SPARK-6235:
I'm doing this work and I'll put the patch in this month.
>
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15413055#comment-15413055
]
Sean Owen commented on SPARK-6235:
--
I think the short answer is, it's very hard. I am not sure it's
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412727#comment-15412727
]
Brian commented on SPARK-6235:
--
How is it possible that Spark 2.0 comes and and this bug isn't solved? A
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957546#comment-14957546
]
Reynold Xin commented on SPARK-6235:
Is your data skewed? i.e. maybe there is a single key that's
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957554#comment-14957554
]
Glenn Strycker commented on SPARK-6235:
---
I don't think so, but I can check. My RDD came from an RDD
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744746#comment-14744746
]
Sean McKibben commented on SPARK-6235:
--
When reading from HBase into spark, the regions seem to
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14744605#comment-14744605
]
Ram Gande commented on SPARK-6235:
--
Any progress on this. We are seeing this issue constantly in our
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724609#comment-14724609
]
Zhen Peng commented on SPARK-6235:
--
Hi [~rxin], is there any update for this issue?
> Address various 2G
32 matches
Mail list logo