[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554404#comment-16554404
]
Brad commented on SPARK-21097:
--
Hi [~menelaus],
I have stalled out on this project, if you would like to
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552847#comment-16552847
]
Brad commented on SPARK-21097:
--
No, I did not consider cpu utilization when redistributing the cache. It
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545240#comment-16545240
]
Brad commented on SPARK-21097:
--
Hi [~menelaus]
The processing time delay is just a way to simulate
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540195#comment-16540195
]
Brad commented on SPARK-21097:
--
Hi [~jothor],
Thanks for the interest, but to be honest I got a new job
[
https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419062#comment-16419062
]
Brad commented on SPARK-22618:
--
Yeah the fix for broadcaset unpersist should be basically the same. Thanks
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Description:
We want to use dynamic allocation to distribute resources among many notebook
users on our spark
[
https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-22618:
-
Fix Version/s: (was: 2.3.0)
> RDD.unpersist can cause fatal exception when used with dynamic allocation
>
[
https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267223#comment-16267223
]
Brad commented on SPARK-22618:
--
I have a PR for this forthcoming.
> RDD.unpersist can cause fatal exception
Brad created SPARK-22618:
Summary: RDD.unpersist can cause fatal exception when used with
dynamic allocation
Key: SPARK-22618
URL: https://issues.apache.org/jira/browse/SPARK-22618
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145708#comment-16145708
]
Brad commented on SPARK-21097:
--
Here is a document with some of my benchmark results. I am working on adding
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113264#comment-16113264
]
Brad commented on SPARK-21097:
--
I'm still working on thoroughly benchmarking and testing this change. If
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Attachment: Preserving Cached Data with Dynamic Allocation.pdf
> Dynamic allocation will preserve cached data
>
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Attachment: (was: Preserving Cached Data with Dynamic Allocation.docx)
> Dynamic allocation will preserve
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Attachment: (was: Preserving Cached Data with Dynamic Allocation.docx)
> Dynamic allocation will preserve
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Attachment: Preserving Cached Data with Dynamic Allocation.docx
> Dynamic allocation will preserve cached data
>
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad updated SPARK-21097:
-
Attachment: Preserving Cached Data with Dynamic Allocation.docx
> Dynamic allocation will preserve cached data
>
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049485#comment-16049485
]
Brad edited comment on SPARK-21097 at 6/14/17 6:29 PM:
---
Hey [~srowen], thanks for
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049485#comment-16049485
]
Brad commented on SPARK-21097:
--
Hey Sean, thanks for your input. I would definitely like to do some
[
https://issues.apache.org/jira/browse/SPARK-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16049443#comment-16049443
]
Brad commented on SPARK-21097:
--
I am working on this now and will be posting a more detailed design document
Brad created SPARK-21097:
Summary: Dynamic allocation will preserve cached data
Key: SPARK-21097
URL: https://issues.apache.org/jira/browse/SPARK-21097
Project: Spark
Issue Type: Improvement
20 matches
Mail list logo
