[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288353#comment-16288353
]
Sean Owen commented on SPARK-22765:
---
I'm not clear why this needs another allocation sc
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288414#comment-16288414
]
Xuefu Zhang commented on SPARK-22765:
-
I wouldn't say that MR is static, at lease not
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289613#comment-16289613
]
Thomas Graves commented on SPARK-22765:
---
why doesn't idle timeout very small < 5 se
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289676#comment-16289676
]
Xuefu Zhang commented on SPARK-22765:
-
Hi [~tgraves], Thanks for your input.
In our
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289700#comment-16289700
]
Thomas Graves commented on SPARK-22765:
---
ok so its basically they idle timeout duri
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289726#comment-16289726
]
Xuefu Zhang commented on SPARK-22765:
-
Yes, we are using Hive on Spark. Our Spark ver
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289793#comment-16289793
]
Thomas Graves commented on SPARK-22765:
---
ok so before you do anything else I would
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292048#comment-16292048
]
Xuefu Zhang commented on SPARK-22765:
-
[~tgraves], I think it would help if SPARK-216
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16292594#comment-16292594
]
Thomas Graves commented on SPARK-22765:
---
I'm not sure how mr style and 4-core execu
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295814#comment-16295814
]
Xuefu Zhang commented on SPARK-22765:
-
As an update, I managed to backport SPARK-2165
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295958#comment-16295958
]
Nan Zhu commented on SPARK-22765:
-
[~xuefuz] Regarding this, "The symptom is that newly a
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296811#comment-16296811
]
Thomas Graves commented on SPARK-22765:
---
[~CodingCat] with SPARK-21656 executors sh
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297276#comment-16297276
]
Xuefu Zhang commented on SPARK-22765:
-
Hi [~CodingCat], yes, we adapted both #1 and #
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297307#comment-16297307
]
Thomas Graves commented on SPARK-22765:
---
yes between stages becomes a problem with
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297316#comment-16297316
]
Xuefu Zhang commented on SPARK-22765:
-
Actually I meant across parallel stages (those
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297335#comment-16297335
]
Thomas Graves commented on SPARK-22765:
---
SPARK-21656 and the dynamic allocation sho
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297346#comment-16297346
]
Xuefu Zhang commented on SPARK-22765:
-
I'm not 100% positive, but that seems to be wh
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297453#comment-16297453
]
Nan Zhu commented on SPARK-22765:
-
I took a look at the code, one of the possibilities is
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297529#comment-16297529
]
Xuefu Zhang commented on SPARK-22765:
-
{quote}
SPARK-21656 and the dynamic allocation
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297549#comment-16297549
]
Thomas Graves commented on SPARK-22765:
---
with SPARK-21656 does upfront allocation v
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297645#comment-16297645
]
Xuefu Zhang commented on SPARK-22765:
-
Haven't got a chance to try upfront allocation
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297914#comment-16297914
]
Xuefu Zhang commented on SPARK-22765:
-
Alright, I tested upfront allocation and its c
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16298589#comment-16298589
]
Thomas Graves commented on SPARK-22765:
---
ok so at least based on this one experimen
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16298634#comment-16298634
]
Xuefu Zhang commented on SPARK-22765:
-
bq: at least based on this one experiment up
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16300926#comment-16300926
]
Xuefu Zhang commented on SPARK-22765:
-
Did some benchmarking with a set of 20 queries
25 matches
Mail list logo
