[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14557911#comment-14557911
]
Sandy Ryza commented on SPARK-7699:
---
[~sowen] I think the possible flaw in your argument
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555871#comment-14555871
]
Sean Owen commented on SPARK-7699:
--
Say initial = 3 and min = 1. There is no load at the
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555837#comment-14555837
]
Sandy Ryza commented on SPARK-7699:
---
Sorry for the delay here. The desired behavior is
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555788#comment-14555788
]
Sean Owen commented on SPARK-7699:
--
[~sandyr] can you comment on the intended behavior? d
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550442#comment-14550442
]
Saisai Shao commented on SPARK-7699:
Yes, you are right after checking the code of {{a
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550401#comment-14550401
]
Sean Owen commented on SPARK-7699:
--
Backing up a bit here, after I re-read the code in mo
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550378#comment-14550378
]
Saisai Shao commented on SPARK-7699:
Yes, {{maxNeeded}} is the correct # if we have lo
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550367#comment-14550367
]
Sean Owen commented on SPARK-7699:
--
Isn't maxNeeded the correct number at that point then
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550357#comment-14550357
]
Saisai Shao commented on SPARK-7699:
I think if we have lots of pending tasks, the act
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550344#comment-14550344
]
Sean Owen commented on SPARK-7699:
--
Does your invocation use enough tasks to need more th
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550339#comment-14550339
]
meiyoula commented on SPARK-7699:
-
[~srowen]Sorry, I have tested it, run SparkPi with spar
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550249#comment-14550249
]
Sean Owen commented on SPARK-7699:
--
Isn't this the normal state of a Spark app, run with
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550223#comment-14550223
]
meiyoula commented on SPARK-7699:
-
[~srowen] you said "I'm just talking about the case whe
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550185#comment-14550185
]
Sean Owen commented on SPARK-7699:
--
The parameter has meaning if the system isn't idle, r
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550163#comment-14550163
]
Saisai Shao commented on SPARK-7699:
The problem is as you mentioned here:
{code}
val
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550157#comment-14550157
]
Saisai Shao commented on SPARK-7699:
I think for the first time called {{updateAndSync
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549956#comment-14549956
]
Sean Owen commented on SPARK-7699:
--
Agree, but this logic change only also happens a bit
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549623#comment-14549623
]
meiyoula commented on SPARK-7699:
-
[~sandyryza] You are the author of the code, can you ex
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549618#comment-14549618
]
meiyoula commented on SPARK-7699:
-
I think the situation you said will never appears. Beca
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547974#comment-14547974
]
Sean Owen commented on SPARK-7699:
--
It would make a difference if the program immediately
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547956#comment-14547956
]
meiyoula commented on SPARK-7699:
-
If so, I think the spark.dynamicAllocation.initialExecu
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547665#comment-14547665
]
Sean Owen commented on SPARK-7699:
--
I think that's maybe intentional? the logic is detect
[
https://issues.apache.org/jira/browse/SPARK-7699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547622#comment-14547622
]
meiyoula commented on SPARK-7699:
-
val oldNumExecutorsTarget = numExecutorsTarget
23 matches
Mail list logo
