[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86090/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #86090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86090/testReport)** for PR 20080 at commit [`b03a496`](https://github.com/apache/spark/commit/b03a4968976781dff03961abc5caedae10ef10aa). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #86090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86090/testReport)** for PR 20080 at commit [`b03a496`](https://github.com/apache/spark/commit/b03a4968976781dff03961abc5caedae10ef10aa). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20080 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86083/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #86083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86083/testReport)** for PR 20080 at commit [`b03a496`](https://github.com/apache/spark/commit/b03a4968976781dff03961abc5caedae10ef10aa). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #86083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86083/testReport)** for PR 20080 at commit [`b03a496`](https://github.com/apache/spark/commit/b03a4968976781dff03961abc5caedae10ef10aa). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 Ping @wangyum --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 This change alone is pretty harmless. If it seems to work as intended, seems OK? Again, just wondering if `cachedExecutorIdleTimeoutS` should be the same, and check for nonnegative values? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 @srowen @jiangxb1987 I have test on my cluster with this patch. ``` bin/spark-sql --master yarn --conf spark.dynamicAllocation.enabled=true --conf spark.shuffle.service.enabled=true --conf spark.dynamicAllocation.executorIdleTimeout=0 ``` ``` 18/01/09 05:49:03.452 INFO DAGScheduler: Job 0 finished: processCmd at CliDriver.java:376, took 26.196061 s 75000 Time taken: 26.383 seconds, Fetched 1 row(s) 18/01/09 05:49:03.455 INFO SparkSQLCLIDriver: Time taken: 26.383 seconds, Fetched 1 row(s) spark-sql> 18/01/09 05:49:03.479 INFO ExecutorAllocationManager: Request to remove executorIds: 972 ``` `05:49:03.479 - 05:49:03.455 = 24 ms` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 @wangyum any comments on the follow-up here? if you've observed it gets the desired behavior, that's pretty convincing. At least we should treat both timeouts consistently. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 Good point @felixcheung though the polling is hard-coded to 100ms, so will be close enough to "immediately". Allowing a value of 0 on both timeouts seems OK, but yeah not clear whether it does have the desired effect. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20080 @wangyum have you already run any jobs on a cluster with this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20080 I wonder if we should do the same for cachedExecutorIdleTimeoutS ? it isn't checked for being nonnegative but should be, I suppose. I think this is fine otherwise. I don't see any reason not to allow it (see JIRA) and the logic is compatible with a value of 0 already. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20080 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85390/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20080 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #85390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85390/testReport)** for PR 20080 at commit [`1dcec41`](https://github.com/apache/spark/commit/1dcec41a3c1e2c001b0f9fed92aa6f03b6c47f3a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20080: [SPARK-22870][CORE] Dynamic allocation should allow 0 id...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20080 **[Test build #85390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85390/testReport)** for PR 20080 at commit [`1dcec41`](https://github.com/apache/spark/commit/1dcec41a3c1e2c001b0f9fed92aa6f03b6c47f3a). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
