Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@vanzin Thanks for merging.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
Merging to master / 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88837/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88837 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88837/testReport)**
for PR 20781 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1912/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88837 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88837/testReport)**
for PR 20781 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@vanzin
Thanks for review~
1. I spent some time but didn't find the reason why same executor is
killed multiple times and I cannot reproduce either.
2. I found that same completed
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20781
The change looks good, but did you look at why the code is trying to kill
the same executor multiple times? That sounds like it could be a possible bug
on the scheduler backend, which should be
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88178/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88178 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88178/testReport)**
for PR 20781 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88178 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88178/testReport)**
for PR 20781 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1466/
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao
Thanks again for review.
It does exist in my cluster that same container can be processed multiple
times, which will make `numExecutorsRunning` negative. I think I've ever
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
Still I'm not so sure about the root cause, but adding defensive code seems
no harm.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88127/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88127 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88127/testReport)**
for PR 20781 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
Since the change for `YarnAllocator: killExecutor` is easy. Do you think
it's worth to have this defense?
Thanks again for review.
---
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao
Thanks for advice. I spent some time digging to find why multiple `kill`
sent from Driver to AM, but didn't figure out a way to reproduce.
I come to find that it's
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1428/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88127 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88127/testReport)**
for PR 20781 at commit
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
This basically means that drive send multiple same kill requests to AM,
right? I'm wondering how this would happen, shall we also guarantee this in the
driver side?
---
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
@jerryshao Thanks for taking look.
Yes, it does happen. we have jobs which have already finished all the tasks
but still holding 40~100 executors.
---
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/20781
Does it happen only in dynamic allocation enabled scenario?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/20781
cc @vanzin @tgravescs @cloud-fan @djvulee
Could you please help review this ?
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88116/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88116 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88116/testReport)**
for PR 20781 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20781
**[Test build #88116 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88116/testReport)**
for PR 20781 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1422/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20781
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
37 matches
Mail list logo
