Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
merged to master. thanks @attilapiros
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86490/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86490 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86490/testReport)**
for PR 20203 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
@attilapiros can you please update the PR description to also mention node
blacklisting (at least briefly) and file a jira for the followup work, and ping
me & tom on it?
lgtm
---
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
I tried this out on a cluster and seemed fine. I also tried with a bad
app, where all tasks fail, its not just because of the host, and all the
executors show up as blacklisted, but I guess that is
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86490 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86490/testReport)**
for PR 20203 at commit
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/20203
Jenkins, test this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
Test failure seems to be unrelated as it it is in
org.apache.spark.sql.hive.client.HiveClientSuites.
@squito, @tgravescs could you please restart testing
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86482/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86482 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86482/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86482 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86482/testReport)**
for PR 20203 at commit
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
One more reason to run tests in sbt / maven. In intelliJ somehow the
complete suite was successful. But the current failure seems to me unrelated,
as org.apache.spark.deploy.history has 0
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86408/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86408 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86408/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86408 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86408/testReport)**
for PR 20203 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86400/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86400 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86400/testReport)**
for PR 20203 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
@attilapiros test failures look real (you probably just need to regenerate
some of those expectations).
---
-
To unsubscribe,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86398/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86398 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86398/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86400 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86400/testReport)**
for PR 20203 at commit
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
btw another way you could test out having a bad host would be something
like this (untested):
```scala
import org.apache.spark.SparkEnv
val hosts = sc.parallelize(1 to 1,
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
The node blacklisting is tested by unit tests:
- HistoryServerSuite
- TaskSetBlacklistSuite
- AppStatusListenerSuite
And manually with a 2 node cluster:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #86398 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86398/testReport)**
for PR 20203 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
that sounds fine with me
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
@tgravescs @squito: But what about doing it in a separate PR as this is
already quite big?
So node backlisting would be still within the scope but executor page
changes would be done for a
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
Of course I can update the executors page too.
@tgravescs I think via handling the onStageCompleted I can do what you
would like to see here.
---
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/20203
I think you only want the stages currently running to show up.
---
-
To unsubscribe, e-mail:
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
yeah I think its probably fine to update the executors page that way --
lets at least see how it looks. Would the list include all stages ever
blacklisted? Only those stages still running? The
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/20203
I think it would be nice to add something on the executors page as well.
Perthaps we can put something like "for stage[, ,...]" if
its only blacklisted for those tasksets.
But I'm also
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/20203
@squito thanks, I missed the img link and misread ExecutorTable as
ExecutorPage. On that note the UI portion of this change LGTM.
---
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
attila attached some screenshots to the jira (showing both what happens
with stage blacklisting and full application blacklisting).
The only change here is to the page for a specific stage,
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/20203
From a code pov the UI change looks fine, but could you upload a few
screenshots of the change? Also the UI simply says if the exec is blacklisted
for the whole app or just a stage, but doesn't
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20203
@ajbozarth maybe you have some thoughts on the UI, and whether it makes
sense to put anything on the executors page?
@CodingCat you also often have good UI suggestions :)
thanks
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85991/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85991 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85991/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85991 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85991/testReport)**
for PR 20203 at commit
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
@tgravescs: You are welcome.
Running Spark as (with lower application.maxFailedTasksPerExecutor than
stage.maxFailedTasksPerExecutor):
```
$ bin/spark-shell --master
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/20203
thanks for working on this, I'm going to try this out and do further
review. Did you test for application failures and on the history server?
cc @squito since he had some comments on the
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/20203
cc @tgravescs
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85872/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85872 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85872/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85872 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85872/testReport)**
for PR 20203 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85869 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85869/testReport)**
for PR 20203 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85869/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20203
**[Test build #85869 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85869/testReport)**
for PR 20203 at commit
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20203
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20203
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
58 matches
Mail list logo
