Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Will do, and it's okay.
My view limited in the source code yet, but you guys have more practical
experience. So I learned from your points. It's beneficial.
---
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20998
@Ngone51 can you instead leave the behavior as is, and just update the
comment?
Sorry that its going to be a small change in the end, and all the extra
work the bad comments led you to do,
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/20998
@squito I completely agree that the comment is inaccurate.
Note that this is for a specific taskset, so impact is limited to that
taskset (w.r.t using executors for spec exec)
---
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20998
I'm not even really concerned about the case for two hosts -- I agree its
fine if we do something sub-optimal. I'm more concerned about code-clarity and
the behavior in general. It seems cleaner
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89228/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89228 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89228/testReport)**
for PR 20998 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89228 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89228/testReport)**
for PR 20998 at commit
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/20998
@squito My concern is, in large workloads, some nodes simply become bad for
some tasks (transient env or hardware issues, colocating containers, etc) while
being fine for others; speculative tasks
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20998
@mridulm more thoughts? I think this is the right change but I will leave
open for a bit to get more input
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89164/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89164 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89164/testReport)**
for PR 20998 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89164 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89164/testReport)**
for PR 20998 at commit
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Hi, @squito . Thank for review and comments.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Hi, @mridulm, thank for your comment. Actually, I have the same worry with
you. May be we can make this change as a second choice for `hasAttemptOnHost `,
in case of there's really no other hosts
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/20998
Adding isRunning can cause a single 'bad' node (from task pov - not
necessarily only bad hardware: just that task fails on node) can keep tasks to
fail repeatedly causing app to exit.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89026/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89026 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89026/testReport)**
for PR 20998 at commit
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
Hi, @felixcheung , thank for trigger a task and your comments.
> shouldn't this be up to the scheduler backend?
Actually, it is `TaskSchedulerImpl` who holds a thread to check whether
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20998
**[Test build #89026 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89026/testReport)**
for PR 20998 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20998
sounds fair, but shouldn't this be up to the scheduler backend? multiple
tasks/attempts can run simultaneously on the same physical host?
---
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/20998
Jenkins, ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user Ngone51 commented on the issue:
https://github.com/apache/spark/pull/20998
ping @pwendell @kayousterhout . pls help review, thanks :)
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20998
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
27 matches
Mail list logo
