Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20888
**[Test build #89184 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89184/testReport)**
for PR 20888 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20888
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89131/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20888
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20888
**[Test build #89131 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89131/testReport)**
for PR 20888 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20888
**[Test build #89131 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89131/testReport)**
for PR 20888 at commit
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
Thank for the hints. I've taken a deeper look at the possible solutions and
the suggested test. The problem is similar but not the same so I would solve it
a different way. So here is my
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20888
Sounds like this approach is just racy by nature. There's a similar test in
SparkContextSuite ("Cancelling stages/jobs with custom reasons") where I fixed
a similar issue using a semaphore. Sounds
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
`SparkStatusTracker` states the following:
```
* These APIs intentionally provide very weak consistency semantics;
consumers of these APIs should
* be prepared to handle empty /
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
@vanzin @squito yeah, there is an issue with threading as well. I'm just
taking a look at it because it's not obvious.
---
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20888
I had sort of the same doubt as Imran, because I thought scalasuite ran
tests in order... but I just ran a suite where the tests were not run in the
order declared in the source file. So it sounds
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20888
ah ok, yes when run in isolation, the stage will be 0, so your change makes
sense. But that is not what is making it flaky in a full test run
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20888
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88536/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20888
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20888
**[Test build #88536 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88536/testReport)**
for PR 20888 at commit
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
Uploaded logs for the jira. You're right when you pointed out a second
issue with the `stageToKill`. The `onJobStart` tries to cancel the same ID
twice.
---
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
I mean on my machine the stage ID is zero for long long time here:
```
DataFrameRangeSuite.stageToKill = TaskContext.get().stageId()
```
and after 200 seconds the other
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20888
> if I execute the test on my machine alone it never pass.
you mean it never fails on your machine, right? its only flaky when you
run everything on jenkins?
---
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20888
hmm you're right, I was looking at a different branch in my editor and
didn't pay attention that it was reset in the code I linked to on master, oops.
I still dont' understand your proposed
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
Just an additional info if I execute the test on my machine alone it never
pass.
---
-
To unsubscribe, e-mail:
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/20888
Where do you think the reset should happen? There is already one inside
`withSQLConf` which makes a reset before job submit.
Related the ID I've just taken a look at the original
Github user squito commented on the issue:
https://github.com/apache/spark/pull/20888
I think you're right about killing the wrong stage, but I don't think its
exactly what you've outlined. The original code doesn't try to kill a stage
with ID == 0 -- instead its just waiting until
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20888
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20888
**[Test build #88536 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88536/testReport)**
for PR 20888 at commit
23 matches
Mail list logo
