Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18084
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wi
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78049/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #78049 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78049/testReport)**
for PR 18084 at commit
[`9ddf23a`](https://github.com/apache/spark/commit/9
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18084
LGTM, pending test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #78049 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78049/testReport)**
for PR 18084 at commit
[`9ddf23a`](https://github.com/apache/spark/commit/9d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78023/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #78023 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78023/testReport)**
for PR 18084 at commit
[`0887eab`](https://github.com/apache/spark/commit/0
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #78023 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78023/testReport)**
for PR 18084 at commit
[`0887eab`](https://github.com/apache/spark/commit/08
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18084
Also please rebase the latest master :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18084
the fix LGTM, it would be better to add some comments to explain it clearly
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18084
I think your point is, in case a LaunchDriver message and a KillDriver
message are send out simultaneously, there is a race condition that which
message arrives to worker earlier is not determin
Github user liyichao commented on the issue:
https://github.com/apache/spark/pull/18084
OK, another scenario:
* driver with driverId1 started on worker1
* worker1 lost
* master add driverId1 to waitingDrivers
* worker1 reconnects and sends DriverStateChanged(driverI
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18084
We should also check in Worker that we don't launch duplicate drivers, I
think the logic should be added in handling `LaunchDriver` message.
---
If your project is set up for it, you can reply
Github user liyichao commented on the issue:
https://github.com/apache/spark/pull/18084
Hi, add a workerId may not work. For example, this scenario:
* driver with driverId1 started on worker1
* worker1 lost
* master add driverId1 to waitingDrivers
* worker1 reconnec
Github user liyichao commented on the issue:
https://github.com/apache/spark/pull/18084
ping @jiangxb1987
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77423/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77423 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77423/testReport)**
for PR 18084 at commit
[`da0f977`](https://github.com/apache/spark/commit/d
Github user liyichao commented on the issue:
https://github.com/apache/spark/pull/18084
Hi, I've thought more thoroughly about this.
The main state involved here is Master.workers, Master.idToWorker, and
WorkerInfo.drivers. Say `driverId1` runs on Worker A. Assume A is network
Github user liyichao commented on the issue:
https://github.com/apache/spark/pull/18084
Thanks for the reply. I have add some more tests to verify the state of
master and worker after relaunching.
I will try think about if there are ways to reuse the old driver struct.
---
I
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77423 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77423/testReport)**
for PR 18084 at commit
[`da0f977`](https://github.com/apache/spark/commit/da
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18084
Maybe some more actions should be done in `relaunchDriver()` such as have
`driver.worker` removes the dependency of the relaunched driver, but it will be
sort of wasting resources to remove and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77353/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77353 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77353/testReport)**
for PR 18084 at commit
[`6ab9a0f`](https://github.com/apache/spark/commit/6
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77353 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77353/testReport)**
for PR 18084 at commit
[`6ab9a0f`](https://github.com/apache/spark/commit/6a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77312/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77312 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77312/testReport)**
for PR 18084 at commit
[`9ea2061`](https://github.com/apache/spark/commit/9
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18084
**[Test build #77312 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77312/testReport)**
for PR 18084 at commit
[`9ea2061`](https://github.com/apache/spark/commit/9e
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18084
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18084
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
34 matches
Mail list logo
