[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-669596854 Merged to the dev branch :) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-669595846 Since it's approved and passing jenkins & GHA I'm going to merge this, but I'll add the comment about the time in the follow up I'll rebase on top of this after merge.

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-669389554 The only outstanding point of discussion is a test timeout length, which I don't believe is critical to address. If CI passes I intend to merge this PR and rebase the next PR

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-668340555 I've added a bunch of tests around the timestamp stuff. I think it's got all of the cases folks were asking for covered, and in the discussion on the PR to this PR it seems

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-667163174 So we don’t reject tasks sent to us and an executor can start decommissioning without the driver knowing yet so it is possible (although unlikely) to get a new task after

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-665379526 So it seems without the thread start, we end up in the situation where there is an orphan worker (see

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663678743 Sure, I can hold off on this one then @attilapiros :) This is an automated message from the Apache Git Service.

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663650784 All of the github actions pass, if no one else wants more time to comment on this, I'll merge this today. This

[GitHub] [spark] holdenk commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

holdenk commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663263429 cc @attilapiros & @agrawaldevesh This is an automated message from the Apache Git Service. To respond to the