HwangDongJun commented on PR #58187: URL: https://github.com/apache/spark/pull/58187#issuecomment-6334794016
@dongjoon-hyun Thank you for the review. I replied in each of the 13 threads; here is the summary. **Default and docs** - `spark.kubernetes.submission.useDriverSchedulingGate` now defaults to `false` in 4.4.0, following your suggestion. The `patch` requirement, the scheduler requirement and the `nodeName` case are documented as conditions of enabling it, and the migration guide entry is removed from this PR. The K8s integration tests set the configuration to `true`, so the gated path stays covered (I only compiled that module). - The wording is "the resources `spark-submit` creates for the driver" everywhere, including the PR title. The admission-webhook `nodeName` case and a driver left in `SchedulingGated` by an interrupted submission are documented. **Code** - The `can-i patch pod` hint is logged only for a 403 and mentions the switch, and a gate of the same name on the resolved pod is replaced instead of duplicated. **Tests** - I first ran your mutations against the previous tests: six of the seven passed all 17 tests, and only the `JSON_MERGE` one failed, which matches what you described. `ClientSuite` now has 22 tests and fails on all seven (and on three more that I added). **Verification** - `kubernetes` module tests (441) and `./dev/scalastyle -Pkubernetes` pass, and the fork CI run on `46dbf98c658` passed. - With the final code, the compiled `Client.run()` on a real cluster: the default submits without a gate, `true` leaves the pod ungated, a template that already has the Spark gate is accepted, and an account without `patch` fails with HTTP 403 and leaves nothing behind. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
