HwangDongJun commented on code in PR #58187:
URL: https://github.com/apache/spark/pull/58187#discussion_r4345271886
##########
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala:
##########
@@ -673,6 +673,17 @@ private[spark] object Config extends Logging {
.booleanConf
.createWithDefault(true)
+ val USE_DRIVER_SCHEDULING_GATE =
+ ConfigBuilder("spark.kubernetes.submission.useDriverSchedulingGate")
+ .doc("If true, in cluster mode the driver pod is created with a
scheduling gate that " +
+ "is removed once the resources the driver mounts exist. Removing the
gate needs the " +
+ "`patch` verb on pods, and the scheduler must honor scheduling gates.
If false, the " +
+ "driver pod can be scheduled before those resources exist.")
+ .version("4.4.0")
+ .withBindingPolicy(ConfigBindingPolicy.NOT_APPLICABLE)
+ .booleanConf
+ .createWithDefault(true)
Review Comment:
Done as you suggested. `spark.kubernetes.submission.useDriverSchedulingGate`
is now `createWithDefault(false)` with `.version("4.4.0")`, so a default
submission needs no `patch` and no gate support, and nothing changes for
deployments that work today. The RBAC bullet and the Pod Spec rows in
`docs/running-on-kubernetes.md` are conditional on the setting, and the
`Config.scala` doc and the PR description say the same.
The K8s integration tests now set the configuration to `true` in
`KubernetesTestComponents.newSparkAppConf()`, so the gated path stays covered
there. I only compiled that module; I did not run the integration tests. I will
leave flipping the default to a separate master-only change with its own
migration guide entry, as you proposed, and I removed the migration guide entry
from this PR (see the `core-migration-guide.md` thread).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
