This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new bbd8fd6af72e [SPARK-55399][K8S] Improve `KubernetesDriverEndpoint` to
use `patch` instead of `edit` API
bbd8fd6af72e is described below
commit bbd8fd6af72e00a8152641fe008353391d8396f1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Feb 6 18:16:21 2026 -0800
[SPARK-55399][K8S] Improve `KubernetesDriverEndpoint` to use `patch`
instead of `edit` API
### What changes were proposed in this pull request?
This PR aims to improve `KubernetesDriverEndpoint` to use `patch` instead
of `edit` API
### Why are the changes needed?
**Network Efficiency**
- `edit` requires fetching the entire resource and sending the full updated
resource back.
- `patch` only transmits the specific changes, making it much more
network-efficient.
**Concurrency & Conflict Resolution**
- `edit` typically follows a Get -> Modify -> Update (PUT) pattern. Using
this pattern creates a race condition where, if another client modifies the
resource in between, a 409 Conflict error occurs due to a mismatched
resourceVersion.
- `patch` sends only the changes (delta) to the server, where the merge is
handled server-side. This significantly reduces the risk of conflicts,
especially for simple operations like adding an annotation.
### Does this PR introduce _any_ user-facing change?
This will reduce the overhead of K8s control plane and the chance of 409
error.
### How was this patch tested?
Pass the CIs with newly updated test case.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: `Sonnet 4.5` on `Claude Code`
Closes #54179 from dongjoon-hyun/SPARK-55399.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
index 3ff61d44e2be..606a59f88269 100644
---
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
+++
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala
@@ -353,10 +353,11 @@ private[spark] class KubernetesClusterSchedulerBackend(
kubernetesClient.pods()
.inNamespace(namespace)
.withName(x.podName)
- .edit({p: Pod => new PodBuilder(p).editMetadata()
+ .patch(PATCH_CONTEXT, new PodBuilder()
+ .withNewMetadata()
.addToLabels(SPARK_EXECUTOR_ID_LABEL, newId)
.endMetadata()
- .build()})
+ .build())
}
}
executorService.execute(labelTask)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
