This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new b98cf95896a2 [SPARK-46786][K8S] Fix `MountVolumesFeatureStep` to use
`ReadWriteOncePod` instead of `ReadWriteOnce`
b98cf95896a2 is described below
commit b98cf95896a2b14e3692f1e7d58660583a2bb175
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Sat Jan 20 17:51:12 2024 -0800
[SPARK-46786][K8S] Fix `MountVolumesFeatureStep` to use `ReadWriteOncePod`
instead of `ReadWriteOnce`
This PR aims to fix a duplicated volume mounting bug by using
`ReadWriteOncePod` instead of `ReadWriteOnce`.
This bug fix is based on the stable K8s feature which is available since
v1.22.
- [KEP-2485: ReadWriteOncePod PersistentVolume
AccessMode](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md)
-
https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes
- v1.22 Alpha
- v1.27 Beta
- v1.29 Stable
For the record, the minimum K8s version of GKE/EKS/AKE is **v1.24** as of
today and the latest v1.29 is supported like the following.
- [2024.01 (GKE Regular
Channel)](https://cloud.google.com/kubernetes-engine/docs/release-schedule)
- [2024.02 (AKE
GA)](https://learn.microsoft.com/en-us/azure/aks/supported-kubernetes-versions?tabs=azure-cli#aks-kubernetes-release-calendar)
This is a bug fix.
Pass the CIs with the existing PV-related tests.
No.
Closes #44817 from dongjoon-hyun/SPARK-46786.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
(cherry picked from commit 45ec74415a4a89851968941b80c490e37ee88daf)
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala | 2 +-
.../org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
index 78dd6ec21ed3..cbbbb9c0bdf5 100644
---
a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
+++
b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala
@@ -126,5 +126,5 @@ private[spark] object MountVolumesFeatureStep {
val PVC_ON_DEMAND = "OnDemand"
val PVC = "PersistentVolumeClaim"
val PVC_POSTFIX = "-pvc"
- val PVC_ACCESS_MODE = "ReadWriteOnce"
+ val PVC_ACCESS_MODE = "ReadWriteOncePod"
}
diff --git
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
index 1d373f3f8066..f8e76012638c 100644
---
a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
+++
b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/PVTestsSuite.scala
@@ -166,7 +166,7 @@ private[spark] trait PVTestsSuite { k8sSuite:
KubernetesSuite =>
}
}
- test("PVs with local hostpath and storageClass on statefulsets", k8sTestTag,
MinikubeTag) {
+ ignore("PVs with local hostpath and storageClass on statefulsets",
k8sTestTag, MinikubeTag) {
sparkAppConf
.set(s"spark.kubernetes.driver.volumes.persistentVolumeClaim.data.mount.path",
CONTAINER_MOUNT_PATH)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
