This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 4d4c044 [SPARK-38392][K8S][TESTS] Add `spark-` prefix to namespaces and `-driver` suffix to drivers during IT 4d4c044 is described below commit 4d4c0444ac71f177711a5dab79b506f5048d9200 Author: Martin Tzvetanov Grigorov <mgrigo...@apache.org> AuthorDate: Wed Mar 2 10:02:13 2022 -0800 [SPARK-38392][K8S][TESTS] Add `spark-` prefix to namespaces and `-driver` suffix to drivers during IT ### What changes were proposed in this pull request? There are two small proposals: 1) prefix the name of the temporary k8s namespaces with `"spark-"` so that the output of `kubectl get ns" is more clear. 2) unify the name of the driver pod in non-test and IT tests to always use `-driver` as a suffix. ### Why are the changes needed? At the moment the name of the temporary namespace is just UUID without the `-`s. When one reads the result of `kubectl get ns` it is a bit cryptic to see UUIDs. The names of the driver pods in ITs are not telling me that they are Drivers. In non-test (i.e. production) the driver pod names are suffixed with `-driver`. I propose the same for IT tests. Executor pods always use `-exec-` in their pod names, both in non-test and ITs. ### Does this PR introduce _any_ user-facing change? Yes! Developers who debug IT tests will see more clear names now. ### How was this patch tested? Manually with `kubectl get ns --watch` and `kubectl get po --watch`. Closes #35711 from martin-g/k8s-test-names-improvement. Authored-by: Martin Tzvetanov Grigorov <mgrigo...@apache.org> Signed-off-by: Dongjoon Hyun <dongj...@apache.org> --- .../org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala | 3 ++- .../spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala index 15ce487..9faf73f 100644 --- a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala +++ b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala @@ -184,7 +184,8 @@ class KubernetesSuite extends SparkFunSuite protected def setUpTest(): Unit = { appLocator = UUID.randomUUID().toString.replaceAll("-", "") - driverPodName = "spark-test-app-" + UUID.randomUUID().toString.replaceAll("-", "") + driverPodName = "spark-test-app-" + + UUID.randomUUID().toString.replaceAll("-", "") + "-driver" sparkAppConf = kubernetesTestComponents.newSparkAppConf() .set("spark.kubernetes.container.image", image) .set("spark.kubernetes.driver.pod.name", driverPodName) diff --git a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala index 411857f..4fdb89e 100644 --- a/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala +++ b/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesTestComponents.scala @@ -37,7 +37,8 @@ private[spark] class KubernetesTestComponents(defaultClient: DefaultKubernetesCl val namespaceOption = Option(System.getProperty(CONFIG_KEY_KUBE_NAMESPACE)) val hasUserSpecifiedNamespace = namespaceOption.isDefined - val namespace = namespaceOption.getOrElse(UUID.randomUUID().toString.replaceAll("-", "")) + val namespace = namespaceOption.getOrElse("spark-" + + UUID.randomUUID().toString.replaceAll("-", "")) val serviceAccountName = Option(System.getProperty(CONFIG_KEY_KUBE_SVC_ACCOUNT)) .getOrElse("default") --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org