[spark] branch master updated (5e17014 -> 83f8e13)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e17014 [SPARK-33107][SQL] Remove hive-2.3 workaround code add 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub No new revisions were added by this update. Summary of changes: python/pyspark/resource/profile.pyi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (5e17014 -> 83f8e13)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e17014 [SPARK-33107][SQL] Remove hive-2.3 workaround code add 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub No new revisions were added by this update. Summary of changes: python/pyspark/resource/profile.pyi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (5e17014 -> 83f8e13)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e17014 [SPARK-33107][SQL] Remove hive-2.3 workaround code add 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub No new revisions were added by this update. Summary of changes: python/pyspark/resource/profile.pyi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (5e17014 -> 83f8e13)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e17014 [SPARK-33107][SQL] Remove hive-2.3 workaround code add 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub No new revisions were added by this update. Summary of changes: python/pyspark/resource/profile.pyi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (5e17014 -> 83f8e13)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 5e17014 [SPARK-33107][SQL] Remove hive-2.3 workaround code add 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub No new revisions were added by this update. Summary of changes: python/pyspark/resource/profile.pyi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (83f8e13 -> c78971b)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub add c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (83f8e13 -> c78971b)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub add c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (83f8e13 -> c78971b)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub add c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (83f8e13 -> c78971b)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub add c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (83f8e13 -> c78971b)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 83f8e13 [SPARK-33086][FOLLOW-UP] Remove unused Optional import from pyspark.resource.profile stub add c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c78971b -> 50b2a49)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT add 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c78971b -> 50b2a49)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT add 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c78971b -> 50b2a49)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT add 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c78971b -> 50b2a49)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT add 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (c78971b -> 50b2a49)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from c78971b [SPARK-33106][BUILD] Fix resolvers clash in SBT add 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue No new revisions were added by this update. Summary of changes: project/SparkBuild.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (50b2a49 -> 4af1ac9)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue add 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/internal/SQLConf.scala | 11 +++ .../jdbc/connection/BasicConnectionProvider.scala | 2 ++ .../datasources/jdbc/connection/ConnectionProvider.scala | 7 +-- .../jdbc/connection/DB2ConnectionProvider.scala | 2 ++ .../jdbc/connection/MSSQLConnectionProvider.scala | 2 ++ .../jdbc/connection/MariaDBConnectionProvider.scala | 2 ++ .../jdbc/connection/OracleConnectionProvider.scala| 2 ++ .../jdbc/connection/PostgresConnectionProvider.scala | 2 ++ .../apache/spark/sql/jdbc/JdbcConnectionProvider.scala| 6 ++ .../jdbc/connection/ConnectionProviderSuite.scala | 15 +-- .../jdbc/connection/ConnectionProviderSuiteBase.scala | 2 +- .../IntentionallyFaultyConnectionProvider.scala | 1 + 12 files changed, 49 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (50b2a49 -> 4af1ac9)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue add 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/internal/SQLConf.scala | 11 +++ .../jdbc/connection/BasicConnectionProvider.scala | 2 ++ .../datasources/jdbc/connection/ConnectionProvider.scala | 7 +-- .../jdbc/connection/DB2ConnectionProvider.scala | 2 ++ .../jdbc/connection/MSSQLConnectionProvider.scala | 2 ++ .../jdbc/connection/MariaDBConnectionProvider.scala | 2 ++ .../jdbc/connection/OracleConnectionProvider.scala| 2 ++ .../jdbc/connection/PostgresConnectionProvider.scala | 2 ++ .../apache/spark/sql/jdbc/JdbcConnectionProvider.scala| 6 ++ .../jdbc/connection/ConnectionProviderSuite.scala | 15 +-- .../jdbc/connection/ConnectionProviderSuiteBase.scala | 2 +- .../IntentionallyFaultyConnectionProvider.scala | 1 + 12 files changed, 49 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (50b2a49 -> 4af1ac9)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue add 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/internal/SQLConf.scala | 11 +++ .../jdbc/connection/BasicConnectionProvider.scala | 2 ++ .../datasources/jdbc/connection/ConnectionProvider.scala | 7 +-- .../jdbc/connection/DB2ConnectionProvider.scala | 2 ++ .../jdbc/connection/MSSQLConnectionProvider.scala | 2 ++ .../jdbc/connection/MariaDBConnectionProvider.scala | 2 ++ .../jdbc/connection/OracleConnectionProvider.scala| 2 ++ .../jdbc/connection/PostgresConnectionProvider.scala | 2 ++ .../apache/spark/sql/jdbc/JdbcConnectionProvider.scala| 6 ++ .../jdbc/connection/ConnectionProviderSuite.scala | 15 +-- .../jdbc/connection/ConnectionProviderSuiteBase.scala | 2 +- .../IntentionallyFaultyConnectionProvider.scala | 1 + 12 files changed, 49 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch branch-2.4 updated: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 5a50e30 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency 5a50e30 is described below commit 5a50e30881d11971efe91b95ca6a20bb33868d2a Author: nssalian AuthorDate: Sun Oct 11 20:37:41 2020 -0700 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency ### What changes were proposed in this pull request? Similar patch to #24748 but applied to the branch-2.4. Backporting the fix to releases 2.4.x. Closes #29877 from nssalian/SPARK-27872. Authored-by: nssalian Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/deploy/k8s/Config.scala | 7 ++-- .../apache/spark/deploy/k8s/KubernetesUtils.scala | 13 +++ .../DriverKubernetesCredentialsFeatureStep.scala | 11 +- .../ExecutorKubernetesCredentialsFeatureStep.scala | 43 ++ .../cluster/k8s/KubernetesExecutorBuilder.scala| 9 - .../k8s/KubernetesExecutorBuilderSuite.scala | 9 - .../k8s/integrationtest/BasicTestsSuite.scala | 7 .../k8s/integrationtest/KubernetesSuite.scala | 4 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala index c7338a7..02471af 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala @@ -61,10 +61,9 @@ private[spark] object Config extends Logging { .stringConf .createOptional - val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = - "spark.kubernetes.authenticate.driver" - val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = - "spark.kubernetes.authenticate.driver.mounted" + val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = "spark.kubernetes.authenticate.driver" + val KUBERNETES_AUTH_EXECUTOR_CONF_PREFIX = "spark.kubernetes.authenticate.executor" + val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = "spark.kubernetes.authenticate.driver.mounted" val KUBERNETES_AUTH_CLIENT_MODE_PREFIX = "spark.kubernetes.authenticate" val OAUTH_TOKEN_CONF_SUFFIX = "oauthToken" val OAUTH_TOKEN_FILE_CONF_SUFFIX = "oauthTokenFile" diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala index 588cd9d..9e32d6e 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala @@ -16,6 +16,8 @@ */ package org.apache.spark.deploy.k8s +import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, ContainerStatus, Pod, PodBuilder} + import org.apache.spark.SparkConf import org.apache.spark.util.Utils @@ -60,4 +62,15 @@ private[spark] object KubernetesUtils { } def parseMasterUrl(url: String): String = url.substring("k8s://".length) + + def buildPodWithServiceAccount(serviceAccount: Option[String], pod: SparkPod): Option[Pod] = { +serviceAccount.map { account => + new PodBuilder(pod.pod) +.editOrNewSpec() + .withServiceAccount(account) + .withServiceAccountName(account) +.endSpec() +.build() +} + } } diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala index ff5ad66..6079381 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala @@ -27,6 +27,7 @@ import io.fabric8.kubernetes.api.model.{ContainerBuilder, HasMetadata, PodBuilde import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod} import org.apache.spark.deploy.k8s.Config._ import org.apache.spark.deploy.k8s.Constants._ +import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: KubernetesConf[_]) extends KubernetesFeatureConfigStep { @@ -70,15 +71,7 @@ private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube o
[spark] branch branch-2.4 updated: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 5a50e30 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency 5a50e30 is described below commit 5a50e30881d11971efe91b95ca6a20bb33868d2a Author: nssalian AuthorDate: Sun Oct 11 20:37:41 2020 -0700 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency ### What changes were proposed in this pull request? Similar patch to #24748 but applied to the branch-2.4. Backporting the fix to releases 2.4.x. Closes #29877 from nssalian/SPARK-27872. Authored-by: nssalian Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/deploy/k8s/Config.scala | 7 ++-- .../apache/spark/deploy/k8s/KubernetesUtils.scala | 13 +++ .../DriverKubernetesCredentialsFeatureStep.scala | 11 +- .../ExecutorKubernetesCredentialsFeatureStep.scala | 43 ++ .../cluster/k8s/KubernetesExecutorBuilder.scala| 9 - .../k8s/KubernetesExecutorBuilderSuite.scala | 9 - .../k8s/integrationtest/BasicTestsSuite.scala | 7 .../k8s/integrationtest/KubernetesSuite.scala | 4 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala index c7338a7..02471af 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala @@ -61,10 +61,9 @@ private[spark] object Config extends Logging { .stringConf .createOptional - val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = - "spark.kubernetes.authenticate.driver" - val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = - "spark.kubernetes.authenticate.driver.mounted" + val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = "spark.kubernetes.authenticate.driver" + val KUBERNETES_AUTH_EXECUTOR_CONF_PREFIX = "spark.kubernetes.authenticate.executor" + val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = "spark.kubernetes.authenticate.driver.mounted" val KUBERNETES_AUTH_CLIENT_MODE_PREFIX = "spark.kubernetes.authenticate" val OAUTH_TOKEN_CONF_SUFFIX = "oauthToken" val OAUTH_TOKEN_FILE_CONF_SUFFIX = "oauthTokenFile" diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala index 588cd9d..9e32d6e 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala @@ -16,6 +16,8 @@ */ package org.apache.spark.deploy.k8s +import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, ContainerStatus, Pod, PodBuilder} + import org.apache.spark.SparkConf import org.apache.spark.util.Utils @@ -60,4 +62,15 @@ private[spark] object KubernetesUtils { } def parseMasterUrl(url: String): String = url.substring("k8s://".length) + + def buildPodWithServiceAccount(serviceAccount: Option[String], pod: SparkPod): Option[Pod] = { +serviceAccount.map { account => + new PodBuilder(pod.pod) +.editOrNewSpec() + .withServiceAccount(account) + .withServiceAccountName(account) +.endSpec() +.build() +} + } } diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala index ff5ad66..6079381 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala @@ -27,6 +27,7 @@ import io.fabric8.kubernetes.api.model.{ContainerBuilder, HasMetadata, PodBuilde import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod} import org.apache.spark.deploy.k8s.Config._ import org.apache.spark.deploy.k8s.Constants._ +import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: KubernetesConf[_]) extends KubernetesFeatureConfigStep { @@ -70,15 +71,7 @@ private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube o
[spark] branch branch-2.4 updated: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 5a50e30 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency 5a50e30 is described below commit 5a50e30881d11971efe91b95ca6a20bb33868d2a Author: nssalian AuthorDate: Sun Oct 11 20:37:41 2020 -0700 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency ### What changes were proposed in this pull request? Similar patch to #24748 but applied to the branch-2.4. Backporting the fix to releases 2.4.x. Closes #29877 from nssalian/SPARK-27872. Authored-by: nssalian Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/deploy/k8s/Config.scala | 7 ++-- .../apache/spark/deploy/k8s/KubernetesUtils.scala | 13 +++ .../DriverKubernetesCredentialsFeatureStep.scala | 11 +- .../ExecutorKubernetesCredentialsFeatureStep.scala | 43 ++ .../cluster/k8s/KubernetesExecutorBuilder.scala| 9 - .../k8s/KubernetesExecutorBuilderSuite.scala | 9 - .../k8s/integrationtest/BasicTestsSuite.scala | 7 .../k8s/integrationtest/KubernetesSuite.scala | 4 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala index c7338a7..02471af 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala @@ -61,10 +61,9 @@ private[spark] object Config extends Logging { .stringConf .createOptional - val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = - "spark.kubernetes.authenticate.driver" - val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = - "spark.kubernetes.authenticate.driver.mounted" + val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = "spark.kubernetes.authenticate.driver" + val KUBERNETES_AUTH_EXECUTOR_CONF_PREFIX = "spark.kubernetes.authenticate.executor" + val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = "spark.kubernetes.authenticate.driver.mounted" val KUBERNETES_AUTH_CLIENT_MODE_PREFIX = "spark.kubernetes.authenticate" val OAUTH_TOKEN_CONF_SUFFIX = "oauthToken" val OAUTH_TOKEN_FILE_CONF_SUFFIX = "oauthTokenFile" diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala index 588cd9d..9e32d6e 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala @@ -16,6 +16,8 @@ */ package org.apache.spark.deploy.k8s +import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, ContainerStatus, Pod, PodBuilder} + import org.apache.spark.SparkConf import org.apache.spark.util.Utils @@ -60,4 +62,15 @@ private[spark] object KubernetesUtils { } def parseMasterUrl(url: String): String = url.substring("k8s://".length) + + def buildPodWithServiceAccount(serviceAccount: Option[String], pod: SparkPod): Option[Pod] = { +serviceAccount.map { account => + new PodBuilder(pod.pod) +.editOrNewSpec() + .withServiceAccount(account) + .withServiceAccountName(account) +.endSpec() +.build() +} + } } diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala index ff5ad66..6079381 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala @@ -27,6 +27,7 @@ import io.fabric8.kubernetes.api.model.{ContainerBuilder, HasMetadata, PodBuilde import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod} import org.apache.spark.deploy.k8s.Config._ import org.apache.spark.deploy.k8s.Constants._ +import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: KubernetesConf[_]) extends KubernetesFeatureConfigStep { @@ -70,15 +71,7 @@ private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube o
[spark] branch branch-2.4 updated: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 5a50e30 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency 5a50e30 is described below commit 5a50e30881d11971efe91b95ca6a20bb33868d2a Author: nssalian AuthorDate: Sun Oct 11 20:37:41 2020 -0700 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency ### What changes were proposed in this pull request? Similar patch to #24748 but applied to the branch-2.4. Backporting the fix to releases 2.4.x. Closes #29877 from nssalian/SPARK-27872. Authored-by: nssalian Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/deploy/k8s/Config.scala | 7 ++-- .../apache/spark/deploy/k8s/KubernetesUtils.scala | 13 +++ .../DriverKubernetesCredentialsFeatureStep.scala | 11 +- .../ExecutorKubernetesCredentialsFeatureStep.scala | 43 ++ .../cluster/k8s/KubernetesExecutorBuilder.scala| 9 - .../k8s/KubernetesExecutorBuilderSuite.scala | 9 - .../k8s/integrationtest/BasicTestsSuite.scala | 7 .../k8s/integrationtest/KubernetesSuite.scala | 4 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala index c7338a7..02471af 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala @@ -61,10 +61,9 @@ private[spark] object Config extends Logging { .stringConf .createOptional - val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = - "spark.kubernetes.authenticate.driver" - val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = - "spark.kubernetes.authenticate.driver.mounted" + val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = "spark.kubernetes.authenticate.driver" + val KUBERNETES_AUTH_EXECUTOR_CONF_PREFIX = "spark.kubernetes.authenticate.executor" + val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = "spark.kubernetes.authenticate.driver.mounted" val KUBERNETES_AUTH_CLIENT_MODE_PREFIX = "spark.kubernetes.authenticate" val OAUTH_TOKEN_CONF_SUFFIX = "oauthToken" val OAUTH_TOKEN_FILE_CONF_SUFFIX = "oauthTokenFile" diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala index 588cd9d..9e32d6e 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala @@ -16,6 +16,8 @@ */ package org.apache.spark.deploy.k8s +import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, ContainerStatus, Pod, PodBuilder} + import org.apache.spark.SparkConf import org.apache.spark.util.Utils @@ -60,4 +62,15 @@ private[spark] object KubernetesUtils { } def parseMasterUrl(url: String): String = url.substring("k8s://".length) + + def buildPodWithServiceAccount(serviceAccount: Option[String], pod: SparkPod): Option[Pod] = { +serviceAccount.map { account => + new PodBuilder(pod.pod) +.editOrNewSpec() + .withServiceAccount(account) + .withServiceAccountName(account) +.endSpec() +.build() +} + } } diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala index ff5ad66..6079381 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala @@ -27,6 +27,7 @@ import io.fabric8.kubernetes.api.model.{ContainerBuilder, HasMetadata, PodBuilde import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod} import org.apache.spark.deploy.k8s.Config._ import org.apache.spark.deploy.k8s.Constants._ +import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: KubernetesConf[_]) extends KubernetesFeatureConfigStep { @@ -70,15 +71,7 @@ private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube o
[spark] branch branch-2.4 updated: [SPARK-27872][K8S][2.4] Fix executor service account inconsistency
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new 5a50e30 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency 5a50e30 is described below commit 5a50e30881d11971efe91b95ca6a20bb33868d2a Author: nssalian AuthorDate: Sun Oct 11 20:37:41 2020 -0700 [SPARK-27872][K8S][2.4] Fix executor service account inconsistency ### What changes were proposed in this pull request? Similar patch to #24748 but applied to the branch-2.4. Backporting the fix to releases 2.4.x. Closes #29877 from nssalian/SPARK-27872. Authored-by: nssalian Signed-off-by: Dongjoon Hyun --- .../scala/org/apache/spark/deploy/k8s/Config.scala | 7 ++-- .../apache/spark/deploy/k8s/KubernetesUtils.scala | 13 +++ .../DriverKubernetesCredentialsFeatureStep.scala | 11 +- .../ExecutorKubernetesCredentialsFeatureStep.scala | 43 ++ .../cluster/k8s/KubernetesExecutorBuilder.scala| 9 - .../k8s/KubernetesExecutorBuilderSuite.scala | 9 - .../k8s/integrationtest/BasicTestsSuite.scala | 7 .../k8s/integrationtest/KubernetesSuite.scala | 4 ++ 8 files changed, 87 insertions(+), 16 deletions(-) diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala index c7338a7..02471af 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala @@ -61,10 +61,9 @@ private[spark] object Config extends Logging { .stringConf .createOptional - val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = - "spark.kubernetes.authenticate.driver" - val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = - "spark.kubernetes.authenticate.driver.mounted" + val KUBERNETES_AUTH_DRIVER_CONF_PREFIX = "spark.kubernetes.authenticate.driver" + val KUBERNETES_AUTH_EXECUTOR_CONF_PREFIX = "spark.kubernetes.authenticate.executor" + val KUBERNETES_AUTH_DRIVER_MOUNTED_CONF_PREFIX = "spark.kubernetes.authenticate.driver.mounted" val KUBERNETES_AUTH_CLIENT_MODE_PREFIX = "spark.kubernetes.authenticate" val OAUTH_TOKEN_CONF_SUFFIX = "oauthToken" val OAUTH_TOKEN_FILE_CONF_SUFFIX = "oauthTokenFile" diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala index 588cd9d..9e32d6e 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala @@ -16,6 +16,8 @@ */ package org.apache.spark.deploy.k8s +import io.fabric8.kubernetes.api.model.{Container, ContainerBuilder, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, ContainerStatus, Pod, PodBuilder} + import org.apache.spark.SparkConf import org.apache.spark.util.Utils @@ -60,4 +62,15 @@ private[spark] object KubernetesUtils { } def parseMasterUrl(url: String): String = url.substring("k8s://".length) + + def buildPodWithServiceAccount(serviceAccount: Option[String], pod: SparkPod): Option[Pod] = { +serviceAccount.map { account => + new PodBuilder(pod.pod) +.editOrNewSpec() + .withServiceAccount(account) + .withServiceAccountName(account) +.endSpec() +.build() +} + } } diff --git a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala index ff5ad66..6079381 100644 --- a/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala +++ b/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverKubernetesCredentialsFeatureStep.scala @@ -27,6 +27,7 @@ import io.fabric8.kubernetes.api.model.{ContainerBuilder, HasMetadata, PodBuilde import org.apache.spark.deploy.k8s.{KubernetesConf, SparkPod} import org.apache.spark.deploy.k8s.Config._ import org.apache.spark.deploy.k8s.Constants._ +import org.apache.spark.deploy.k8s.KubernetesUtils.buildPodWithServiceAccount private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: KubernetesConf[_]) extends KubernetesFeatureConfigStep { @@ -70,15 +71,7 @@ private[spark] class DriverKubernetesCredentialsFeatureStep(kubernetesConf: Kube o
[spark] branch master updated (50b2a49 -> 4af1ac9)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue add 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/internal/SQLConf.scala | 11 +++ .../jdbc/connection/BasicConnectionProvider.scala | 2 ++ .../datasources/jdbc/connection/ConnectionProvider.scala | 7 +-- .../jdbc/connection/DB2ConnectionProvider.scala | 2 ++ .../jdbc/connection/MSSQLConnectionProvider.scala | 2 ++ .../jdbc/connection/MariaDBConnectionProvider.scala | 2 ++ .../jdbc/connection/OracleConnectionProvider.scala| 2 ++ .../jdbc/connection/PostgresConnectionProvider.scala | 2 ++ .../apache/spark/sql/jdbc/JdbcConnectionProvider.scala| 6 ++ .../jdbc/connection/ConnectionProviderSuite.scala | 15 +-- .../jdbc/connection/ConnectionProviderSuiteBase.scala | 2 +- .../IntentionallyFaultyConnectionProvider.scala | 1 + 12 files changed, 49 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (50b2a49 -> 4af1ac9)
This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 50b2a49 [SPARK-21708][BUILD][FOLLOWUP] Rename hdpVersion to hadoopVersionValue add 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility No new revisions were added by this update. Summary of changes: .../scala/org/apache/spark/sql/internal/SQLConf.scala | 11 +++ .../jdbc/connection/BasicConnectionProvider.scala | 2 ++ .../datasources/jdbc/connection/ConnectionProvider.scala | 7 +-- .../jdbc/connection/DB2ConnectionProvider.scala | 2 ++ .../jdbc/connection/MSSQLConnectionProvider.scala | 2 ++ .../jdbc/connection/MariaDBConnectionProvider.scala | 2 ++ .../jdbc/connection/OracleConnectionProvider.scala| 2 ++ .../jdbc/connection/PostgresConnectionProvider.scala | 2 ++ .../apache/spark/sql/jdbc/JdbcConnectionProvider.scala| 6 ++ .../jdbc/connection/ConnectionProviderSuite.scala | 15 +-- .../jdbc/connection/ConnectionProviderSuiteBase.scala | 2 +- .../IntentionallyFaultyConnectionProvider.scala | 1 + 12 files changed, 49 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4af1ac9 -> 543d59d)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility add 543d59d [SPARK-33107][BUILD][FOLLOW-UP] Remove com.twitter:parquet-hadoop-bundle:1.6.0 and orc.classifier No new revisions were added by this update. Summary of changes: examples/pom.xml | 5 - pom.xml | 12 sql/core/pom.xml | 2 -- sql/hive/pom.xml | 5 - 4 files changed, 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4af1ac9 -> 543d59d)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility add 543d59d [SPARK-33107][BUILD][FOLLOW-UP] Remove com.twitter:parquet-hadoop-bundle:1.6.0 and orc.classifier No new revisions were added by this update. Summary of changes: examples/pom.xml | 5 - pom.xml | 12 sql/core/pom.xml | 2 -- sql/hive/pom.xml | 5 - 4 files changed, 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4af1ac9 -> 543d59d)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility add 543d59d [SPARK-33107][BUILD][FOLLOW-UP] Remove com.twitter:parquet-hadoop-bundle:1.6.0 and orc.classifier No new revisions were added by this update. Summary of changes: examples/pom.xml | 5 - pom.xml | 12 sql/core/pom.xml | 2 -- sql/hive/pom.xml | 5 - 4 files changed, 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4af1ac9 -> 543d59d)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility add 543d59d [SPARK-33107][BUILD][FOLLOW-UP] Remove com.twitter:parquet-hadoop-bundle:1.6.0 and orc.classifier No new revisions were added by this update. Summary of changes: examples/pom.xml | 5 - pom.xml | 12 sql/core/pom.xml | 2 -- sql/hive/pom.xml | 5 - 4 files changed, 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
[spark] branch master updated (4af1ac9 -> 543d59d)
This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 4af1ac9 [SPARK-32047][SQL] Add JDBC connection provider disable possibility add 543d59d [SPARK-33107][BUILD][FOLLOW-UP] Remove com.twitter:parquet-hadoop-bundle:1.6.0 and orc.classifier No new revisions were added by this update. Summary of changes: examples/pom.xml | 5 - pom.xml | 12 sql/core/pom.xml | 2 -- sql/hive/pom.xml | 5 - 4 files changed, 24 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org