[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919346#comment-16919346 ] Björn edited comment on SPARK-28444 at 8/30/19 9:07 AM: We're running into the same issue. As I'm developing with a local ansible/Vagrant setup (Kubernetes deployed through kubeadm, 3 nodes) I did some testings on different versions with SparkPi example (Spark 2.4.3). My results were: * spark-submit in cluster and client mode works fine in local docker-desktop running Kubernetes 1.14.3 * spark-submit in client mode works fine for Kubernetes 1.15.3 in the Vagrant multinode cluster * spark-submit in cluster mode does not work for Kubernetes 1.15.3, 1.14.3,1.13.10 in the Vagrant multinode cluster spark-submit in cluster mode starts the driver, spawns the executors but fails when trying to watch the pod with HTTP 403 Exception with an empty message (esp. not complaining about permissions). The log is more or less the same as posted above. I think neither the compatibility nor the permissions (as executor pods can be created with the service account) are the cause for this. Does anyone have ideas how to further debug this? was (Author: dolkemeier): We're running into the same issue. As I'm developing with a local ansible/Vagrant setup (Kubernetes deployed through kubeadm, 3 nodes) I did some testings on different versions with SparkPi example (Spark 2.4.3). My results were: * spark-submit in cluster and client mode works fine in local docker-desktop running Kubernetes 1.14.3 * spark-submit in client mode works fine for Kubernetes 1.15.3 in the Vagrant multinode cluster * spark-submit in cluster mode does not work for Kubernetes 1.15.3, 1.14.3,1.13.10 in the Vagrant multinode cluster spark-submit in cluster mode starts the driver, spawns the executors but fails when trying to watch the pod with HTTP 403 Exception with an empty message (esp. not complaining about permissions). The log is more or less the same as posted above. I think neither the compatibility nor the permissions (as executor pods can be created with the service account, spark sa has cluster role) are the cause for this. Does anyone have ideas how to further debug this? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888970#comment-16888970 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 3:26 PM: -- Right now on master we have 4.1.2 [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]. Afaik this is the same version for 2.4.3. Something else is not right. was (Author: skonto): Right now on master we have 4.1.2 [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]. Afaik this is the same version for 2.4.3. > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888970#comment-16888970 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 3:26 PM: -- Right now on master we have 4.1.2 [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]. Afaik this is the same version for 2.4.3. was (Author: skonto): Right now on master we have 4.1.2 [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]. Did you try 3.0.0? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888970#comment-16888970 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 3:25 PM: -- Right now on master we have [4.1.2 | [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]] was (Author: skonto): Right now on master we have [4.1.2 |[https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]] > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888970#comment-16888970 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 3:25 PM: -- Right now on master we have 4.1.2 [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]. Did you try 3.0.0? was (Author: skonto): Right now on master we have [4.1.2 | [https://github.com/apache/spark/blob/453cbf3dd8df5ec4da844c93eb6000610b551541/resource-managers/kubernetes/core/pom.xml#L32]] > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 1:46 PM: -- Am I not sure this is a k8s client version issue, it seems more like a credentials issue. But let's find out. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8io k8s client in different versions? Does it work with minikube 1.14? was (Author: skonto): Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8io k8s client in different versions? Does it work with minikube 1.14? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 1:46 PM: -- Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8io k8s client in different versions? Does it work with minikube 1.14? was (Author: skonto): Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8io k8s client in different versions? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 1:45 PM: -- Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8io k8s client in different versions? was (Author: skonto): Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8ios client in different versions? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 1:45 PM: -- Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can/cant create pods with a simple app (outside Spark) using the fabric8ios client in different versions? was (Author: skonto): Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can create pods with a simple app using the fabric8ios client in different versions? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1671#comment-1671 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 1:44 PM: -- Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? Can you verify you can create pods with a simple app using the fabric8ios client in different versions? was (Author: skonto): Am I not sure this is a k8s client version issue, it is more like a credentials issue. Have you tried to update the k8s client? > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1608#comment-1608 ] Stavros Kontopoulos edited comment on SPARK-28444 at 7/19/19 11:44 AM: --- Hi [~patrick-winter-swisscard]. On our ci we are using v1.15 and tests pass, could you add some log output showing why pods are not created. We need to be compliant with the compatibility matrix but still we dont have a good answer to the problem of catching up with k8s, it moves fast. was (Author: skonto): Hi [~patrick-winter-swisscard]. On our ci we are using v1.15 and tests pass, could you add some log output showing why pods are not created. We need to be compliant with the compatibility matrix but still we dotn have a good answer to the problem of catching up with k8s, it moves fast. > Bump Kubernetes Client Version to 4.3.0 > --- > > Key: SPARK-28444 > URL: https://issues.apache.org/jira/browse/SPARK-28444 > Project: Spark > Issue Type: Dependency upgrade > Components: Kubernetes >Affects Versions: 3.0.0, 2.4.3 >Reporter: Patrick Winter >Priority: Major > > Spark is currently using the Kubernetes client version 4.1.2. This client > does not support the current Kubernetes version 1.14, as can be seen on the > [compatibility > matrix|[https://github.com/fabric8io/kubernetes-client#compatibility-matrix]]. > Therefore the Kubernetes client should be bumped up to version 4.3.0. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
