[jira] [Commented] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace
[
https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705965#comment-17705965
]
Dongjoon Hyun commented on SPARK-41006:
---
Thank you for reporting and pinging me, [~dhkold]. Let me take a look at your
PR.
> ConfigMap has the same name when launching two pods on the same namespace
> -
>
> Key: SPARK-41006
> URL: https://issues.apache.org/jira/browse/SPARK-41006
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
>Affects Versions: 3.1.0, 3.2.0, 3.3.0
>Reporter: Eric
>Priority: Minor
>
> If we use the Spark Launcher to launch our spark apps in k8s:
> {code:java}
> val sparkLauncher = new InProcessLauncher()
> .setMaster(k8sMaster)
> .setDeployMode(deployMode)
> .setAppName(appName)
> .setVerbose(true)
> sparkLauncher.startApplication(new SparkAppHandle.Listener { ...{code}
> We have an issue when we launch another spark driver in the same namespace
> where other spark app was running:
> {code:java}
> kp -n audit-exporter-eee5073aac -w
> NAME READY STATUS RESTARTS AGE
> audit-exporter-71489e843d8085c0-driver 1/1 Running 0
> 9m54s
> audit-exporter-7e6b8b843d80b9e6-exec-1 1/1 Running 0
> 9m40s
> data-io-120204843d899567-driver 0/1 Terminating 0 1s
> data-io-120204843d899567-driver 0/1 Terminating 0 2s
> data-io-120204843d899567-driver 0/1 Terminating 0 3s
> data-io-120204843d899567-driver 0/1 Terminating 0
> 3s{code}
> The error is:
> {code:java}
> {"time":"2022-11-03T12:49:45.626Z","lvl":"WARN","logger":"o.a.s.l.InProcessAppHandle","thread":"spark-app-38:
> 'data-io'","msg":"Application failed with
> exception.","stack_trace":"io.fabric8.kubernetes.client.KubernetesClientException:
> Failure executing: PUT at:
> https://kubernetes.default/api/v1/namespaces/audit-exporter-eee5073aac/configmaps/spark-drv-d19c37843d80350c-conf-map.
> Message: ConfigMap \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data:
> Forbidden: field is immutable when `immutable` is set. Received status:
> Status(apiVersion=v1, code=422,
> details=StatusDetails(causes=[StatusCause(field=data, message=Forbidden:
> field is immutable when `immutable` is set, reason=FieldValueForbidden,
> additionalProperties={})], group=null, kind=ConfigMap,
> name=spark-drv-d19c37843d80350c-conf-map, retryAfterSeconds=null, uid=null,
> additionalProperties={}), kind=Status, message=ConfigMap
> \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data: Forbidden: field is
> immutable when `immutable` is set, metadata=ListMeta(_continue=null,
> remainingItemCount=null, resourceVersion=null, selfLink=null,
> additionalProperties={}), reason=Invalid, status=Failure,
> additionalProperties={}).\n\tat
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:682)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:661)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:555)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:518)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:342)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:322)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:649)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:195)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$$Lambda$5360/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:200)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:141)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation$$Lambda$4618/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.replace(CreateOrReplaceHelper.java:69)\n\tat
>
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:61)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:318)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:83)\n\tat
>
>
[jira] [Commented] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace
[
https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17705918#comment-17705918
]
Cedric van Eetvelde commented on SPARK-41006:
-
Anyone to check this? [~dongjoon] (Sorry if tagging the wrong person, I don't
know who I can tag)
> ConfigMap has the same name when launching two pods on the same namespace
> -
>
> Key: SPARK-41006
> URL: https://issues.apache.org/jira/browse/SPARK-41006
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
>Affects Versions: 3.1.0, 3.2.0, 3.3.0
>Reporter: Eric
>Priority: Minor
>
> If we use the Spark Launcher to launch our spark apps in k8s:
> {code:java}
> val sparkLauncher = new InProcessLauncher()
> .setMaster(k8sMaster)
> .setDeployMode(deployMode)
> .setAppName(appName)
> .setVerbose(true)
> sparkLauncher.startApplication(new SparkAppHandle.Listener { ...{code}
> We have an issue when we launch another spark driver in the same namespace
> where other spark app was running:
> {code:java}
> kp -n audit-exporter-eee5073aac -w
> NAME READY STATUS RESTARTS AGE
> audit-exporter-71489e843d8085c0-driver 1/1 Running 0
> 9m54s
> audit-exporter-7e6b8b843d80b9e6-exec-1 1/1 Running 0
> 9m40s
> data-io-120204843d899567-driver 0/1 Terminating 0 1s
> data-io-120204843d899567-driver 0/1 Terminating 0 2s
> data-io-120204843d899567-driver 0/1 Terminating 0 3s
> data-io-120204843d899567-driver 0/1 Terminating 0
> 3s{code}
> The error is:
> {code:java}
> {"time":"2022-11-03T12:49:45.626Z","lvl":"WARN","logger":"o.a.s.l.InProcessAppHandle","thread":"spark-app-38:
> 'data-io'","msg":"Application failed with
> exception.","stack_trace":"io.fabric8.kubernetes.client.KubernetesClientException:
> Failure executing: PUT at:
> https://kubernetes.default/api/v1/namespaces/audit-exporter-eee5073aac/configmaps/spark-drv-d19c37843d80350c-conf-map.
> Message: ConfigMap \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data:
> Forbidden: field is immutable when `immutable` is set. Received status:
> Status(apiVersion=v1, code=422,
> details=StatusDetails(causes=[StatusCause(field=data, message=Forbidden:
> field is immutable when `immutable` is set, reason=FieldValueForbidden,
> additionalProperties={})], group=null, kind=ConfigMap,
> name=spark-drv-d19c37843d80350c-conf-map, retryAfterSeconds=null, uid=null,
> additionalProperties={}), kind=Status, message=ConfigMap
> \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data: Forbidden: field is
> immutable when `immutable` is set, metadata=ListMeta(_continue=null,
> remainingItemCount=null, resourceVersion=null, selfLink=null,
> additionalProperties={}), reason=Invalid, status=Failure,
> additionalProperties={}).\n\tat
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:682)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:661)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:555)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:518)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:342)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:322)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:649)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:195)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$$Lambda$5360/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:200)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:141)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation$$Lambda$4618/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.replace(CreateOrReplaceHelper.java:69)\n\tat
>
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:61)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:318)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:83)\n\tat
>
[jira] [Commented] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace
[
https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703145#comment-17703145
]
Cedric van Eetvelde commented on SPARK-41006:
-
As mentionned above, I create a Pull Request with the correction. I updated the
Unit Tests accordingly.
> ConfigMap has the same name when launching two pods on the same namespace
> -
>
> Key: SPARK-41006
> URL: https://issues.apache.org/jira/browse/SPARK-41006
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
>Affects Versions: 3.1.0, 3.2.0, 3.3.0
>Reporter: Eric
>Priority: Minor
>
> If we use the Spark Launcher to launch our spark apps in k8s:
> {code:java}
> val sparkLauncher = new InProcessLauncher()
> .setMaster(k8sMaster)
> .setDeployMode(deployMode)
> .setAppName(appName)
> .setVerbose(true)
> sparkLauncher.startApplication(new SparkAppHandle.Listener { ...{code}
> We have an issue when we launch another spark driver in the same namespace
> where other spark app was running:
> {code:java}
> kp -n audit-exporter-eee5073aac -w
> NAME READY STATUS RESTARTS AGE
> audit-exporter-71489e843d8085c0-driver 1/1 Running 0
> 9m54s
> audit-exporter-7e6b8b843d80b9e6-exec-1 1/1 Running 0
> 9m40s
> data-io-120204843d899567-driver 0/1 Terminating 0 1s
> data-io-120204843d899567-driver 0/1 Terminating 0 2s
> data-io-120204843d899567-driver 0/1 Terminating 0 3s
> data-io-120204843d899567-driver 0/1 Terminating 0
> 3s{code}
> The error is:
> {code:java}
> {"time":"2022-11-03T12:49:45.626Z","lvl":"WARN","logger":"o.a.s.l.InProcessAppHandle","thread":"spark-app-38:
> 'data-io'","msg":"Application failed with
> exception.","stack_trace":"io.fabric8.kubernetes.client.KubernetesClientException:
> Failure executing: PUT at:
> https://kubernetes.default/api/v1/namespaces/audit-exporter-eee5073aac/configmaps/spark-drv-d19c37843d80350c-conf-map.
> Message: ConfigMap \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data:
> Forbidden: field is immutable when `immutable` is set. Received status:
> Status(apiVersion=v1, code=422,
> details=StatusDetails(causes=[StatusCause(field=data, message=Forbidden:
> field is immutable when `immutable` is set, reason=FieldValueForbidden,
> additionalProperties={})], group=null, kind=ConfigMap,
> name=spark-drv-d19c37843d80350c-conf-map, retryAfterSeconds=null, uid=null,
> additionalProperties={}), kind=Status, message=ConfigMap
> \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data: Forbidden: field is
> immutable when `immutable` is set, metadata=ListMeta(_continue=null,
> remainingItemCount=null, resourceVersion=null, selfLink=null,
> additionalProperties={}), reason=Invalid, status=Failure,
> additionalProperties={}).\n\tat
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:682)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:661)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:555)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:518)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:342)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:322)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:649)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:195)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$$Lambda$5360/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:200)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:141)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation$$Lambda$4618/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.replace(CreateOrReplaceHelper.java:69)\n\tat
>
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:61)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:318)\n\tat
>
>
[jira] [Commented] (SPARK-41006) ConfigMap has the same name when launching two pods on the same namespace
[
https://issues.apache.org/jira/browse/SPARK-41006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17702710#comment-17702710
]
Apache Spark commented on SPARK-41006:
--
User 'DHKold' has created a pull request for this issue:
https://github.com/apache/spark/pull/40491
> ConfigMap has the same name when launching two pods on the same namespace
> -
>
> Key: SPARK-41006
> URL: https://issues.apache.org/jira/browse/SPARK-41006
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
>Affects Versions: 3.1.0, 3.2.0, 3.3.0
>Reporter: Eric
>Priority: Minor
>
> If we use the Spark Launcher to launch our spark apps in k8s:
> {code:java}
> val sparkLauncher = new InProcessLauncher()
> .setMaster(k8sMaster)
> .setDeployMode(deployMode)
> .setAppName(appName)
> .setVerbose(true)
> sparkLauncher.startApplication(new SparkAppHandle.Listener { ...{code}
> We have an issue when we launch another spark driver in the same namespace
> where other spark app was running:
> {code:java}
> kp -n audit-exporter-eee5073aac -w
> NAME READY STATUS RESTARTS AGE
> audit-exporter-71489e843d8085c0-driver 1/1 Running 0
> 9m54s
> audit-exporter-7e6b8b843d80b9e6-exec-1 1/1 Running 0
> 9m40s
> data-io-120204843d899567-driver 0/1 Terminating 0 1s
> data-io-120204843d899567-driver 0/1 Terminating 0 2s
> data-io-120204843d899567-driver 0/1 Terminating 0 3s
> data-io-120204843d899567-driver 0/1 Terminating 0
> 3s{code}
> The error is:
> {code:java}
> {"time":"2022-11-03T12:49:45.626Z","lvl":"WARN","logger":"o.a.s.l.InProcessAppHandle","thread":"spark-app-38:
> 'data-io'","msg":"Application failed with
> exception.","stack_trace":"io.fabric8.kubernetes.client.KubernetesClientException:
> Failure executing: PUT at:
> https://kubernetes.default/api/v1/namespaces/audit-exporter-eee5073aac/configmaps/spark-drv-d19c37843d80350c-conf-map.
> Message: ConfigMap \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data:
> Forbidden: field is immutable when `immutable` is set. Received status:
> Status(apiVersion=v1, code=422,
> details=StatusDetails(causes=[StatusCause(field=data, message=Forbidden:
> field is immutable when `immutable` is set, reason=FieldValueForbidden,
> additionalProperties={})], group=null, kind=ConfigMap,
> name=spark-drv-d19c37843d80350c-conf-map, retryAfterSeconds=null, uid=null,
> additionalProperties={}), kind=Status, message=ConfigMap
> \"spark-drv-d19c37843d80350c-conf-map\" is invalid: data: Forbidden: field is
> immutable when `immutable` is set, metadata=ListMeta(_continue=null,
> remainingItemCount=null, resourceVersion=null, selfLink=null,
> additionalProperties={}), reason=Invalid, status=Failure,
> additionalProperties={}).\n\tat
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:682)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:661)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:612)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:555)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:518)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:342)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleUpdate(OperationSupport.java:322)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleUpdate(BaseOperation.java:649)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.lambda$replace$1(HasMetadataOperation.java:195)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation$$Lambda$5360/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:200)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.HasMetadataOperation.replace(HasMetadataOperation.java:141)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation$$Lambda$4618/00.apply(Unknown
> Source)\n\tat
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.replace(CreateOrReplaceHelper.java:69)\n\tat
>
> io.fabric8.kubernetes.client.utils.CreateOrReplaceHelper.createOrReplace(CreateOrReplaceHelper.java:61)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:318)\n\tat
>
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.createOrReplace(BaseOperation.java:83)\n\tat
>
>
