[
https://issues.apache.org/jira/browse/SPARK-24894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554602#comment-16554602
]
Yinan Li commented on SPARK-24894:
----------------------------------
[~mcheah]. We need to make sure the truncation leads to a valid hostname.
> Invalid DNS name due to hostname truncation
> --------------------------------------------
>
> Key: SPARK-24894
> URL: https://issues.apache.org/jira/browse/SPARK-24894
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 2.3.1
> Reporter: Dharmesh Kakadia
> Priority: Major
>
> The truncation for hostname happening here
> [https://github.com/apache/spark/blob/5ff1b9ba1983d5601add62aef64a3e87d07050eb/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala#L77]
> is a problematic and can lead to DNS names starting with "-".
> Originally filled here :
> https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/229
> ```
> {{2018-07-23 21:21:42 ERROR Utils:91 - Uncaught exception in thread
> kubernetes-pod-allocator
> io.fabric8.kubernetes.client.KubernetesClientException: Failure executing:
> POST at: https://kubernetes.default.svc/api/v1/namespaces/default/pods.
> Message: Pod
> "user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9" is
> invalid: spec.hostname: Invalid value:
> "-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a DNS-1123
> label must consist of lower case alphanumeric characters or '-', and must
> start and end with an alphanumeric character (e.g. 'my-name', or '123-abc',
> regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'). Received
> status: Status(apiVersion=v1, code=422,
> details=StatusDetails(causes=[StatusCause(field=spec.hostname,
> message=Invalid value:
> "-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a DNS-1123
> label must consist of lower case alphanumeric characters or '-', and must
> start and end with an alphanumeric character (e.g. 'my-name', or '123-abc',
> regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'),
> reason=FieldValueInvalid, additionalProperties={})], group=null, kind=Pod,
> name=user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9,
> retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status,
> message=Pod
> "user-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9" is
> invalid: spec.hostname: Invalid value:
> "-archetypes-all-weekly-1532380861251850404-1532380862321-exec-9": a DNS-1123
> label must consist of lower case alphanumeric characters or '-', and must
> start and end with an alphanumeric character (e.g. 'my-name', or '123-abc',
> regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?'),
> metadata=ListMeta(resourceVersion=null, selfLink=null,
> additionalProperties={}), reason=Invalid, status=Failure,
> additionalProperties={}). at
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:470)
> at
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:409)
> at
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
> at
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:343)
> at
> io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:226)
> at
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:769)
> at
> io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:356)
> at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3$$anonfun$apply$3.apply(KubernetesClusterSchedulerBackend.scala:140)
> at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3$$anonfun$apply$3.apply(KubernetesClusterSchedulerBackend.scala:140)
> at org.apache.spark.util.Utils$.tryLog(Utils.scala:1922) at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3.apply(KubernetesClusterSchedulerBackend.scala:139)
> at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1$$anonfun$3.apply(KubernetesClusterSchedulerBackend.scala:138)
> at
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> at
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> at
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
> at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) at
> scala.collection.mutable.HashMap.foreach(HashMap.scala:99) at
> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
> at scala.collection.MapLike$MappedValues.foreach(MapLike.scala:245) at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:234) at
> scala.collection.AbstractTraversable.map(Traversable.scala:104) at
> org.apache.spark.scheduler.cluster.k8s.KubernetesClusterSchedulerBackend$$anon$1.run(KubernetesClusterSchedulerBackend.scala:145)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)}}
> ```
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
