Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS resolution of driver fails

2019-06-18 Thread Prudhvi Chennuru (CONT)
019 at 10:06 > *To: *"Prudhvi Chennuru (CONT)" > *Cc: *Li Gao , dev , user < > user@spark.apache.org> > *Subject: *Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS > resolution of driver fails > > > > Hi Prudhvi, > > not really but we took a dras

Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS resolution of driver fails

2019-06-18 Thread Olivier Girardot
Hi Prudhvi, not really but we took a drastic approach mitigating this, modifying the bundled launch script to be more resilient. In the kubernetes/dockerfiles/spark/entrypoint.sh in the executor case we added something like that : executor) DRIVER_HOST=$(echo $SPARK_DRIVER_URL | cut -d "@"

Re: [External Sender] Re: Spark 2.4.1 on Kubernetes - DNS resolution of driver fails

2019-06-11 Thread Prudhvi Chennuru (CONT)
Hey Oliver, I am also facing the same issue on my kubernetes cluster(v1.11.5) on AWS with spark version 2.3.3, any luck in figuring out the root cause? On Fri, May 3, 2019 at 5:37 AM Olivier Girardot < o.girar...@lateral-thoughts.com> wrote: > Hi, > I did not try on another