Re: Service Account not being honored using pyspark on Kubernetes

2020-01-29 Thread pisymbol .
On Wed, Jan 29, 2020 at 9:58 PM pisymbol . wrote: > > > On Wed, Jan 29, 2020 at 5:02 PM pisymbol . wrote: > >> >> The problem is when spark initailizes I see the following error: >> >> io.fabric8.kubernetes.client.KubernetesClientE

Re: Service Account not being honored using pyspark on Kubernetes

2020-01-29 Thread pisymbol .
On Wed, Jan 29, 2020 at 5:02 PM pisymbol . wrote: > > The problem is when spark initailizes I see the following error: > > io.fabric8.kubernetes.client.KubernetesClientException: pods is forbidden: > User "system:serviceaccount:default:default" cannot watch resourc

Service Account not being honored using pyspark on Kubernetes

2020-01-29 Thread pisymbol .
I am on k8s 1.17 in a small 4 node cluster. I am running Spark 2.4.4 but with updated kubernetes-client jars to work around the 403 CVE issue. I am running on a pod in the 'default' namespace of my cluster in a Jupyter notebook. I am trying to configure 'client mode' so I can use pyspark