Re: Spark with Kubernetes connecting to pod ID, not address

2019-02-13 Thread Pat Ferrel
Hmm, I’m not asking about using k8s to control Spark as a Job manager or scheduler like Yarn. We use the built-in standalone Spark Job Manager and sparl://spark-api:7077 as the master not k8s. The problem is using k8s to manage a cluster consisting of our app, some databases, and Spark (one

Spark with Kubernetes connecting to pod id, not address

2019-02-12 Thread Pat Ferrel
From: Pat Ferrel Reply: Pat Ferrel Date: February 12, 2019 at 5:40:41 PM To: user@spark.apache.org Subject:  Spark with Kubernetes connecting to pod id, not address We have a k8s deployment of several services including Apache Spark. All services seem to be operational. Our application