Hi guys,

I try to use Spark 3 on top of Kubernetes and to specify a pod template for
the driver.

Here is my pod manifest or the driver and when I do a spark-submit with the
option:
--conf
spark.kubernetes.driver.podTemplateFile=/data/k8s/podtemplate_driver3.yaml

I got the error message that I need to specify an image, but it's the
manifest.
Does my manifest file is wrong, How should it look like?

Thanks for your help,
Michel

--------
The pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: mySpark3App
  labels:
    app: mySpark3App
    customlabel/app-id: "1"
spec:
  securityContext:
    runAsUser: 1000
  volumes:
    - name: "test-volume"
      emptyDir: {}
  containers:
    - name: spark3driver
      image: mydockerregistry.example.com/images/dev/spark3:latest
      instances: 1
      resources:
        requests:
          cpu: "1000m"
          memory: "512Mi"
        limits:
          cpu: "1000m"
          memory: "512Mi"
      volumeMounts:
       - name: "test-volume"
         mountPath: "/tmp"

Reply via email to