trouble using spark in kubernetes

    - containerPort:  20022   name:  exposed     volumeMounts:     - name: data   mountPath: /data   volumes:     - name: data   persistentVolumeClaim:     claimName: spark-geodata-nfs-pvc-20220503   restartPolicy: Always I expose port 20022 of the pod with a headless service kubec

Re: Spark error with jupyter

I use jupyterlab and spark and I have not seen this before. Jupyter has a docker stack with pyspark you can try it. tor. 21. apr. 2022 kl. 11:07 skrev Wassim Yaich : > Hi Folks, > I am working

REMINDER - Travel Assistance available for ApacheCon NA New Orleans 2022

Hi All Contributors and Committers, This is a first reminder email that travel assistance applications for ApacheCon NA 2022 are now open! We will be supporting ApacheCon North America in New Orleans, Louisiana, on October 3rd through 6th, 2022. TAC exists to help those that would like to attend

Re: [EXTERNAL] Parse Execution Plan from PySpark

Hi Pablo, Do you mean an in-memory plan? You can access one by implementing a Spark Listener. Here is an example from the Datahub project [1]. If you end up parsing the SQL plan string, you may consider using/extending Coral [2, 3]. There is already a POC for that. See some test cases [4]. Thank

RE: [EXTERNAL] Parse Execution Plan from PySpark

Hi Pablo, As you probably know, Spark SQL generates custom Java code for the SQL functions. You can use geometry.debugCodegen() to print out the generated code. Shay From: Pablo Alcain Sent: Tuesday, May 3, 2022 6:07 AM To: user@spark.apache.org Subject: [EXTERNAL] Parse Execution Plan from Py