Re: [PySpark] Failed to add file [file:///tmp/app-submodules.zip] specified in 'spark.submit.pyFiles' to Python path:

2023-08-09 Thread lnxpgn
which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 9 Aug 2023 at 13:41, lnxpgn wrote: Hi, I am using Spark 3.4.1, running o

[PySpark] Failed to add file [file:///tmp/app-submodules.zip] specified in 'spark.submit.pyFiles' to Python path:

2023-08-09 Thread lnxpgn
Hi, I am using Spark 3.4.1, running on YARN. Hadoop runs on a single-node in a pseudo-distributed mode. spark-submit --master yarn --deploy-mode cluster --py-files /tmp/app-submodules.zip app.py The YARN application ran successfully, but have a warning log message: