While launching a spark job from Zeppelin against a standalone spark
cluster (Spark 3.0 with multiple workers without hadoop), we have
encountered a Spark interpreter exception caused by a I/O File Not Found
exception due to the non-existence of the /tmp/spark-events directory.
We had to create the /tmp/spark-events directory manually in order to
resolve the problem.
As a reference, the same notebook code run on Spark 2.4.6 (also a
standalone cluster) without any problems.
What is /tmp/spark-events for and is there anyway to pre-define this
directory as some config parameter so we don't end up manually add it in
/tmp?
Thanks!
-- ND
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
