Re: File Not Found: /tmp/spark-events in Spark 3.0

[Xin Jinhan]

Hi, 

First, the '/tmp/spark-events' is the default storage location of spark
eventLog, but the log will be stored in it only when the
'spark.eventLog.enabled' is true, which your spark 2.4.6 may set to false.
So you can try to set false and the error may disappear.

Second, I suggest enable eventLog and you can set the storage location by
set  'spark.eventLog.dir' to a fileSystem or local path, in case you want to
check the log later.(can simplely use spark-history-server)

Regards,
Jinhan



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: File Not Found: /tmp/spark-events in Spark 3.0

[Xin Jinhan]

Hi,

First, the /tmp/spark-events is the default storage location of spark
eventLog, but the log is stored only when you set the
'spark.eventLog.enabled=true', which maybe your spark 2.4.6 set to false. So
you can just set it to false and the error will disappear. 
 
Second, I suggest to open the eventLog and you can specify the log location
with 'spark.eventLog.dir' either a filesystem or local one, because you
maybe to check the log later.(can simplely use spark-history-server)

Regards
Jinhan



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org