Hi, I am trying to do some logging in my PySpark jobs, particularly in map that is performed on workers. Unfortunately I am not able tofind these logs. Based on the documentation it seems that the logs should be on masters in the SPARK_KOME, directory work http://spark.apache.org/docs/latest/spark-standalone.html#monitoring-and-logging . But unfortunately I am can't see this directory.
So I would like to ask, how do you do logging on workers? Is it somehow possible to send this logs also to master, for example after the job fininshes? Than you in advance for any suggestions and advices. Best regards, Jan --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
