Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread John Zeng
/tmp/logs/root/logs/application_1501197841826_0013does not exist. Log aggregation has not completed or is not enabled. Any other way to see my logs? Thanks John From: ayan guha <guha.a...@gmail.com> Sent: Sunday, July 30, 2017 10:34 PM To: John Zeng; Ri

Re: Logging in RDD mapToPair of Java Spark application

2017-07-30 Thread John Zeng
Thanks Riccardo for the valuable info. Following your guidance, I looked at the Spark UI and figured out the default logs location for executors is 'yarn/container-logs'. I ran my Spark app again and I can see a new folder was created for it: [root@john2 application_1501197841826_0013]# ls