How to access lost executor log file

Hi, there When running a Spark job on YARN, 2 executors somehow got lost during the execution. The message on the history server GUI is “CANNOT find address”. Two extra executors were launched by YARN and eventually finished the job. Usually I go to the “Executors” tab on the UI to check the

Re: How to access lost executor log file

Ted, Thanks for your reply. First of all, after sending email to the mailing list, I use yarn logs applicationId to retrieve the aggregated log successfully. I found the exceptions I am looking for. Now as to your suggestion, when I go to the YARN RM UI, I can only see the "Tracking URL" in

Re: How to access lost executor log file

Can you go to YARN RM UI to find all the attempts for this Spark Job ? The two lost executors should be found there. On Thu, Oct 1, 2015 at 10:30 AM, Lan Jiang wrote: > Hi, there > > When running a Spark job on YARN, 2 executors somehow got lost during the > execution. The

Re: How to access lost executor log file

Looks like the spark history server should take the lost exectuors into account by analyzing the output from 'yarn logs applicationId' command. Cheers On Thu, Oct 1, 2015 at 11:46 AM, Lan Jiang wrote: > Ted, > > Thanks for your reply. > > First of all, after sending email to