On top of what's been said... On Wed, Apr 22, 2015 at 10:48 PM, ÐΞ€ρ@Ҝ (๏̯͡๏) <deepuj...@gmail.com> wrote: > 1) I can go to Spark UI and see the status of the APP but cannot see the > logs as the job progresses. How can i see logs of executors as they progress > ?
Spark 1.3 should have links to the executor logs in the UI while the application is running. Not yet in the history server, though. > 2) In case the App fails/completes then Spark UI vanishes and i get a YARN > Job page that says job failed, there are no link to Spark UI again. Now i > take the job ID and run yarn application logs appid and my console fills up > (with huge scrolling) with logs of all executors. Then i copy and paste into > a text editor and search for keywords "Exception" , "Job aborted due to ". > Is this the right way to view logs ? Aside from Ted's suggestion, you could also pipe the "yarn logs" output to "less". -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
