Associating spark jobs with logs

Hello Spark experts! I was wondering if somebody has solved the problem which we are facing. We want to achieve the following: Given a spark job id fetch all the logs generated by that job. We looked at spark job server it seems to be lacking such a feature. Any ideas, suggestions are

Re: Associating spark jobs with logs

Have you looked at the REST API section of: https://spark.apache.org/docs/latest/monitoring.html FYI On Tue, Dec 8, 2015 at 8:57 AM, sunil m <260885smanik...@gmail.com> wrote: > Hello Spark experts! > > I was wondering if somebody has solved the problem which we are facing. > > We want to

Re: Associating spark jobs with logs

Thanks for replying ... Yes i did. I am not seeing the application-ids for jobs submitted to YARN when i query http://MY_HOST:18080/api/v1/applications/ When I query http://MY_HOST:18080/api/v1/applications/application_1446812769803_0011 it does not understand the application_id since it belongs