Re: [Please Help] Log redirection on EMR

Hi Sabarish Thanks for your help, i was able to get the logs from archive, is there a way i can adjust archival policy, say i want to persist the logs of last 2 jobs on resource manage and archive others on the file system. On Mon, Feb 22, 2016 at 12:25 PM Sabarish Sasidharan <

Re: [Please Help] Log redirection on EMR

Your logs are getting archived in your logs bucket in S3. http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-debugging.html Regards Sab On Mon, Feb 22, 2016 at 12:14 PM, HARSH TAKKAR wrote: > Hi > > In am using an EMR cluster for running my

[Please Help] Log redirection on EMR

Hi In am using an EMR cluster for running my spark jobs, but after the job finishes logs disappear, I have added a log4j.properties in my jar, but all the logs still redirects to EMR resource manager which vanishes after jobs completes, is there a way i could redirect the logs to a location in