Spark 1.6 and Application History not working correctly

2016-01-13 Thread Darin McBeath
I tried using Spark 1.6 in a stand-alone cluster this morning. I submitted 2 jobs (and they both executed fine). In fact, they are the exact same jobs with just some different parameters. I was able to view the application history for the first job. However, when I tried to view the second

Re: Spark 1.6 and Application History not working correctly

2016-01-13 Thread Steve Loughran
Cc: User <user@spark.apache.org> > Sent: Wednesday, January 13, 2016 10:10 AM > Subject: Re: Spark 1.6 and Application History not working correctly > > > > I noticed a similar problem going from 1.5.x to 1.6.0 on YARN. > > I resolved it be setting the fol

Re: Spark 1.6 and Application History not working correctly

2016-01-13 Thread Darin McBeath
Drake <dondr...@gmail.com> To: Darin McBeath <ddmcbe...@yahoo.com> Cc: User <user@spark.apache.org> Sent: Wednesday, January 13, 2016 10:10 AM Subject: Re: Spark 1.6 and Application History not working correctly I noticed a similar problem going from 1.5.x to 1.6.0 on

Re: Spark 1.6 and Application History not working correctly

2016-01-13 Thread Don Drake
I noticed a similar problem going from 1.5.x to 1.6.0 on YARN. I resolved it be setting the following command-line parameters: spark.eventLog.enabled=true spark.eventLog.dir= -Don On Wed, Jan 13, 2016 at 8:29 AM, Darin McBeath wrote: > I tried using Spark 1.6 in