Re: spark streaming - checkpointing - looking at old application directory and failure to start streaming context

Any idea why this happens? On Wed, Jun 10, 2015 at 9:28 AM, Ashish Nigam ashnigamt...@gmail.com wrote: BTW, I am using spark streaming 1.2.0 version. On Wed, Jun 10, 2015 at 9:26 AM, Ashish Nigam ashnigamt...@gmail.com wrote: I did not change driver program. I just shutdown the context

spark streaming - checkpointing - looking at old application directory and failure to start streaming context

Hi, If checkpoint data is already present in HDFS, driver fails to load as it is performing lookup on previous application directory. As that folder already exists, it fails to start context. Failed job's application id was application_1432284018452_0635 and job was performing lookup on

Re: spark streaming - checkpointing - looking at old application directory and failure to start streaming context

I did not change driver program. I just shutdown the context and again started. BTW, I see this ticket already open in unassigned state - SPARK-6892 https://issues.apache.org/jira/browse/SPARK-6892 that talks about this issue. Is this a known issue? Also, any workarounds? On Wed, Jun 10,

Re: spark streaming - checkpointing - looking at old application directory and failure to start streaming context

Delete the checkpoint directory, you might have modified your driver program. Thanks Best Regards On Wed, Jun 10, 2015 at 9:44 PM, Ashish Nigam ashnigamt...@gmail.com wrote: Hi, If checkpoint data is already present in HDFS, driver fails to load as it is performing lookup on previous