Re: Timed out while stopping the job generator plus subsequent failures

2015-03-12 Thread Sean Owen
I don't think that's the same issue I was seeing, but you can have a look at https://issues.apache.org/jira/browse/SPARK-4545 for more detail on my issue. On Thu, Mar 12, 2015 at 12:51 AM, Tobias Pfeiffer t...@preferred.jp wrote: Sean, On Wed, Mar 11, 2015 at 7:43 PM, Tobias Pfeiffer

Re: Timed out while stopping the job generator plus subsequent failures

2015-03-11 Thread Tobias Pfeiffer
Sean, On Wed, Mar 11, 2015 at 7:43 PM, Tobias Pfeiffer t...@preferred.jp wrote: it seems like I am unable to shut down my StreamingContext properly, both in local[n] and yarn-cluster mode. In addition, (only) in yarn-cluster mode, subsequent use of a new StreamingContext will raise an

Re: Timed out while stopping the job generator plus subsequent failures

2015-03-11 Thread Tobias Pfeiffer
Hi, I discovered what caused my issue when running on YARN and was able to work around it. On Wed, Mar 11, 2015 at 7:43 PM, Tobias Pfeiffer t...@preferred.jp wrote: The processing itself is complete, i.e., the batch currently processed at the time of stop() is finished and no further batches