Which version of spark? and what is your data source? For some reason, your processing delay is exceeding the batch duration. And its strange that you are not seeing any scheduling delay.
Thanks Best Regards On Thu, Jun 18, 2015 at 7:29 AM, Mike Fang <chyfan...@gmail.com> wrote: > Hi, > > > > I have a spark streaming program running for ~ 25hrs. When I check the > Streaming UI tab. I found the “Waiting batches” is 144. But the “scheduling > delay” is 0. I am a bit confused. > > If the “waiting batches” is 144, that means many batches are waiting in > the queue to be processed? If this is the case, the scheduling delay should > be high rather than 0. Am I missing anything? > > > > Thanks, > > Mike >
