6:05 PM
To: Mike Fang mailto:chyfan...@gmail.com>>
Cc: "user@spark.apache.org<mailto:user@spark.apache.org>"
mailto:user@spark.apache.org>>
Subject: Re: understanding on the "waiting batches" and "scheduling delay" in
Streaming UI
Which version of
Also, could you give a screenshot of the streaming UI. Even better, could
you run it on Spark 1.4 which has a new streaming UI and then use that for
debugging/screenshot?
TD
On Thu, Jun 18, 2015 at 3:05 AM, Akhil Das
wrote:
> Which version of spark? and what is your data source? For some reason
Which version of spark? and what is your data source? For some reason, your
processing delay is exceeding the batch duration. And its strange that you
are not seeing any scheduling delay.
Thanks
Best Regards
On Thu, Jun 18, 2015 at 7:29 AM, Mike Fang wrote:
> Hi,
>
>
>
> I have a spark streamin
Hi,
I have a spark streaming program running for ~ 25hrs. When I check the
Streaming UI tab. I found the “Waiting batches” is 144. But the “scheduling
delay” is 0. I am a bit confused.
If the “waiting batches” is 144, that means many batches are waiting in the
queue to be processed? If this is
