Hi, Spark users,
Following images are copied from spark streaming UI. I observed for about 30
minutes, and see that the Processed records(438768, at the moment I copied the
image) are always lagging behind Received records(480783) by about 40k records,
Since the waiting batches is 1 and the
Well, the scheduling delay is the time a batch has to wait for getting
resources. So even if there is no backlog in processing and scheduling
delay is 0, there is one batch that is being processed at any point of
time, which explains the difference.
On Tue, Jun 30, 2015 at 2:42 AM,
