I'm using the Kafka direct stream api but I can have a look on extending it to have this behaviour
Thanks! On 11 Feb 2016 9:07 p.m., "Shixiong(Ryan) Zhu" <shixi...@databricks.com> wrote: > Are you using a custom input dstream? If so, you can make the `compute` > method return None to skip a batch. > > On Thu, Feb 11, 2016 at 1:03 PM, Sebastian Piu <sebastian....@gmail.com> > wrote: > >> I was wondering if there is there any way to skip batches with zero >> events when streaming? >> By skip I mean avoid the empty rdd from being created at all? >> > >
