Re: Pausing spark kafka streaming (direct) or exclude/include some partitions on the fly per batch

2016-09-03 Thread Cody Koeninger
g kafka consumer pluggable, so you may be able to wrap a >> consumer to do what you need. >> >> On Fri, Sep 2, 2016 at 12:28 PM, sagarcasual . <sagarcas...@gmail.com> >> wrote: >> > Hello, this is for >> > Pausing spark kafka streaming (direct) or exclude/includ

Re: Pausing spark kafka streaming (direct) or exclude/include some partitions on the fly per batch

2016-09-02 Thread sagarcasual .
is for > > Pausing spark kafka streaming (direct) or exclude/include some > partitions on > > the fly per batch > > = > > I have following code that creates a direct stream using Kafka connector > for > >

Re: Pausing spark kafka streaming (direct) or exclude/include some partitions on the fly per batch

2016-09-02 Thread Cody Koeninger
a consumer to do what you need. On Fri, Sep 2, 2016 at 12:28 PM, sagarcasual . <sagarcas...@gmail.com> wrote: > Hello, this is for > Pausing spark kafka streaming (direct) or exclude/include some partitions on > the fly per batch >

Pausing spark kafka streaming (direct) or exclude/include some partitions on the fly per batch

2016-09-02 Thread sagarcasual .
Hello, this is for Pausing spark kafka streaming (direct) or exclude/include some partitions on the fly per batch = I have following code that creates a direct stream using Kafka connector for Spark. final JavaInputDStream msgRecords