Yes, and I actually want to use high level api, i just didn't understand how kafka works inside. For some reason i thought in case of SimpleConsumer, kafka would implement locking and rebalancing on server side, letting client know that it can't attach to specific partition, etc. But i see now that all management should happen on the client.
Thanks. On Tue, Oct 1, 2013 at 8:48 AM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > I think you want locking to prevent two processes from consuming the same > partition. You can just use the high level consumer to get this group > management functionality. It ensures that a partition is only consumed by > one high level consumer at any given time. Is there any particular reason > why you want to use SimpleConsumer instead? > > Thanks, > Neha > > > On Tue, Oct 1, 2013 at 8:42 AM, Kane Kane <kane.ist...@gmail.com> wrote: > > > So essentially you can't do "queue" pattern, unless you somehow implement > > locking on the client? > > > > > > On Tue, Oct 1, 2013 at 8:35 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > SimpleConsumer do not have any concept of group management, only the > > > high-level consumers have. So multiple simple consumers can > independently > > > consume from the same partition(s). > > > > > > Guozhang > > > > > > > > > On Tue, Oct 1, 2013 at 8:11 AM, Kane Kane <kane.ist...@gmail.com> > wrote: > > > > > > > Yeah, I noticed that, i'm curious how balancing happens if > > SimpleConsumer > > > > is used. I.e. i can provide a partition to read from if i use > > > > SimpleConsumer, but what if someone else already attached to that > > > > partition, what would happen? Also what would happen if one > > > SimpleConsumer > > > > attached to all partitions? Noone would be able to join? > > > > > > > > > > > > On Tue, Oct 1, 2013 at 6:33 AM, Neha Narkhede < > neha.narkh...@gmail.com > > > > >wrote: > > > > > > > > > There are 2 types of consumer clients in Kafka - > > > > ZookeeperConsumerConnector > > > > > and SimpleConsumer. Only the former has the re balancing logic. > > > > > > > > > > Thanks, > > > > > Neha > > > > > On Oct 1, 2013 6:30 AM, "Kane Kane" <kane.ist...@gmail.com> wrote: > > > > > > > > > > > But it looks like some clients don't implement it? > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > >
