Re: High level consumer with separate zk
Storing consumer offsets in the broker will add a bit overhead to the broker, but shouldn't be too bad. Compared with traditional messaging systems, Kafka delivers messages to consumers in order and therefore only needs to keep track of a single last consumed offset per consumer client per topic partition. Also, Kafka can scale horizontally and the tracked consumer offsets can be spread over multiple brokers. Thanks, Jun On Wed, Sep 10, 2014 at 8:31 AM, Mike Marzo wrote: > But that could potentially impact the brokers directly. Which is my core > concern. > Is there any idea of the perf impact of having brokers be client aware. > This moves kafka more toward the traditional mom style messaging broker... > > mike marzo > 908 209-4484 > On Sep 10, 2014 11:24 AM, "Jonathan Weeks" > wrote: > > > When 0.8.2 arrives in the near future, consumer offsets will be stored by > > the brokers, and thus that workload will not be impacting ZK. > > > > Best Regards, > > > > -Jonathan > > > > > > On Sep 10, 2014, at 8:20 AM, Mike Marzo > > wrote: > > > > > Is it possible for the high level consumer to use a different zk > cluster > > > than the cluster that manages broker leader electivity? > > > The high level consumer adds a lot of value but I don't like the idea > > that > > > bad user code could pound the core zk and effectively hurt the kafka > > > brokers > > > mike marzo > > > 908 209-4484 > > > > >
Re: High level consumer with separate zk
But that could potentially impact the brokers directly. Which is my core concern. Is there any idea of the perf impact of having brokers be client aware. This moves kafka more toward the traditional mom style messaging broker... mike marzo 908 209-4484 On Sep 10, 2014 11:24 AM, "Jonathan Weeks" wrote: > When 0.8.2 arrives in the near future, consumer offsets will be stored by > the brokers, and thus that workload will not be impacting ZK. > > Best Regards, > > -Jonathan > > > On Sep 10, 2014, at 8:20 AM, Mike Marzo > wrote: > > > Is it possible for the high level consumer to use a different zk cluster > > than the cluster that manages broker leader electivity? > > The high level consumer adds a lot of value but I don't like the idea > that > > bad user code could pound the core zk and effectively hurt the kafka > > brokers > > mike marzo > > 908 209-4484 > >
Re: High level consumer with separate zk
When 0.8.2 arrives in the near future, consumer offsets will be stored by the brokers, and thus that workload will not be impacting ZK. Best Regards, -Jonathan On Sep 10, 2014, at 8:20 AM, Mike Marzo wrote: > Is it possible for the high level consumer to use a different zk cluster > than the cluster that manages broker leader electivity? > The high level consumer adds a lot of value but I don't like the idea that > bad user code could pound the core zk and effectively hurt the kafka > brokers > mike marzo > 908 209-4484
High level consumer with separate zk
Is it possible for the high level consumer to use a different zk cluster than the cluster that manages broker leader electivity? The high level consumer adds a lot of value but I don't like the idea that bad user code could pound the core zk and effectively hurt the kafka brokers mike marzo 908 209-4484
