This will force a rewrite of those monitoring tools and UI tools that read
offsets from Zookeeper in order directly to get lag information for
reporting on consumer clients. It seems a good thing to know this is coming
down the pipe.
Dennis
On Wed, Jun 4, 2014 at 6:50 PM, Neha Narkhede
wrote:
At LinkedIn, we are just beginning to move our applications to Kafka based
offset management. I'm not sure if Todd/Clark/Joel have any numbers to
share right away, but depending on the number of consumers you have and the
number of topics those consumers pull, it will likely make a sizable impact
o
Very nice.
Do you guys have any stats on what kind of load was reduced on ZK? Just
trying to understand if this changes the type of servers required to host
ZK.
On Wed, Jun 4, 2014 at 1:10 PM, Guozhang Wang wrote:
> Yes, we are migrating the offset management from ZK to the broker as a
> s
Yes, we are migrating the offset management from ZK to the broker as a
special log.
https://cwiki.apache.org/confluence/display/KAFKA/Inbuilt+Consumer+Offset+Management
The code is in trunk, and it is running in production at LinkedIn now.
Guozhang
On Wed, Jun 4, 2014 at 10:00 AM, S Ahmed wro
Consumers in a future release (0.8.2) or trunk will write offsets to a
Kafka topic, not zookeeper.
Thanks,
Neha
On Jun 4, 2014 10:00 AM, "S Ahmed" wrote:
> I swear I read that Jay Kreps wrote somewhere that consumers now write
> their offsets in a logfile (not in zookeeper).
>
> Is this true or
I swear I read that Jay Kreps wrote somewhere that consumers now write
their offsets in a logfile (not in zookeeper).
Is this true or did I misread? Sorry I can't find the article I was
reading.
