Re: redis versus zookeeper to track consumer offsets

2013-12-17 Thread Benjamin Black
ZK was designed from the start as a clustered, consistent, highly available store for this sort of data and it works extremely well. Redis wasn't and I don't know anyone using Redis in production, including me, who doesn't have stories of Redis losing data. I'm sticking with ZK. On Tue, Dec 17,

Re: redis versus zookeeper to track consumer offsets

2013-12-17 Thread S Ahmed
Interesting, wasn't aware of that. Can you comment on how you go about monitoring your ZK cluster in terms of throughput and if it is reaching its limits? Or is it even possible to do this? On Tue, Dec 17, 2013 at 2:01 PM, Benjamin Black b...@b3k.us wrote: ZK was designed from the start as a

Re: redis versus zookeeper to track consumer offsets

2013-12-17 Thread Neha Narkhede
You can do this indirectly by monitoring the avg/max latency of operations on zookeeper. There is no direct way of measuring the requests/sec to zookeeper since they don't expose the relevant jmx metrics. Thanks, Neha On Tue, Dec 17, 2013 at 11:13 AM, S Ahmed sahmed1...@gmail.com wrote: