Hi, I'm working on streaming systems in AMPLab at UC Berkeley. This article ( https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka) mentioned the ZooKeeper performance issue when consumers store offsets in ZooKeeper.
"In Kafka releases through 0.8.1.1, consumers commit their offsets to ZooKeeper. ZooKeeper does not scale extremely well (especially for writes) when there are a large number of offsets (i.e., consumer-count * partition-count)." Can anyone tell me in production scenarios, how many consumers and partitions do you have? How much write (offset update) traffic do you generate that ZooKeeper cannot handle? Thank you very much! Xin
