I might not be the best one to answer this question. It will be better if
someone has more operation experience can help out here.
Jiangjie (Becket) Qin
On 4/21/15, 12:00 PM, "Achanta Vamsi Subhash"
wrote:
>@Qin
>Thanks for the inputs. Could you point us what we should monitor in
>zookeeper if
@Qin
Thanks for the inputs. Could you point us what we should monitor in
zookeeper if we have 20k partitions (this might go ~1 lakh in the coming 6
months)? We are not running zookeeper on SSDs and we were facing
ZkConnectionLoss in HighLevelConsumer, sometimes for all the topics. What
are the crit
In my knowledge if you are using 0.8.2.1 which is latest stable you can
sync up your consumer offsets in kafka itself instead of Zk which further
brings down write load on ZKs.
Regards,
Pushkar
On Tue, Apr 21, 2015 at 1:13 PM, Jiangjie Qin
wrote:
> 2 partitions should be OK.
>
> On 4/21/15,
2 partitions should be OK.
On 4/21/15, 12:33 AM, "Achanta Vamsi Subhash"
wrote:
>We are planning to have ~2 partitions. Will it be a bottleneck?
>
>On Mon, Apr 20, 2015 at 10:48 PM, Jiangjie Qin
>wrote:
>
>> Producers usually do not query zookeeper at all.
>> Consumers usually query zoo
We are planning to have ~2 partitions. Will it be a bottleneck?
On Mon, Apr 20, 2015 at 10:48 PM, Jiangjie Qin
wrote:
> Producers usually do not query zookeeper at all.
> Consumers usually query zookeeper at beginning or rebalance. It is
> supposed to be in frequent if you don¹t have consume
Producers usually do not query zookeeper at all.
Consumers usually query zookeeper at beginning or rebalance. It is
supposed to be in frequent if you don¹t have consumers come and go all the
time. One exception is that if you are using zookeeper based consumer
offset commit, it will commit offset t
Hi,
Could anyone help with this?
Thanks.
On Sun, Apr 19, 2015 at 12:58 AM, Achanta Vamsi Subhash <
achanta.va...@flipkart.com> wrote:
> Hi,
>
> How often does Kafka query zookeeper while producing and consuming?
>
> Ex:
> If there is a single partition to which we produce and a HighLevel
> cons
Hi,
How often does Kafka query zookeeper while producing and consuming?
Ex:
If there is a single partition to which we produce and a HighLevel consumer
running on it, how many read/write queries to zookeeper happen.
Extending further, multiple topics with ~100 partitions each, how many
zookeeper
