Users group in quota

2016-12-05 Thread Jiayue Zhang (Bravo)
I want to set quotas configuration for my Kafka 0.10.1 cluster but I have
some questions.

1. How to set 'user' group? From documentation, 'In a cluster that supports
unauthenticated clients, user principal is a grouping of unauthenticated
users chosen by the broker using a configurable PrincipalBuilder'. Is this
something I can control? The cluster doesn't have authentication settings.

2. The producers don't have client-id set up. Will Kafka differentiate them
through IP/Port or all of them will share the '/config/clients/'
quota?

3. We are using simple consumer API and don't set up client-id. Will Kafka
differentiate them through IP/Port or all of them will share the
'/config/clients/' quota?

Thanks,
Bravo


What happens after broker reaches disk write limit

2016-10-03 Thread Jiayue Zhang (Bravo)
Hi,

My Kafka version is 0.8.2.2 Replica factor is
2. auto.leader.rebalance.enable=true

I stopped a broker in my cluster. After a few minutes I started this
broker. The broker was busy catching up huge lag and reached 120MB/s disk
write limit. Additionally there are 23 partitions whose only undead replica
lives on this broker. So shortly after this broker was up, it became the
leader of the 23 partitions. After about 10 minutes this broker was up, all
brokers' network and message in started to fall. Messages in dropped to
close to zero after another 5 minutes. It was only when I stopped this
broker that producers recovered to send messages to cluster again.

1. When a broker reaches 120MB/s disk write limit, how will it affect other
brokers and producers?
2. What caused other brokers' or producers' problem?
3. If broker 3 became leaders of NO partitions after I restarted it, would
it still cause problem even if it reached disk write limit when it was busy
catching up as follows?
4. If question 3's answer is 'yes', what should I do to safely restart a
broker with huge traffic?
5. Does auto.leader.rebalance.enable=true do anything harmful?

Thanks