Hi, I deployed Kafka about a week ago, but there are a few problems with how Kafka behaves. The first is the surprisingly high resource usage, one this the memory (1.5-2 GB for each broker, 3 brokers) although this might be normal. The other one is the cpu usage, which starts with 20% minimum on each broker, which I find strange with the current throughput (which is < 1 msg/s).
This might has something to do with something else which I find strange, Kafka disconnects clients about every 10-20 minutes per broker. It might have something to do with the configuration: Deployed in kubernetes, bootstrapping with a single dns name (which is backed by all alive kafka brokers), and then every broker has a separate dns address which is used after the bootstrap. This means that a client is connected twice to one of the brokers. The reason for the bootstrap dns name is to make sure I don't have to update all clients to include other brokers. Any advice on how to solve these 2 problems? Thanks, Paul On Tue, Mar 21, 2017 at 10:30 AM, Paul van der Linden <p...@sportr.co.uk> wrote: > Hi, > > I deployed Kafka about a week ago, but there are a few problems with how > Kafka behaves. > The first is the surprisingly high resource usage, one this the memory > (1.5-2 GB for each broker, 3 brokers) although this might be normal. The > other one is the cpu usage, which starts with 20% minimum on each broker, > which I find strange with the current throughput (which is < 1 msg/s). > > This might has something to do with something else which I find strange, > Kafka disconnects clients about every 10-20 minutes per broker. It might > have something to do with the configuration: Deployed in kubernetes, > bootstrapping with a single dns name (which is backed by all alive kafka > brokers), and then every broker has a separate dns address which is used > after the bootstrap. This means that a client is connected twice to one of > the brokers. The reason for the bootstrap dns name is to make sure I don't > have to update all clients to include other brokers. > > Any advice on how to solve these 2 problems? > > Thanks, > Paul >
