1. may be you can monitor thread wise cpu usage and correlate with thread
dump
to identify the bottleneck
2. Broker config property connections.max.idle.ms is used to close
idle connections.
default is 10min.
On Thu, Mar 23, 2017 at 3:55 PM, Paul van der Linden <p...@sportr.co.uk>
wrote:
> Hi,
>
> I deployed Kafka about a week ago, but there are a few problems with how
> Kafka behaves.
> The first is the surprisingly high resource usage, one this the memory
> (1.5-2 GB for each broker, 3 brokers) although this might be normal. The
> other one is the cpu usage, which starts with 20% minimum on each broker,
> which I find strange with the current throughput (which is < 1 msg/s).
>
> This might has something to do with something else which I find strange,
> Kafka disconnects clients about every 10-20 minutes per broker. It might
> have something to do with the configuration: Deployed in kubernetes,
> bootstrapping with a single dns name (which is backed by all alive kafka
> brokers), and then every broker has a separate dns address which is used
> after the bootstrap. This means that a client is connected twice to one of
> the brokers. The reason for the bootstrap dns name is to make sure I don't
> have to update all clients to include other brokers.
>
> Any advice on how to solve these 2 problems?
>
> Thanks,
> Paul
>
> On Tue, Mar 21, 2017 at 10:30 AM, Paul van der Linden <p...@sportr.co.uk>
> wrote:
>
> > Hi,
> >
> > I deployed Kafka about a week ago, but there are a few problems with how
> > Kafka behaves.
> > The first is the surprisingly high resource usage, one this the memory
> > (1.5-2 GB for each broker, 3 brokers) although this might be normal. The
> > other one is the cpu usage, which starts with 20% minimum on each broker,
> > which I find strange with the current throughput (which is < 1 msg/s).
> >
> > This might has something to do with something else which I find strange,
> > Kafka disconnects clients about every 10-20 minutes per broker. It might
> > have something to do with the configuration: Deployed in kubernetes,
> > bootstrapping with a single dns name (which is backed by all alive kafka
> > brokers), and then every broker has a separate dns address which is used
> > after the bootstrap. This means that a client is connected twice to one
> of
> > the brokers. The reason for the bootstrap dns name is to make sure I
> don't
> > have to update all clients to include other brokers.
> >
> > Any advice on how to solve these 2 problems?
> >
> > Thanks,
> > Paul
> >
>
