Re: High CPU on kafka nodes

2018-11-29 Thread Johan Rask
What about CPU and load on your consumers? We had a similar problem with high cpu and high network traffic on our Kafka cluster and high cpu and load and traffic on our consumer machines. Thiis was caused by la to ow setting on kafka consumer fetch.max.wait.ms which was only 5 ms. Setting this to 1

Re: Cli (or other tool?) with structured output like json for administration?

2018-08-13 Thread Johan Rask
kl. 14:28 skrev M. Manna : > If you are working on automating your kafka setup, wouldn't it be better > for you to use confluent OSS? Or, are you aiming for something very much > home-grown? > > On 13 August 2018 at 13:25, Johan Rask wrote: > > > Hi, > > we are

Cli (or other tool?) with structured output like json for administration?

2018-08-13 Thread Johan Rask
Hi, we are working on automating our kafka setup (topics, users, acls etc) but we are missing structured output (think json) from the cli tools (i.e kafka-acls.sh, kafka-topics.sh). It only gives us human readable output which is hard to parse. We would like to be able to use it with common tools

Kafka Consumer SASL fails after cluster restart

2018-05-31 Thread Johan Rask
Hi, We are running a 4 broker cluster with kafka 1.1 (confluent), and we are currently securing our cluster with SASL_SSL. Before we introduced SASL, we had no problems taking all brokers down and up again without any issues with producers or consumers, they would all resume processing once the