Checking Connection with Kafka Broker from Client-side

Hi everyone, I'm evaluating how to validate connection to Kafka Brokers in an application that uses Consumer API by making health check using AdminClient. Is there any consideration around Authorization that I should take into consideration/any best practice? (I'm considering calling `describeClus

Re: Kafka streams - runs out of memory

Disabling the stream cache prevents the unbounded memory usage, however the throughput is low (with ROCKSDB cache enabled). Can you please advise why the cache objects reference doesn't get released in time (for GC cleanup) and grows continuously? On Tue, Aug 14, 2018 at 11:17 PM, AshokKumar J w

Re: unable to reconfigure ssl truststore dynamically on broker

Thanks Manikumar - that's very helpful. I never thought to treat the AdminClient like the broker or clients and look for a configuration options set on that page. I should point out to those monitoring that have some control over the code (perhaps yourself even) - it seems wrong to have a program

how to detect kafka protocol

I have a situation where I have to upgrade a distributed system in a piecemeal from plaintext to ssl between kafka clients and broker. Is there a simple way to detect from the client if it should be trying to use plaintext or ssl - from a kafka API perspective? I can do a low-level trick like open

Re: how to detect kafka protocol

Zookeeper definitely has the information about endpoints and protocol. The /brokers/ids/ paths in zookeeper contains the endpoints that are open on the broker. I have doubts that you'll be able to make this change without any downtime though. To my knowledge, clients are only capable of using one

Re: how to detect kafka protocol

Thanks Eric, The situation is like this - we have only a single broker and it has to make a clean break from PLAINTEXT, but our clients are all either old or new (upgraded to SSL). The old ones will obviously not be able to work against the upgraded broker. The problem is I have 18 (only a slight

How to reduce kafka's rebalance time ?

hello: How to reduce kafka's rebalance time ? It takes a lot of time to rebalance each time. Why?

Re: How to reduce kafka's rebalance time ?

I am also facing the same issue. Whenever I am restarting my consumers it is taking upto 10 minutes to start consumption. Also some of the consumers randomly rebalance and it again takes same amount of time to complete rebalance. I haven't been able to figure out any solution for this issue, nor ha