Replica assignment with rack awareness

2019-07-25 Thread Ashish Bhushan
Hi, I wanted to confirm the behavior when a new topic is created in a cluster where brokers have rack.id parameter configured. Lets say we have a 6 broker cluster 3 brokers with rack.id 'RACK1' ( broker ids : 1,2,3 ) 3 brokers with rack.id 'RACK2' ( brokers ids : 4,5,6 ) I created a topic with

Re: Rebalancing algorithm is extremely suboptimal for long processing

2019-07-25 Thread Guozhang Wang
That seems to be a real bug -- and a pretty common one. We will look into it asap. Guozhang On Thu, Jul 25, 2019 at 7:26 AM Raman Gupta wrote: > I'm looking forward to the incremental rebalancing protocol. In the > meantime, I've updated to Kafka 2.3.0 to take advantage of the static > group m

Re: [EXTERNAL] Handling of inter.broker.protocol.version and lo.message.format.version after upgrading

2019-07-25 Thread Sebastian Schmitz
Hi Manuel, thanks. I did the three rolling restarts, or in my case three deployments of containers with different config... But the question now is if I should do these three restarts each time I update to latest Kafka or if I now just change both versions to latest one without having to do t

Re: Rebalancing algorithm is extremely suboptimal for long processing

2019-07-25 Thread Raman Gupta
I'm looking forward to the incremental rebalancing protocol. In the meantime, I've updated to Kafka 2.3.0 to take advantage of the static group membership, and this has actually already helped tremendously. However, unfortunately while it was working initially, some streams are now unable to start

Getting started with Kafka topic to store multiple types

2019-07-25 Thread Pushkar Deole
Hi All, I am new to Kafka and still getting myself acquainted with the product. I have a basic question around using Kafka. I want to store in a Kafka topic, a string value against some keys while a HashMap value against some of the keys. For this purpose, I have created two different producers as