Re: Kafka take too long to update the client with metadata when a broker is gone

I digged it furthermore... It seems the API blockingSendAndReceive hanging for a long to send/receive response from the broker which is not affected. I just checked send and receive time its taking near about 30 sec. On Tue, Jun 14, 2016 at 11:03 AM, safique ahemad

Re: Kafka take too long to update the client with metadata when a broker is gone

Hello guys, Below is the link where kafka logs can be seens with TRACE enabled. https://drive.google.com/file/d/0B-nANlrsm5ogQkh1NUR2UHYtbkU/view?usp=sharing I have truncated log as it was very big but it has all the cover of the time of problem. Scenario: 1) There were three kafka running

Re: Kafka take too long to update the client with metadata when a broker is gone

Hi Gerard, When trying to reproduce this, did you use the go sarama client Safique mentioned? On Fri, Jun 3, 2016 at 5:10 AM, Gerard Klijs wrote: > I asume you use a replication factor of 3 for the topics? When I ran some > test with producer/consumers in a dockerized

Re: Kafka take too long to update the client with metadata when a broker is gone

I asume you use a replication factor of 3 for the topics? When I ran some test with producer/consumers in a dockerized setup, there where only few failures before the producer switched to to correct new broker again. I don't know the exact time, but seemed like a few seconds at max, this was with

Re: Kafka take too long to update the client with metadata when a broker is gone

Hi Steve, There is no way to access that from public side so I won't be able to do that. Sorry for that. But the step is quite simple. The only difference is that we have deployed Kafka cluster using mesos url. 1) launch 3 Kafka broker cluster and create a topic with multiple partitions at least

Re: Kafka take too long to update the client with metadata when a broker is gone

I see. I'm not sure if this is a known issue. Do you mind share the brokers/topics setup and the steps to reproduce this issue? Cheers, Steve On Fri, Jun 3, 2016, 9:45 AM safique ahemad wrote: > you got it right... > > But DialTimeout is not a concern here. Client try

Re: Kafka take too long to update the client with metadata when a broker is gone

you got it right... But DialTimeout is not a concern here. Client try fetching metadata from Kafka brokers but Kafka give them stale metadata near 30-40 sec. It try to fetch 3-4 time in between until it get updated metadata. This is completely different problem than

Re: Kafka take too long to update the client with metadata when a broker is gone

So you are coming from https://github.com/Shopify/sarama/issues/661 , right? I'm not sure if anything from broker side can help but looks like you already found DialTimeout on client side can help? Cheers, Steve On Fri, Jun 3, 2016, 8:33 AM safique ahemad wrote: > kafka

Re: Kafka take too long to update the client with metadata when a broker is gone

kafka version:0.9.0.0 go sarama client version: 1.8 On Thu, Jun 2, 2016 at 5:14 PM, Steve Tian wrote: > Client version? > > On Fri, Jun 3, 2016, 4:44 AM safique ahemad wrote: > > > Hi All, > > > > We are using Kafka broker cluster in our data

Re: Kafka take too long to update the client with metadata when a broker is gone

Client version? On Fri, Jun 3, 2016, 4:44 AM safique ahemad wrote: > Hi All, > > We are using Kafka broker cluster in our data center. > Recently, It is realized that when a Kafka broker goes down then client try > to refresh the metadata but it get stale metadata upto

Kafka take too long to update the client with metadata when a broker is gone

Hi All, We are using Kafka broker cluster in our data center. Recently, It is realized that when a Kafka broker goes down then client try to refresh the metadata but it get stale metadata upto near 30 seconds. After near 30-35 seconds, updated metadata is obtained by client. This is really a