Hi Gerard, When trying to reproduce this, did you use the go sarama client Safique mentioned?
On Fri, Jun 3, 2016 at 5:10 AM, Gerard Klijs <gerard.kl...@dizzit.com> wrote: > I asume you use a replication factor of 3 for the topics? When I ran some > test with producer/consumers in a dockerized setup, there where only few > failures before the producer switched to to correct new broker again. I > don't know the exact time, but seemed like a few seconds at max, this was > with with 0.9.0.0. > > On Fri, Jun 3, 2016 at 9:00 AM safique ahemad <saf.jnu...@gmail.com> > wrote: > > > Hi Steve, > > > > There is no way to access that from public side so I won't be able to do > > that. Sorry for that. > > But the step is quite simple. The only difference is that we have > deployed > > Kafka cluster using mesos url. > > > > 1) launch 3 Kafka broker cluster and create a topic with multiple > > partitions at least 3 so that one partition land at least on a broker. > > 2) launch consumer/producer client. > > 3) kill a broker > > 4) just observe the behavior of producer client > > > > > > > > On Thu, Jun 2, 2016 at 8:15 PM, Steve Tian <steve.cs.t...@gmail.com> > > wrote: > > > > > I see. I'm not sure if this is a known issue. Do you mind share the > > > brokers/topics setup and the steps to reproduce this issue? > > > > > > Cheers, Steve > > > > > > On Fri, Jun 3, 2016, 9:45 AM safique ahemad <saf.jnu...@gmail.com> > > wrote: > > > > > > > you got it right... > > > > > > > > But DialTimeout is not a concern here. Client try fetching metadata > > from > > > > Kafka brokers but Kafka give them stale metadata near 30-40 sec. > > > > It try to fetch 3-4 time in between until it get updated metadata. > > > > This is completely different problem than > > > > https://github.com/Shopify/sarama/issues/661 > > > > > > > > > > > > > > > > On Thu, Jun 2, 2016 at 6:05 PM, Steve Tian <steve.cs.t...@gmail.com> > > > > wrote: > > > > > > > > > So you are coming from > https://github.com/Shopify/sarama/issues/661 > > , > > > > > right? I'm not sure if anything from broker side can help but > looks > > > > like > > > > > you already found DialTimeout on client side can help? > > > > > > > > > > Cheers, Steve > > > > > > > > > > On Fri, Jun 3, 2016, 8:33 AM safique ahemad <saf.jnu...@gmail.com> > > > > wrote: > > > > > > > > > > > kafka version:0.9.0.0 > > > > > > go sarama client version: 1.8 > > > > > > > > > > > > On Thu, Jun 2, 2016 at 5:14 PM, Steve Tian < > > steve.cs.t...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Client version? > > > > > > > > > > > > > > On Fri, Jun 3, 2016, 4:44 AM safique ahemad < > > saf.jnu...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > We are using Kafka broker cluster in our data center. > > > > > > > > Recently, It is realized that when a Kafka broker goes down > > then > > > > > client > > > > > > > try > > > > > > > > to refresh the metadata but it get stale metadata upto near > 30 > > > > > seconds. > > > > > > > > > > > > > > > > After near 30-35 seconds, updated metadata is obtained by > > client. > > > > > This > > > > > > is > > > > > > > > really a large time for the client continuously gets send > > failure > > > > for > > > > > > so > > > > > > > > long. > > > > > > > > > > > > > > > > Kindly, reply if any configuration may help here or something > > > else > > > > or > > > > > > > > required. > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > Regards, > > > > > > > > Safique Ahemad > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Regards, > > > > > > Safique Ahemad > > > > > > GlobalLogic | Leaders in software R&D services > > > > > > P :+91 120 4342000-2990 | M:+91 9953533367 > > > > > > www.globallogic.com > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Regards, > > > > Safique Ahemad > > > > GlobalLogic | Leaders in software R&D services > > > > P :+91 120 4342000-2990 | M:+91 9953533367 > > > > www.globallogic.com > > > > > > > > > > > > > > > -- > > > > Regards, > > Safique Ahemad > > GlobalLogic | Leaders in software R&D services > > P :+91 120 4342000-2990 | M:+91 9953533367 > > www.globallogic.com > > >
