Re: Producer Async Issue
Hey Jay, What will be the disadvantages of setting max.block.ms=0? Regards, Abhimanyu On Mon, May 29, 2017 at 3:15 AM, Jay Kreps wrote: > I think setting max.block.ms=0 does what you want. > > -Jay > > On Sat, May 27, 2017 at 12:40 PM, Abhimanyu Nagrath < > abhimanyunagr...@gmail.com> wrote: > > > HI Hans, > > > > What exactly I meant by asynchronous is that when my Kafka broker is down > > and I am trying to produce the message . It is getting stuck till the > > configured max.block.ms and after that further code is executed. What I > am > > looking for is that whether the broker is down or not it should not get > > stuck. > > > > > > > > Regards, > > Abhimanyu > > > > On Sat, May 27, 2017 at 10:30 PM, Hans Jespersen > > wrote: > > > > > The producer is asynchronous (assuming you mean the Java Producer) > > > > > > https://kafka.apache.org/0102/javadoc/index.html?org/apache/ > > > kafka/clients/producer/KafkaProducer.html > > > > > > -hans > > > > > > > On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath < > > > abhimanyunagr...@gmail.com> wrote: > > > > > > > > Hi, > > > > I am using Kafka 0.10.2 single node cluster and I want to know > > > > Kafka producer completely asynchronous. So what configuration I need > to > > > > change in order to make producer completely asynchronous. > > > > > > > > > > > > > > > > Regards, > > > > Abhimanyu > > > > > >
Re: Producer Async Issue
I think setting max.block.ms=0 does what you want. -Jay On Sat, May 27, 2017 at 12:40 PM, Abhimanyu Nagrath < abhimanyunagr...@gmail.com> wrote: > HI Hans, > > What exactly I meant by asynchronous is that when my Kafka broker is down > and I am trying to produce the message . It is getting stuck till the > configured max.block.ms and after that further code is executed. What I am > looking for is that whether the broker is down or not it should not get > stuck. > > > > Regards, > Abhimanyu > > On Sat, May 27, 2017 at 10:30 PM, Hans Jespersen > wrote: > > > The producer is asynchronous (assuming you mean the Java Producer) > > > > https://kafka.apache.org/0102/javadoc/index.html?org/apache/ > > kafka/clients/producer/KafkaProducer.html > > > > -hans > > > > > On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath < > > abhimanyunagr...@gmail.com> wrote: > > > > > > Hi, > > > I am using Kafka 0.10.2 single node cluster and I want to know > > > Kafka producer completely asynchronous. So what configuration I need to > > > change in order to make producer completely asynchronous. > > > > > > > > > > > > Regards, > > > Abhimanyu > > >
Re: Producer Async Issue
So you just want to not block and just silently throw away the messages and lose them forever? Kafka's persistence is all in the broker so there is no client side storing of data on disk. The client will send in async mode until the client memory buffer is full. Only then does it block, and this is by design because then it's up to your app to decide to either throw the messages away, stop publishing, or store them somewhere outside of Kafka. The easiest solution would be to run more than one broker so that they are fault tolerant and will take over for any failed broker nodes. -hans > On May 27, 2017, at 12:40 PM, Abhimanyu Nagrath > wrote: > > HI Hans, > > What exactly I meant by asynchronous is that when my Kafka broker is down > and I am trying to produce the message . It is getting stuck till the > configured max.block.ms and after that further code is executed. What I am > looking for is that whether the broker is down or not it should not get > stuck. > > > > Regards, > Abhimanyu > >> On Sat, May 27, 2017 at 10:30 PM, Hans Jespersen wrote: >> >> The producer is asynchronous (assuming you mean the Java Producer) >> >> https://kafka.apache.org/0102/javadoc/index.html?org/apache/ >> kafka/clients/producer/KafkaProducer.html >> >> -hans >> >>> On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath < >> abhimanyunagr...@gmail.com> wrote: >>> >>> Hi, >>> I am using Kafka 0.10.2 single node cluster and I want to know >>> Kafka producer completely asynchronous. So what configuration I need to >>> change in order to make producer completely asynchronous. >>> >>> >>> >>> Regards, >>> Abhimanyu >>
Re: Producer Async Issue
HI Hans, What exactly I meant by asynchronous is that when my Kafka broker is down and I am trying to produce the message . It is getting stuck till the configured max.block.ms and after that further code is executed. What I am looking for is that whether the broker is down or not it should not get stuck. Regards, Abhimanyu On Sat, May 27, 2017 at 10:30 PM, Hans Jespersen wrote: > The producer is asynchronous (assuming you mean the Java Producer) > > https://kafka.apache.org/0102/javadoc/index.html?org/apache/ > kafka/clients/producer/KafkaProducer.html > > -hans > > > On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath < > abhimanyunagr...@gmail.com> wrote: > > > > Hi, > > I am using Kafka 0.10.2 single node cluster and I want to know > > Kafka producer completely asynchronous. So what configuration I need to > > change in order to make producer completely asynchronous. > > > > > > > > Regards, > > Abhimanyu >
Re: Producer Async Issue
The producer is asynchronous (assuming you mean the Java Producer) https://kafka.apache.org/0102/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html -hans > On May 27, 2017, at 5:15 AM, Abhimanyu Nagrath > wrote: > > Hi, > I am using Kafka 0.10.2 single node cluster and I want to know > Kafka producer completely asynchronous. So what configuration I need to > change in order to make producer completely asynchronous. > > > > Regards, > Abhimanyu
Producer Async Issue
Hi, I am using Kafka 0.10.2 single node cluster and I want to know Kafka producer completely asynchronous. So what configuration I need to change in order to make producer completely asynchronous. Regards, Abhimanyu
