queue.enqueue.timeout.ms only applies if the producer is configured in async mode.
Thanks, Jun On Thu, Nov 20, 2014 at 5:01 AM, Devendra Tagare < devendra.tag...@pubmatic.com> wrote: > Hi, > > We are using an async producer to send data to a kafka cluster.The event > rate at peak is around 250 events/second of size 25KB each. > > In the producer code base we have added specific debug statements to > capture the time taken to create a producer,create a keyed message with a > byte payload & send the message. > > We have added the below properties to the ProducerConfig > > queue.enqueue.timeout.ms=20 > send.buffer.bytes=1024000 > topic.metadata.refresh.interval.ms=30000 > > We also checked if these properties are being picked. > > Based on the documentation, producer.send() queues the message on the > async producer's queue. > > So, ideally if the queue is full then the enqueue operation should result > in an kafka.common.QueueFullException in 20 ms. > > The logs indicate that the enqueue operation is taking more than 20ms > (takes around 250ms) without throwing any exceptions. > > Is there any other property that could conflict with > queue.enqueue.timeout.ms which is causing this behavior ? > > Or is it possible that the queue is not full & yet the producer.send() > call is still taking around 200ms under peak load ? > > Also, could you suggest any other alternatives so that we can either > enforce a timeout or throw an exception in-case the async producer is > taking more than a specified amount of time to queue an event. > > > Regards, > Dev >
