devendra tagare created KAFKA-1789:
--------------------------------------
Summary: Issue with Async producer
Key: KAFKA-1789
URL: https://issues.apache.org/jira/browse/KAFKA-1789
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.1, 0.8.0
Reporter: devendra tagare
Hi,
We are using an async producer to send data to a kafka cluster.The event rate
at peak is around 250 events/second of size 25KB each.
In the producer code base we have added specific debug statements to capture
the time taken to create a producer,create a keyed message with a byte payload
& send the message.
We have added the below properties to the producerConfig
queue.enqueue.timeout.ms=20
send.buffer.bytes=1024000
topic.metadata.refresh.interval.ms=30000
Based on the documentation, producer.send() queues the message on the async
producer's queue.
So, ideally if the queue is full then the enqueue operation should result in an
kafka.common.QueueFullException in 20 ms.
The logs indicate that the enqueue operation is taking more than 20ms (takes
around 250ms) without throwing any exceptions.
Is there any other property that could conflict with queue.enqueue.timeout.ms
which is causing this behavior ?
Or is it possible that the queue is not full & yet the producer.send() call is
still taking around 200ms under peak load ?
Also, could you suggest any other alternatives so that we can either enforce a
timeout or throw an exception in-case the async producer is taking more than a
specified amount of time.
Regards,
Dev
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
