[
https://issues.apache.org/jira/browse/KAFKA-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224907#comment-14224907
]
Jiangjie Qin commented on KAFKA-1789:
-------------------------------------
Do you mean that the enqueue took 250ms or the entire sending took that much
time? It looks enqueue finished within 20 ms but the send took 250ms to finish.
Since it's an async send, there are some other settings could affect the
sending time.
queue.buffering.max.messages: the size of the message queue
batch.num.messages: the max number of messages in a sending batch
Also, if you somehow have a slow broker, it could also makes the sending time
longer.
> Issue with Async producer
> -------------------------
>
> Key: KAFKA-1789
> URL: https://issues.apache.org/jira/browse/KAFKA-1789
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.0, 0.8.1
> Reporter: devendra tagare
> Priority: Critical
>
> Hi,
> We are using an async producer to send data to a kafka cluster.The event rate
> at peak is around 250 events/second of size 25KB each.
> In the producer code base we have added specific debug statements to capture
> the time taken to create a producer,create a keyed message with a byte
> payload & send the message.
> We have added the below properties to the producerConfig
> queue.enqueue.timeout.ms=20
> send.buffer.bytes=1024000
> topic.metadata.refresh.interval.ms=30000
> Based on the documentation, producer.send() queues the message on the async
> producer's queue.
> So, ideally if the queue is full then the enqueue operation should result in
> an kafka.common.QueueFullException in 20 ms.
> The logs indicate that the enqueue operation is taking more than 20ms (takes
> around 250ms) without throwing any exceptions.
> Is there any other property that could conflict with queue.enqueue.timeout.ms
> which is causing this behavior ?
> Or is it possible that the queue is not full & yet the producer.send() call
> is still taking around 200ms under peak load ?
> Also, could you suggest any other alternatives so that we can either enforce
> a timeout or throw an exception in-case the async producer is taking more
> than a specified amount of time.
> Regards,
> Dev
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
