[
https://issues.apache.org/jira/browse/KAFKA-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342739#comment-17342739
]
Moses Nakamura commented on KAFKA-3539:
---------------------------------------
We've been wrestling with this problem for a while. I think my company would
like to start the process to try to patch it. However, it's definitely thorny,
so we had some issues we wanted to talk about! The main questions seem to be:
# Which thread should wait for the partition metadata?
# How can we preserve the ordering semantics that send gives us today?
I think the ideal answer to (1) would be, "no thread" and that we should do it
asynchronously. This would be a pretty invasive change though, so I think it's
probably more expedient to pass in a threadpool and do it there. How would you
feel about optionally passing in a threadpool to KafkaProducer#send, or else on
construction to KafkaProducer?
For (2), it seems like we have two plausible approaches, either pass async data
to the RecordAccumulator, or else layer another queue on top. I think layering
another queue on top would be less invasive, so I would lean toward that
approach. We could have a central queue that orders all of the work,
conditioned on the metadata that we're waiting for being ready.
Does that sound good to you? Should I put up a patch so we can talk about it?
> KafkaProducer.send() may block even though it returns the Future
> ----------------------------------------------------------------
>
> Key: KAFKA-3539
> URL: https://issues.apache.org/jira/browse/KAFKA-3539
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Reporter: Oleg Zhurakousky
> Priority: Critical
> Labels: needs-discussion, needs-kip
>
> You can get more details from the us...@kafka.apache.org by searching on the
> thread with the subject "KafkaProducer block on send".
> The bottom line is that method that returns Future must never block, since it
> essentially violates the Future contract as it was specifically designed to
> return immediately passing control back to the user to check for completion,
> cancel etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
