[
https://issues.apache.org/jira/browse/KAFKA-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede closed KAFKA-233.
-------------------------------
> The producer's load balancing logic can send requests to dead brokers, when
> using the async producer option
> -----------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-233
> URL: https://issues.apache.org/jira/browse/KAFKA-233
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 0.7
> Reporter: Neha Narkhede
> Labels: newbie
> Fix For: 0.8.0
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> The ZK producer, when used with the async producer option does the following
> 1. Create a pool of async producers, one each for a broker registered under
> /broker/ids
> 2. On each send request, apply the Partitioner, to decide the broker and
> partition to send the data
> 3. Use the Async producer's send API to enqueue that data into the async
> producer's queue
> 4. When the data is dequeued by the ProducerSendThread, use the underlying
> sync producer to send it to the broker
> The load balancing decision is taken in step 2, before entering the queue.
> This leaves a window of error, equal to the queue length, when a broker can
> go down. When this happens, potentially, a queue worth of data can fail to
> reach a broker, and will be dropped by the EventHandler.
> To correct this, the Producer, with the async option, needs to be refactored
> to allow only a single queue to hold all requests. And the application of the
> Partitioner should be moved to the end of the queue, in the EventHandler.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
