[
https://issues.apache.org/jira/browse/KAFKA-3236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150778#comment-15150778
]
ASF GitHub Bot commented on KAFKA-3236:
---------------------------------------
GitHub user knusbaum opened a pull request:
https://github.com/apache/kafka/pull/929
KAFKA-3236: Honor Producer Configuration "block.on.buffer.full"
In Kafka-0.9, `max.block.ms` is used to control how long the following
methods will block.
`KafkaProducer.send()` when
- Buffer is full
- Metadata is unavailable
`KafkaProducer.partitionsFor()` when
- Metadata is unavailable
However when `block.on.buffer.full` is set to false, `max.block.ms` is in
effect whenever a buffer is requested/allocated from the Producer BufferPool.
Instead it should throw a BufferExhaustedException without waiting for
`max.block.ms`
This is particulary useful if a producer application does not wish to block
at all on `KafkaProducer.send()`. We avoid waiting on `KafkaProducer.send()`
when metadata is unavailable by invoking `send()` only if the producer instance
has fetched the metadata for the topic in a different thread using the same
producer instance. However `max.block.ms` is still required to specify a
timeout for bootstrapping the metadata fetch.
We should resolve this limitation by decoupling `max.block.ms` and
`block.on.buffer.full`.
- `max.block.ms` will be used exclusively for fetching metadata when
`block.on.buffer.full = false` (in pure non-blocking mode )
- `max.block.ms` will be applicable to both fetching metadata as well as
buffer allocation when `block.on.buffer.full = true`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/knusbaum/kafka KAFKA-3236
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/929.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #929
----
commit 8cc454d2bc9d2be1b0ee916a5dd76042b79954fb
Author: Sanjiv Raj <sanj...@yahoo-inc.com>
Date: 2016-02-11T22:20:26Z
[ADDHR-1240] Honor block.on.buffer.full producer configuration
----
> Honor Producer Configuration "block.on.buffer.full"
> ---------------------------------------------------
>
> Key: KAFKA-3236
> URL: https://issues.apache.org/jira/browse/KAFKA-3236
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.9.0.0
> Reporter: Thomas Graves
> Assignee: Thomas Graves
>
> In Kafka-0.9, "max.block.ms" is used to control how long the following
> methods will block.
> KafkaProducer.send() when
> * Buffer is full
> * Metadata is unavailable
> KafkaProducer.partitionsFor() when
> * Metadata is unavailable
> However when "block.on.buffer.full" is set to false, "max.block.ms" is in
> effect whenever a buffer is requested/allocated from the Producer BufferPool.
> Instead it should throw a BufferExhaustedException without waiting for
> "max.block.ms"
> This is particulary useful if a producer application does not wish to block
> at all on KafkaProducer.send() . We avoid waiting on KafkaProducer.send()
> when metadata is unavailable by invoking send() only if the producer instance
> has fetched the metadata for the topic in a different thread using the same
> producer instance. However "max.block.ms" is still required to specify a
> timeout for bootstrapping the metadata fetch.
> We should resolve this limitation by decoupling "max.block.ms" and
> "block.on.buffer.full".
> * "max.block.ms" will be used exclusively for fetching metadata when
> "block.on.buffer.full" = false (in pure non-blocking mode )
> * "max.block.ms" will be applicable to both fetching metadata as well as
> buffer allocation when "block.on.buffer.full = true
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
