@Ewen
On Fri, Mar 20, 2015 at 10:04 PM, Ewen Cheslack-Postava
e...@confluent.io wrote:
Even if you have metadata cached, if the broker isn't available then
messages can get stuck in the producer indefinitely. Currently the new
producer doesn't have any client-side timeouts, which is a bug. See
@Mayuresh
On Fri, Mar 20, 2015 at 10:30 PM, Mayuresh Gharat
gharatmayures...@gmail.com wrote:
But if the entire kafka cluster is down, it would try for some configured
number of retries and would return back an error in future. This is my
understanding. Please correct me if I am wrong.
When I
You can set producer property retries not equal to 0. Details can be found
here
http://kafka.apache.org/documentation.html#newproducerconfigs
On Fri, Mar 20, 2015 at 3:01 PM, Samuel Chase samebch...@gmail.com wrote:
Hello Everyone,
In the the new Java Producer API, the Callback code in
Hello Everyone,
In the the new Java Producer API, the Callback code in
KafkaProducer.send is run after there is a response from the Kafka
server. This can be used if some error handling needs to be done based
on the response.
When using the new Java Kafka Producer, I've noticed that when the
Hello Tao,
On Fri, Mar 20, 2015 at 12:39 PM, tao xiao xiaotao...@gmail.com wrote:
You can set producer property retries not equal to 0. Details can be found
here
http://kafka.apache.org/documentation.html#newproducerconfigs
Thanks! I shall try that.
Samuel
On Fri, Mar 20, 2015 at 3:24 PM, tao xiao xiaotao...@gmail.com wrote:
The underlining send runs in a different thread and doesn't block
producer.send(). One way I can think of to detect this is to set
block.on.buffer.full=false and catch BufferExhaustedException then check if
the broker is
@Sunil
On Fri, Mar 20, 2015 at 3:36 PM, sunil kalva sambarc...@gmail.com wrote:
I think KafkaProducer.send method blocks until it fetches partition
metadata for configured time using metadata.fetch.timeout.ms, once time
out it throws TimeoutException. You might be experiencing TimeoutException
@Samuel
My point was
The else branch of the code will be executed when metadata is not
available, and metadata is not available when kafka cluster is not rachable.
please correct me if i am wrong..
On Fri, Mar 20, 2015 at 3:43 PM, Samuel Chase samebch...@gmail.com wrote:
@Sunil
On Fri, Mar
@Sunil
The else branch will be executed if
`metadata.fetch().partitionsForTopic(topic)` returns non NULL value. I
assume that when Kafka is unreachable, it will return NULL.
`waitOnMetadata()` then returns; we never enter the else branch when
Kafka is unreachable.
@Everyone: Is this explanation
This is correct when you send to a topic for the first time. After that
the metadata will be cached, the metadata cache has an age and after it
expires, metadata will be refreshed.
So the time a producer found a broker is not reachable is the minimum
value of the following times:
1. Linger.ms +
I think if the leader is down, it a leader is down, the producer would
issue a metadata request and get the new leader and start producing to it.
But if the entire kafka cluster is down, it would try for some configured
number of retries and would return back an error in future. This is my
@Tao,
On Fri, Mar 20, 2015 at 12:39 PM, tao xiao xiaotao...@gmail.com wrote:
You can set producer property retries not equal to 0. Details can be found
here
http://kafka.apache.org/documentation.html#newproducerconfigs
I set retries to 1, but send is still blocking until the Kafka
Server is
I think KafkaProducer.send method blocks until it fetches partition
metadata for configured time using metadata.fetch.timeout.ms, once time
out it throws TimeoutException. You might be experiencing TimeoutException ?
ref: KafkaProducer.java(waitOnMetadata)
On Fri, Mar 20, 2015 at 2:42 PM, Samuel
13 matches
Mail list logo
