If you have enough memory for batch size then the scenario is no different
than your original question because block on full memory is true by default
(and if it is false you actually get a notification via exception). So when
you said reduce the buffer memory to "minimum", I assumed you are referr
kishore
How is going to reduce throughput if we have enough memory for batch size
?, could you please explain
On Tue, Aug 18, 2015 at 11:47 AM, Kishore Senji wrote:
> But this will reduce the throughput in a good scenario. May be we need to
> enhance the Callback interface appropriately.
>
> On
But this will reduce the throughput in a good scenario. May be we need to
enhance the Callback interface appropriately.
On Mon, Aug 17, 2015 at 7:15 PM, sunil kalva wrote:
> tx jeff,
> Actually we need to set "buffer.memory" to minimum (default is ~35 MB) and
> "block.on.buffer.full" to "true"
tx jeff,
Actually we need to set "buffer.memory" to minimum (default is ~35 MB) and
"block.on.buffer.full" to "true" so that the sender will block as soon as
these conditions met. And then release once the cluster is healthy.
--
SunilKalva
On Mon, Aug 17, 2015 at 11:20 PM, Jeff Holoman
wrote:
Thanks for you explanation Jeff !
On Mon, Aug 17, 2015 at 11:23 PM, Jeff Holoman
wrote:
> I should've been more specific...if the producer loses total access to all
> brokers...eg. some kind of network issue.
>
> On Mon, Aug 17, 2015 at 1:50 PM, Jeff Holoman
> wrote:
>
> > Actually this won't w
I should've been more specific...if the producer loses total access to all
brokers...eg. some kind of network issue.
On Mon, Aug 17, 2015 at 1:50 PM, Jeff Holoman wrote:
> Actually this won't work. The problem is if the producer loses
> connectivity to the broker, then messages will continue to
Actually this won't work. The problem is if the producer loses connectivity
to the broker, then messages will continue to queue up until batch.size is
exhausted. Then the send will block. At this point, if you gain
connectivity again, then the messages will be resent.
If all brokers die, you shou
Hi Sunil,
Producer will throw an Exception in callback if there is problem while
sending data. You can check like:
public void onCompletion(RecordMetadata arg0, Exception arg1) {
if (arg1 != null) {
System.out.println("exception occured");
}
System.out.println("sent")
On Mon, Aug 17, 2015 at
Hi all
I am using new java producer in async mode, when my entire cluster is down
i am loosing all my messages. How do we get notification when the cluster
is down so that i can send messages to another cluster. The callback is
only triggered when the cluster is reachable .
--SK
