Re: High latency for Kafka transactions

2020-12-02 Thread Liam Clarke-Hutchinson
Hi John, The default for the C# client is actually 5ms, not that it really makes a difference. Are you able to share any of your producer code, especially around the configuration? Kind regards, Liam Clarke On Thu, Dec 3, 2020 at 5:01 PM John Coleman wrote: > Hi, > > We are using Confluent.Ka

Re: High latency for Kafka transactions

2020-12-02 Thread John Coleman
Hi, We are using Confluent.Kafka (dotnet) and according to the docs the linger.ms is default to 0.5ms which would cover a lot of serializations since each occurs in < 10 microseconds. A typical transaction batch would be 2 to 3 messages, occasionally more. Regards, John On 14/11/20 05:24, "G

Re: High latency for Kafka transactions

2020-12-02 Thread John Coleman
Hi, We send small transactions (2 to 3 messages) a few times per second from each of 5 to 10 producers, depending on the consumer partition size. Regards, John On 14/11/20 03:55, "Guozhang Wang" wrote: Hello John, It's a bit hard to reason what's your total producing traffic to the

Re: Kafka Streams: SessionWindows times out immediately and writes out a Null record

2020-12-02 Thread Matthias J. Sax
Kafka Streams uses an incremental computation model, ie, each time a new message arrives, the current session is updated immediately (note that the result is a `KTable`). -- Kafka Streams does not accumulate the raw events and computes the session result when the window closes, but it refines the s

Re: Kafka Streams: Read from one Bootstrap server & write to other

2020-12-02 Thread Matthias J. Sax
Eric, this idea came up multiple time already, but it is much more difficult to build than it seems, especially for supporting "exactly once". Kafka transaction only work on a single cluster... Maybe we will support it at some point, but the "workaround" to either replicate the input or output to

Re: [VOTE] 2.7.0 RC3

2020-12-02 Thread Bill Bejeck
Thanks for the heads-up. I agree this should go into 2.7. I'll roll a new RC once you merge the patch. -Bill On Wed, Dec 2, 2020 at 1:28 PM Jason Gustafson wrote: > Hi Bill, > > I found a potential blocker here: > https://issues.apache.org/jira/browse/KAFKA-10799. A patch should be along > sh

Re: [VOTE] 2.7.0 RC3

2020-12-02 Thread Jason Gustafson
Hi Bill, I found a potential blocker here: https://issues.apache.org/jira/browse/KAFKA-10799. A patch should be along shortly. Best, Jason On Mon, Nov 30, 2020 at 7:27 AM Bill Bejeck wrote: > Thanks for the vote, Gwen. > > Here's an update for Jenkins build > > * Successful Jenkins builds fo

Subscribe to Kafka dev mailing list

2020-12-02 Thread ????
Subscribe to Kafka dev mailing list