Re: KIP to Gracefully handle timeout exception on kafka streams

2020-12-15 Thread Matthias J. Sax
Sounds like `default.api.timeout.ms`. -Matthias On 12/11/20 8:22 AM, Pushkar Deole wrote: > Matthias, > > By the way, one more of our service recently encountered this exception: > can you suggest if this can also be avoided by tuning any specific > configuration ? > >

Re: KIP to Gracefully handle timeout exception on kafka streams

2020-12-11 Thread Pushkar Deole
Matthias, By the way, one more of our service recently encountered this exception: can you suggest if this can also be avoided by tuning any specific configuration ? {"@timestamp":"2020-11-24T13:33:38.617+00:00","@version":"1","message":"Exception processing processor thread -

Re: KIP to Gracefully handle timeout exception on kafka streams

2020-11-21 Thread Matthias J. Sax
KIP-572 will only ship in 2.8.0. For the exception you hit, it's `max.block.ms` -- you might also look into `default.api.timeout.ms`. In general, the relevant configs are documented in the JavaDocs of the corresponding client method. -Matthias On 11/20/20 9:11 PM, Pushkar Deole wrote: >

Re: KIP to Gracefully handle timeout exception on kafka streams

2020-11-20 Thread Pushkar Deole
Thanks Matthias... We are already on kafka 2.5.0, and https://issues.apache.org/jira/browse/KAFKA-8803 mentions that these type of issues are fixed in 2.5.0 Is KIP-572 planned for 2.7.0 ? Also, for timeout and retries, can you provide which parameters should we configure to higher values for

Re: KIP to Gracefully handle timeout exception on kafka streams

2020-11-20 Thread Matthias J. Sax
Yes, if brokers are upgraded via rolling bounce, and the embedded clients are configured with large enough timeouts and retries, they should just fail over to running brokers if a single broker is bounced. If you get a timeout exception, than KafkaStreams dies atm -- we have KIP-572 in-flight

KIP to Gracefully handle timeout exception on kafka streams

2020-11-18 Thread Pushkar Deole
Matthias, We recently ran into an issue where kafka brokers upgraded (i guess it was rolling update) for Aiven business plan 4 to plan 8. This involves change to cpu, memory and storage for each broker. Since this should be rolling upgrade, we expected services to survive, however in one service