Have you considered increasing max.block.ms or buffer.memory? Thanks, Jamie
-----Original Message----- From: Matthias J. Sax <[email protected]> To: users <[email protected]> Sent: Thu, Oct 24, 2019 06:35 AM Subject: Re: Keep getting expiring X record(s) for changelog topic:326554 ms has passed since batch creation I would recommend to investigate broker and network health. -Matthias On 10/19/19 7:28 PM, Xiyuan Hu wrote: > Hi, > > I'm running Kafka Streams v2.3.0. During peak hours, I noticed some > nodes had Timeout exception and it will mark the node status to DEAD. > Even though, I implement a CustomProductionExceptionHandler to set the > ProductionExceptionHandlerResponse.CONTINUE, it doesn't solve the > problem, just keep the nodes running. > > Could anyone help me to understand why it throws above Timeout > exception? I did a lot search and people mention it's due to producer > slowness. But GC, CPU are all low. > > I have set request.timeout to 5 min, default.api.timeout to 3min, > session.timeout to 2min, retry 1000 and retry backoff 100ms. > > Really appreciated any help!! > > Thanks. >
