Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-10 Thread Yoda Jedi Master
"REPLACE_THREAD - Replaces the thread receiving the exception and processing continues with the same number of configured threads. (Note: this can result in duplicate records depending on the application’s processing mode determined by the PROCESSING_GUARANTEE_CONFIG value)" If an application gets

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-10 Thread Luke Chen
Hi Yoda, For your question: > If an application gets an uncaught exception, then the failed thread will be replaced with another thread and it will continue processing messages, skipping that failed message? --> Yes, if everything goes well after `replace thread`, you can ignore this failed messag

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-10 Thread Bruno Cadonna
Hi Yoda, What do you mean exactly with "skipping that failed message"? Do you mean a record consumed from a topic that caused an exception that killed the stream thread? If the record killed the stream thread due to an exception, for example, a deserialization exception, it will probably als

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-10 Thread Yoda Jedi Master
Hi Luke, thank you for your answer. I will try it, I think I will set an alert if there are too many messages. To ignore the message should I simply return "replace_thread" in the handler? On Tue, Aug 10, 2021 at 12:16 PM Luke Chen wrote: > Hi Yoda, > For your question: > > If an application get

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-10 Thread Yoda Jedi Master
Hi Bruno, thank you for your answer. I mean that the message that caused the exception was consumed and replaced thread will continue from the next message. How then does it handle uncaught exceptions, if it will fail again? On Tue, Aug 10, 2021 at 12:33 PM Bruno Cadonna wrote: > Hi Yoda, > > W

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-16 Thread Bruno Cadonna
Hi Yoda, for certain cases, Kafka Streams allows you to specify handlers that skip the problematic record. Those handlers are: 1. deserialization exception handler configured in default.deserialization.exception.handler 2. time extractor set in default.timestamp.extractor and in the Consumed

Re: Kafka Streams Handling uncaught exceptions REPLACE_THREAD

2021-08-24 Thread Yoda Jedi Master
Thank you for your help, I will check it and try it :-) On Mon, Aug 16, 2021 at 11:45 AM Bruno Cadonna wrote: > Hi Yoda, > > for certain cases, Kafka Streams allows you to specify handlers that > skip the problematic record. Those handlers are: > > 1. deserialization exception handler configured