Hi Yoda,
for certain cases, Kafka Streams allows you to specify handlers that
skip the problematic record. Those handlers are:
1. deserialization exception handler configured in
default.deserialization.exception.handler
2. time extractor set in default.timestamp.extractor and in the Consumed
object
3. production exception handler configured in
default.production.exception.handler
Kafka Streams provides implementations for handlers 1 and 2 to skip the
problematic records, that are LogAndContinueExceptionHandler and
LogAndSkipOnInvalidTimestamp, respectively.
For some more details have a look at
https://docs.confluent.io/platform/current/streams/faq.html#failure-and-exception-handling
If problematic records cause an exception in user code, the user code
needs to provide functionality to skip the problematic record.
Best,
Bruno
On 10.08.21 13:26, Yoda Jedi Master wrote:
Hi Bruno, thank you for your answer.
I mean that the message that caused the exception was consumed and replaced
thread will continue from the next message. How then does it handle
uncaught exceptions, if it will fail again?
On Tue, Aug 10, 2021 at 12:33 PM Bruno Cadonna <cado...@apache.org> wrote:
Hi Yoda,
What do you mean exactly with "skipping that failed message"?
Do you mean a record consumed from a topic that caused an exception that
killed the stream thread?
If the record killed the stream thread due to an exception, for example,
a deserialization exception, it will probably also kill the next stream
thread that will read that record. Replacing a stream thread does not
skip records but it can result in duplicate records depending on the
application’s processing mode determined by the
PROCESSING_GUARANTEE_CONFIG value as stated in the docs you cited.
Best,
Bruno
On 10.08.21 11:15, Luke Chen wrote:
Hi Yoda,
For your question:
If an application gets an uncaught exception, then the failed thread
will
be replaced with another thread and it will continue processing messages,
skipping that failed message?
--> Yes, if everything goes well after `replace thread`, you can ignore
this failed message. Just one reminder that you should check the failed
message to avoid this `uncaught exception` thrown again, because if this
happens frequently, it'll impact application performance.
Thank you.
Luke
On Tue, Aug 10, 2021 at 4:25 PM Yoda Jedi Master <yodaj...@gmail.com>
wrote:
"REPLACE_THREAD - Replaces the thread receiving the exception and
processing continues with the same number of configured threads. (Note:
this can result in duplicate records depending on the application’s
processing mode determined by the PROCESSING_GUARANTEE_CONFIG value)"
If an application gets an uncaught exception, then the failed thread
will
be replaced with another thread and it will continue processing
messages,
skipping that failed message?
