A. Sophie Blee-Goldman created KAFKA-12737:
----------------------------------------------
Summary: Commit all healthy tasks after a task-specific error for
better task isolation and reduced overcounting under ALOS
Key: KAFKA-12737
URL: https://issues.apache.org/jira/browse/KAFKA-12737
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: A. Sophie Blee-Goldman
At the moment any time we hit the exception handler, an unclean shutdown will
be triggered on that thread, which means no tasks will be committed. For
certain kinds of exceptions this is unavoidable: for example if the consumer
has dropped out of the group then by definition it can’t commit during
shutdown, and the task will have already been reassigned to another
StreamThread. However there are many common scenarios in which we can (and
should) attempt to commit all the tasks which are in a clean state, ie everyone
except for the task currently being processed when the exception occurred. A
good example of this is de/serialization or user code errors, as well as
exceptions that occur during an operation like closing or suspending a
particular task. In all those cases, there’s no need to throw away all the
progress that has been made by the unaffected tasks who just happened to be
assigned to the same StreamThread.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
