Ewen Cheslack-Postava created KAFKA-2786:
--------------------------------------------
Summary: Only invoke SinkTask onPartitionsRevoked and
commitOffsets after task has fully started
Key: KAFKA-2786
URL: https://issues.apache.org/jira/browse/KAFKA-2786
Project: Kafka
Issue Type: Bug
Components: copycat
Reporter: Ewen Cheslack-Postava
Assignee: Ewen Cheslack-Postava
Priority: Minor
Fix For: 0.9.0.0
Currently we're invoking task.onPartitionsRevoked and commitOffsets on any
consumer onPartitionsRevoked callback. Since this happens even on the first
rebalance (which is somewhat unexpected given that partitions cannot actually
be revoked at that point), we will not be fully setup in the SinkTask and this
can throw exceptions.
It turns out this is easy to miss for two reasons. First, the code is likely to
throw an exception, but the consumer catches this and handles it. We end up not
missing any important steps as a result. Second, the logging was a bit screwed
up and we weren't getting the full exception stacktrace, just the toString(),
so it was easy to miss that there was a problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
