Joel Koshy created KAFKA-1787:
---------------------------------
Summary: Consumer should delete offsets and release partition
ownership for deleted topics
Key: KAFKA-1787
URL: https://issues.apache.org/jira/browse/KAFKA-1787
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.2
Reporter: Joel Koshy
Priority: Blocker
Fix For: 0.8.2
Marking as a blocker for now, but we can evaluate.
If a topic is deleted, the consumer currently does not rebalance
(handleDataDeleted is a TODO in ZooKeeperConsumerConnector)
As a result, it will continue owning that (deleted) partition and continue to
issue fetch requests. Those fetch requests will result in an
UnknownTopicOrPartition error and thus will result in TopicMetadataRequests
issued by the leader finder thread which can recreate the topic if auto-create
is turned on.
Furthermore if we don't delete the offsets it is possible to lose messages if
the topic is recreated immediately and the previously checkpointed offset (for
the old data) is small. E.g., if a consumer is at offset 10 on some partition,
and if the partition is deleted and recreated and 15 messages are sent to it
then the first 10 messages will be lost.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
