Jason Gustafson created KAFKA-3072:
--------------------------------------
Summary: Client should only allow commits for assigned partitions
Key: KAFKA-3072
URL: https://issues.apache.org/jira/browse/KAFKA-3072
Project: Kafka
Issue Type: Improvement
Reporter: Jason Gustafson
Assignee: Jason Gustafson
Priority: Minor
In the initial design of the group management protocol for 0.9 Kafka, the
coordinator only permitted commits for partitions that were actually assigned
to the consumer sending the commit request. If a consumer attempted to commit
offsets for an unassigned partition, then the coordinator returned an error
which was then propagated to the user. The capability to do this was lost when
we generalized the group management protocol since the coordinator was no
longer aware of partition assignments.
Although the server can no longer do this check, there's no reason why the
client can't. Instead of blindly forwarding the offsets passed in
commitSync(offsets) to the broker, we can validate that the partitions are
contained in the local assignment set. If not all of them are, then we can
either throw a generic KafkaException or we can reuse the CommitFailedException
(which is currently only used for commit failures caused by illegal
generations). The main advantage of having this check is that it can detect
bugs caused by inadvertent commits earlier. Otherwise, such bugs might be
difficult to detect at all since the only indication might be duplicate
consumption.
Note that any code which depended on the ability to commit offsets for
unassigned partitions would have to be updated. But since such usages aren't
likely to be common and the interface is still unstable, I think the change is
OK.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
