Pradeep G created KAFKA-2025:
--------------------------------
Summary: In multi-consumer setup - explicit commit, commits on all
partitions
Key: KAFKA-2025
URL: https://issues.apache.org/jira/browse/KAFKA-2025
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.8.2.0
Environment: 1. Tested in Windows
2. Not tested on Linux
Reporter: Pradeep G
Assignee: Neha Narkhede
Priority: Blocker
In a setup where there are two consumers C1 & C2 belonging to consumer group
CG, two partitions P1 & P2; with auto-commit disabled.
An explicit commit on ConsumerConnect commits on all the consumers i.e. a
commit called by C1 commits all messages being processed by other consumers too
here C2.
Ideally C1 should be able to commit only those messages it has consumed and not
what is being processed by C2. The effect of this behavior is that; suppose C2
crashes while processing message M after C1 commits, is that message M being
processed by C2 is not available on recovery and is lost forever; and in kafka
M is marked as consumed.
I read that this would be addressed in the rewrite -
https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI
Any thoughts on which release this would be addressed ?. A quick response
would be greatly appreciated.
Thanks,
Pradeep
This
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
