[ https://issues.apache.org/jira/browse/KAFKA-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365011#comment-14365011 ]
Pradeep G commented on KAFKA-2025: ---------------------------------- Workaround available at --> http://grokbase.com/t/kafka/users/138x3vbfcq/is-it-possible-to-commit-offsets-on-a-per-stream-basis (changed priority from Blocker to Critical). -Pradeep > In multi-consumer setup - explicit commit, commits on all partitions > -------------------------------------------------------------------- > > Key: KAFKA-2025 > URL: https://issues.apache.org/jira/browse/KAFKA-2025 > Project: Kafka > Issue Type: Bug > Components: consumer > Affects Versions: 0.8.2.0 > Environment: 1. Tested in Windows > 2. Not tested on Linux > Reporter: Pradeep G > Assignee: Neha Narkhede > Priority: Critical > > In a setup where there are two consumers C1 & C2 belonging to consumer group > CG, two partitions P1 & P2; with auto-commit disabled. > An explicit commit on ConsumerConnect commits on all the consumers i.e. a > commit called by C1 commits all messages being processed by other consumers > too here C2. > Ideally C1 should be able to commit only those messages it has consumed and > not what is being processed by C2. The effect of this behavior is that; > suppose C2 crashes while processing message M after C1 commits, is that > message M being processed by C2 is not available on recovery and is lost > forever; and in kafka M is marked as consumed. > I read that this would be addressed in the rewrite - > https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ConsumerAPI > Any thoughts on which release this would be addressed ?. A quick response > would be greatly appreciated. > Thanks, > Pradeep -- This message was sent by Atlassian JIRA (v6.3.4#6332)