[
https://issues.apache.org/jira/browse/NIFI-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18057090#comment-18057090
]
ASF subversion and git services commented on NIFI-15464:
--------------------------------------------------------
Commit ede26f48d7ee3d6b54b8bc4da5cda205abcc9917 in nifi's branch
refs/heads/main from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ede26f48d7 ]
NIFI-15464 Commit pending offsets for revoked partitions in ConsumeKafka
(#10769)
Signed-off-by: David Handermann <[email protected]>
> ConsumeKafka processor causes duplicate message processing during consumer
> group rebalance due to missing offset commit in onPartitionsRevoked
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-15464
> URL: https://issues.apache.org/jira/browse/NIFI-15464
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Pierre Villard
> Assignee: Pierre Villard
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The new ConsumeKafka processor (using Kafka3ConnectionService) causes
> duplicate message processing when a consumer group rebalance occurs (e.g.,
> when a consumer starts or stops). Unlike the legacy ConsumeKafka_2_6
> processor which commits pending offsets during the onPartitionsRevoked
> callback, the new Kafka3ConsumerService only performs a rollback, causing
> uncommitted but already-processed messages to be reprocessed by other
> consumers in the group.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
