[ https://issues.apache.org/jira/browse/KAFKA-14757 ]
Philip Nee deleted comment on KAFKA-14757: ------------------------------------ was (Author: JIRAUSER283568): Hey [~sanand] - I'm unfamiliar with ConsumerAwareRebalanceListener, but do you know if it revokes the partitions upon invoking onPartitionsLost? I think that's the default behavior of the default rebalance listener, as these partitions are already owned by other consumers. > Kafka Cooperative Sticky Assignor results in significant duplicate consumption > ------------------------------------------------------------------------------ > > Key: KAFKA-14757 > URL: https://issues.apache.org/jira/browse/KAFKA-14757 > Project: Kafka > Issue Type: Bug > Components: consumer > Affects Versions: 3.1.1 > Environment: AWS MSK (broker) and Spring Kafka (2.8.7) for use in > Spring Boot consumers. > Reporter: Siddharth Anand > Priority: Critical > Attachments: image-2023-03-16-10-34-49-358.png, > image-2023-03-16-10-39-18-510.png > > > Details may be found within the linked document: > [Kafka Cooperative Sticky Assignor Issue : Duplicate Consumption | > [https://docs.google.com/document/d/1E7qAwGOpF8jo_YhF4NwUx9CXxUGJmT8OhHEqIg7-GfI/edit?usp=sharing]] > In a nutshell, we noticed that the Cooperative Sticky Assignor resulted in > significant duplicate message consumption. During last year's F1 Grand Prix > events and World Cup soccer events, our company's Kafka-based platform > received live-traffic. This live traffic, coupled with autoscaled consumers > resulted in as much as 70% duplicate message consumption at the Kafka > consumers. > In December 2022, we ran a synthetic load test to confirm that duplicate > message consumption occurs during consumer scale out/in and Kafka partition > rebalancing when using the Cooperative Sticky Assignor. This issue does not > occur when using the Range Assignor. > -- This message was sent by Atlassian Jira (v8.20.10#820010)