[ https://issues.apache.org/jira/browse/KAFKA-16514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sal Sorrentino updated KAFKA-16514: ----------------------------------- Description: Working with Kafka Streams 3.7.0, but may affect earlier versions as well. When attempting to shutdown a streams application and leave the associated consumer group, the supplied `leaveGroup` option seems to have no effect. Sample code: {code:java} CloseOptions options = new CloseOptions().leaveGroup(true); stream.close(options);{code} The expected behavior here is that the group member would shutdown and leave the group, immediately triggering a consumer group rebalance. In practice, the rebalance happens after the appropriate timeout configuration has expired. I understand the default behavior in that there is an assumption that any associated StateStores would be persisted to disk and that in the case of a rolling restart/deployment, the rebalance delay may be preferable. However, in our application we are using in-memory state stores and standby replicas. There is no benefit in delaying the rebalance in this setup and we are in need of a way to force a member to leave the group when shutting down. The workaround we found is to set an undocumented internal StreamConfig to enforce this behavior: {code:java} props.put("internal.leave.group.on.close", true); {code} To state the obvious, this is less than ideal. was: Working with Kafka Streams 3.7.0, but may affect earlier versions as well. When attempting to shutdown a streams application and leave the associated consumer group, the supplied `leaveGroup` option seems to have no effect. Sample code: {code:java} CloseOptions options = new CloseOptions().leaveGroup(true); stream.close(options);{code} The expected behavior here is that the group member would shutdown and leave the group, immediately triggering a consumer group rebalance. In practice, the rebalance happens after the appropriate timeout configuration has expired. I understand the default behavior in that there is an assumption that ant associated StateStores would be persisted to disk and that in the case of a rolling restart/deployment, the rebalance delay may be preferable. However, in our application we are using in-memory state stores and standby replicas. There is no benefit in delaying the rebalance in this setup and we are in need of a way to force a member to leave the group when shutting down. The workaround we found is to set an undocumented internal StreamConfig to enforce this behavior: {code:java} props.put("internal.leave.group.on.close", true); {code} To state the obvious, this is less than ideal. > Kafka Streams: stream.close(CloseOptions) does not respect options.leaveGroup > flag. > ----------------------------------------------------------------------------------- > > Key: KAFKA-16514 > URL: https://issues.apache.org/jira/browse/KAFKA-16514 > Project: Kafka > Issue Type: Bug > Components: streams > Reporter: Sal Sorrentino > Priority: Major > > Working with Kafka Streams 3.7.0, but may affect earlier versions as well. > > When attempting to shutdown a streams application and leave the associated > consumer group, the supplied `leaveGroup` option seems to have no effect. > Sample code: > {code:java} > CloseOptions options = new CloseOptions().leaveGroup(true); > stream.close(options);{code} > The expected behavior here is that the group member would shutdown and leave > the group, immediately triggering a consumer group rebalance. In practice, > the rebalance happens after the appropriate timeout configuration has expired. > I understand the default behavior in that there is an assumption that any > associated StateStores would be persisted to disk and that in the case of a > rolling restart/deployment, the rebalance delay may be preferable. However, > in our application we are using in-memory state stores and standby replicas. > There is no benefit in delaying the rebalance in this setup and we are in > need of a way to force a member to leave the group when shutting down. > The workaround we found is to set an undocumented internal StreamConfig to > enforce this behavior: > {code:java} > props.put("internal.leave.group.on.close", true); > {code} > To state the obvious, this is less than ideal. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)