Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru merged PR #15511: URL: https://github.com/apache/kafka/pull/15511 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on PR #15511: URL: https://github.com/apache/kafka/pull/15511#issuecomment-2000323516 Went over the latests updates, LGTM. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526716168 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -565,6 +565,58 @@ public void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526715547 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -565,6 +565,58 @@ public void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526581796 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,42 +894,39 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on PR #15511: URL: https://github.com/apache/kafka/pull/15511#issuecomment-2000134008 Thanks for the updates @lucasbru, took another look, LGTM considering the latests nits. Just for the record, I will follow-up with

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526609264 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -29,6 +29,8 @@ import

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526609264 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -29,6 +29,8 @@ import

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526609264 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -29,6 +29,8 @@ import

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526605829 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -976,6 +974,10 @@ void maybeReconcile() { }

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526605292 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -911,9 +911,13 @@ void maybeReconcile() { SortedSet

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

dajac commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526420387 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -970,7 +973,11 @@ void maybeReconcile() {

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526349331 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -529,25 +530,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1526346873 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -911,9 +911,13 @@ void maybeReconcile() { SortedSet

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525940513 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -911,9 +911,13 @@ void maybeReconcile() { SortedSet

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524592939 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManager.java: ## @@ -185,4 +185,18 @@ public interface MembershipManager extends

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525912448 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -529,25 +530,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on PR #15511: URL: https://github.com/apache/kafka/pull/15511#issuecomment-1998471082 Thanks for the updates @lucasbru, left some minor comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525452774 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -976,6 +974,10 @@ void maybeReconcile() { }

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525446624 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -1028,9 +1028,9 @@ public void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525418611 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -2279,22 +2277,23 @@ private MembershipManagerImpl

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525386165 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -911,9 +911,13 @@ void maybeReconcile() { SortedSet

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1525360383 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -988,7 +989,8 @@ long getExpirationTimeForTimeout(final long

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524575611 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -1497,4 +1502,93 @@ public PollResult poll(final long

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

dajac commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524608939 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -529,25 +530,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524569366 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -530,19 +530,12 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524568814 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

dajac commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1524430731 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -529,25 +530,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on PR #15511: URL: https://github.com/apache/kafka/pull/15511#issuecomment-1995835817 hey @lucasbru, I took another full pass, left a few comments. Thanks for the changes! -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523858796 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523903844 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -1952,19 +1942,22 @@ private void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523885245 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -529,25 +530,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523863939 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -683,7 +681,10 @@ public void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523880045 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -508,9 +508,30 @@ private void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523863939 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -683,7 +681,10 @@ public void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523858796 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523734711 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -1497,4 +1502,93 @@ public PollResult poll(final long

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523709384 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -530,19 +530,12 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523523053 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -566,18 +567,18 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523522542 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -538,16 +539,16 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523522030 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -210,12 +213,12 @@ public class MembershipManagerImpl

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1523520659 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -530,19 +530,12 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1522964906 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -508,9 +508,30 @@ private void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1522901846 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521937262 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -210,12 +213,12 @@ public class MembershipManagerImpl implements

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521933499 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521933499 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -889,43 +914,36 @@ private void transitionToStale() { */

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521750747 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -508,9 +508,30 @@ private void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521745679 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -508,9 +508,30 @@ private void

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lianetm commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521632809 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -538,16 +539,16 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

dajac commented on code in PR #15511: URL: https://github.com/apache/kafka/pull/15511#discussion_r1521552751 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -538,16 +539,16 @@ public ConsumerGroupHeartbeatRequestData

Re: [PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru commented on PR #15511: URL: https://github.com/apache/kafka/pull/15511#issuecomment-1988762078 @lianetm @dajac Could you please have a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-16312, KAFKA-16185: Local epochs in reconciliation [kafka]

lucasbru opened a new pull request, #15511: URL: https://github.com/apache/kafka/pull/15511 The goal of this PR is to change the following internals of the reconciliation: - Introduce a "local epoch" to the local target assignment. When a new target is received by the server, we