Re: [PR] MINOR: Disable FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor [kafka]

2023-12-01 Thread via GitHub
dajac merged PR #14876: URL: https://github.com/apache/kafka/pull/14876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-12-01 Thread via GitHub
satishd commented on PR #14649: URL: https://github.com/apache/kafka/pull/14649#issuecomment-1835672440 @nikramakrishnan Can you pull out the integration test from these changes and raise a separate PR resolving the intermittent failure so that we can unblock merging this change with the ad

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-01 Thread via GitHub
dajac commented on PR #14640: URL: https://github.com/apache/kafka/pull/14640#issuecomment-1835695981 @kirktrue Thanks for the PR. Could you please update the description? The current one looks pretty outdated. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (KAFKA-15945) Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest

2023-12-01 Thread Atul Sharma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17791951#comment-17791951 ] Atul Sharma commented on KAFKA-15945: - Raised PR: https://github.com/apache/kafka/pu

Re: [PR] KAFKA-15945 : Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest [kafka]

2023-12-01 Thread via GitHub
atu-sharm commented on PR #14893: URL: https://github.com/apache/kafka/pull/14893#issuecomment-1835716040 Hi @mimaison @C0urante can you have a look at this once? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15945 : Flaky test - testSyncTopicConfigs() – org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationBaseTest [kafka]

2023-12-01 Thread via GitHub
ashwinpankaj commented on code in PR #14893: URL: https://github.com/apache/kafka/pull/14893#discussion_r1411813212 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationBaseTest.java: ## @@ -770,6 +771,9 @@ public void testSyncTo

Re: [PR] KAFKA-14509: [2/2] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1411799674 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -547,6 +549,69 @@ public CompletableFuture listGroups(

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1411824345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -365,6 +366,16 @@ private void replaceUnresolvedAssignmentWithNewAs

Re: [PR] KAFKA-14509: [2/N] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14544: URL: https://github.com/apache/kafka/pull/14544#discussion_r1411829247 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -547,6 +549,69 @@ public CompletableFuture listGroups(

Re: [PR] KAFKA-15890: Consumer.poll with long timeout unaware of assigned partitions [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield commented on code in PR #14835: URL: https://github.com/apache/kafka/pull/14835#discussion_r1411833816 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -205,6 +205,26 @@ public void wakeup() { ne

Re: [PR] KAFKA-14987 [2/2]; customize retry backoff for group/offsets expiration [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14870: URL: https://github.com/apache/kafka/pull/14870#discussion_r1411843255 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -620,8 +621,9 @@ public boolean cleanupExpiredOffsets(String group

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411855963 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryProvider.java: ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411859695 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

[jira] [Created] (KAFKA-15952) Create public doc for telemetry state transition

2023-12-01 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15952: - Summary: Create public doc for telemetry state transition Key: KAFKA-15952 URL: https://issues.apache.org/jira/browse/KAFKA-15952 Project: Kafka Issue Type

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411866555 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411873720 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1411872217 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -322,16 +314,14 @@ private void onErrorResponse(final ConsumerGr

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411883131 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411884637 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411900986 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411901649 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411905307 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] MINOR: Fix flaky test RefreshingHttpsJwksTest.testBasicScheduleRefresh [kafka]

2023-12-01 Thread via GitHub
ashwinpankaj commented on code in PR #14888: URL: https://github.com/apache/kafka/pull/14888#discussion_r1411930564 ## clients/src/test/java/org/apache/kafka/common/security/oauthbearer/internals/secured/RefreshingHttpsJwksTest.java: ## @@ -62,14 +62,14 @@ public class Refreshin

Re: [PR] MINOR: Fix flaky test RefreshingHttpsJwksTest.testBasicScheduleRefresh [kafka]

2023-12-01 Thread via GitHub
ashwinpankaj commented on code in PR #14888: URL: https://github.com/apache/kafka/pull/14888#discussion_r1411930564 ## clients/src/test/java/org/apache/kafka/common/security/oauthbearer/internals/secured/RefreshingHttpsJwksTest.java: ## @@ -62,14 +62,14 @@ public class Refreshin

[PR] KAFKA-15831: KIP-1000 protocol and admin client [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield opened a new pull request, #14894: URL: https://github.com/apache/kafka/pull/14894 This adds the new ListClientMetricsResources RPC to the Kafka protocol and puts support into the Kafka admin client. The broker-side implementation in this PR is just to return an empty

Re: [PR] MINOR: Fix flaky test RefreshingHttpsJwksTest.testBasicScheduleRefresh [kafka]

2023-12-01 Thread via GitHub
ashwinpankaj commented on code in PR #14888: URL: https://github.com/apache/kafka/pull/14888#discussion_r1411942462 ## clients/src/test/java/org/apache/kafka/common/security/oauthbearer/internals/secured/RefreshingHttpsJwksTest.java: ## @@ -62,14 +62,14 @@ public class Refreshin

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1411947974 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -136,11 +139,11 @@ public long put(final Bytes key, final byte[

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14724: URL: https://github.com/apache/kafka/pull/14724#discussion_r1411951695 ## clients/src/main/java/org/apache/kafka/clients/ClientTelemetryReporter.java: ## @@ -0,0 +1,952 @@ +/* + * Licensed to the Apache Software Foundation (ASF) und

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1411961568 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -266,75 +269,28 @@ public VersionedRecord get(final Bytes key,

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1411969845 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStore.java: ## @@ -266,75 +269,28 @@ public VersionedRecord get(final Bytes key,

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 commented on PR #14724: URL: https://github.com/apache/kafka/pull/14724#issuecomment-1835985666 > Hi @apoorvmittal10 thanks for writing this PR - also sorry for the delay but I made the first pass and left some stylistic comments. Hi @philipnee Thanks for the review. I

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-12-01 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1412009752 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1791,6 +1794,74 @@ protected int processStreamThread(final Consumer consumer) { retu

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
vamossagar12 commented on PR #14882: URL: https://github.com/apache/kafka/pull/14882#issuecomment-1836000366 @dajac , I ran the test a few times on my laptop. It passes most of the times but sometimes fails with ```[2023-12-01 17:21:23,530] WARN [QuorumController id=3000] Performing control

Re: [PR] KAFKA-15361: Process and persist dir info with broker registration [kafka]

2023-12-01 Thread via GitHub
soarez commented on PR #14838: URL: https://github.com/apache/kafka/pull/14838#issuecomment-1836006504 @cmccabe @rondagostino @pprovenzano please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-12-01 Thread via GitHub
mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1412011082 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -1791,6 +1795,146 @@ protected int processStreamThread(final Consumer consumer) { ret

[jira] [Commented] (KAFKA-15140) Improve TopicCommandIntegrationTest to be less flaky

2023-12-01 Thread Owen C.H. Leung (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792025#comment-17792025 ] Owen C.H. Leung commented on KAFKA-15140: - I've created a PR to fix it :  https

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836027702 @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang I was recommended to tag you all in these Kafka Streams PRs. -- This is an automated message from the A

Re: [PR] MINOR: Add junit properties to display parameterized test names [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14687: URL: https://github.com/apache/kafka/pull/14687#discussion_r1412039372 ## streams/src/test/java/org/apache/kafka/streams/integration/utils/IntegrationTestUtils.java: ## @@ -239,10 +240,14 @@ public static String safeUniqueTestName(final Cl

Re: [PR] KAFKA-14412: Generalise over RocksDB WriteBatch [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14853: URL: https://github.com/apache/kafka/pull/14853#issuecomment-1836027902 @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1412048452 ## streams/src/main/java/org/apache/kafka/streams/state/internals/LogicalSegmentIterator.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1412048452 ## streams/src/main/java/org/apache/kafka/streams/state/internals/LogicalSegmentIterator.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412057240 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -278,18 +280,74 @@ private void addValueProvidersToMetricsRecorder() {

Re: [PR] KAFKA-15910: New group coordinator needs to generate snapshots while loading [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14849: URL: https://github.com/apache/kafka/pull/14849#discussion_r1412044990 ## core/src/main/scala/kafka/coordinator/group/CoordinatorLoaderImpl.scala: ## @@ -153,6 +154,15 @@ class CoordinatorLoaderImpl[T]( } curr

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
dajac commented on PR #14882: URL: https://github.com/apache/kafka/pull/14882#issuecomment-1836072757 @vamossagar12 Hum... `ERROR Unexpected error handling org.apache.kafka.server.AssignmentsManager$DispatchEvent@3909ac4e` is not related to the group coordinator. -- This is an automated

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
vamossagar12 commented on PR #14882: URL: https://github.com/apache/kafka/pull/14882#issuecomment-1836078586 oh I see. Didn't realise that, sorry. The fact that I see this on the local as well means it's flaky.. -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

2023-12-01 Thread via GitHub
apoorvmittal10 closed pull request #14724: KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) URL: https://github.com/apache/kafka/pull/14724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14882: URL: https://github.com/apache/kafka/pull/14882#discussion_r1412090389 ## core/src/test/scala/unit/kafka/server/ConsumerGroupHeartbeatRequestTest.scala: ## @@ -137,6 +137,98 @@ class ConsumerGroupHeartbeatRequestTest(cluster: ClusterInstanc

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14882: URL: https://github.com/apache/kafka/pull/14882#discussion_r1412090130 ## core/src/test/scala/unit/kafka/server/ConsumerGroupHeartbeatRequestTest.scala: ## @@ -137,6 +137,98 @@ class ConsumerGroupHeartbeatRequestTest(cluster: ClusterInstanc

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-12-01 Thread via GitHub
aliehsaeedii commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1412093650 ## streams/src/main/java/org/apache/kafka/streams/state/internals/LogicalSegmentIterator.java: ## @@ -0,0 +1,138 @@ +/* + * Licensed to the Apache Software Foundat

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-12-01 Thread via GitHub
dajac commented on PR #14570: URL: https://github.com/apache/kafka/pull/14570#issuecomment-1836106444 Hey folks. It seems that this PR has introduced failures in trunk. From the last build: ``` Build / JDK 11 and Scala 2.13 / shouldHaveSamePositionBoundActiveAndStandBy – org.apach

Re: [PR] KAFKA-15448: Streams Standby Update Listener (KIP-988) [kafka]

2023-12-01 Thread via GitHub
eduwercamacaro commented on code in PR #14735: URL: https://github.com/apache/kafka/pull/14735#discussion_r1412102373 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -175,13 +176,15 @@ public class KafkaStreams implements AutoCloseable { private f

Re: [PR] KAFKA-15280: Implement client support for KIP-848 server-side assignors [kafka]

2023-12-01 Thread via GitHub
lucasbru commented on PR #14878: URL: https://github.com/apache/kafka/pull/14878#issuecomment-1836125795 @cadonna Could you please review this? @lianetm This includes a small fix in the reconciliation logic. Please check it, nor sure if this is 100% correct. I had trouble getting part

Re: [PR] KAFKA-15842: Correct handling of KafkaConsumer.committed for new consumer [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14859: URL: https://github.com/apache/kafka/pull/14859#discussion_r1412110677 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -663,6 +662,10 @@ public Map committed(final Set

Re: [PR] KAFKA-15842: Correct handling of KafkaConsumer.committed for new consumer [kafka]

2023-12-01 Thread via GitHub
cadonna merged PR #14859: URL: https://github.com/apache/kafka/pull/14859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836136670 @cadonna > Do we have unit tests in place that test all this logic? We don't have any tests dedicated to this, but `RocksDBStoreTest` and `RocksDBTimestampedStoreTest` impli

[jira] [Resolved] (KAFKA-15842) Correct handling of KafkaConsumer.committed for new consumer

2023-12-01 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15842. -- Resolution: Fixed > Correct handling of KafkaConsumer.committed for new consumer > ---

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412123554 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,45 +62,27 @@ public RocksDBTimestampedStore(final String name,

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1412137956 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,45 +62,27 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

2023-12-01 Thread via GitHub
nicktelford commented on PR #14852: URL: https://github.com/apache/kafka/pull/14852#issuecomment-1836167995 Sorry about the compile error. It should now build. Regarding tests: agreed. When KIP-892 lands, it adds an extra CF, but at that point RocksDBStoreTest and RocksDBTimestampedSt

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412141829 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -398,6 +466,38 @@ public class AsyncKafkaConsumer implements Consum

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412142819 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -849,9 +953,22 @@ public OptionalLong currentLag(TopicPartition top

Re: [PR] KAFKA-15448: Streams Standby Update Listener (KIP-988) [kafka]

2023-12-01 Thread via GitHub
eduwercamacaro commented on code in PR #14735: URL: https://github.com/apache/kafka/pull/14735#discussion_r1412148951 ## streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java: ## @@ -580,6 +582,23 @@ public void setGlobalStateRestoreListener(final StateRestoreListene

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412153908 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -849,9 +953,22 @@ public OptionalLong currentLag(TopicPartition top

[PR] HOTFIX: fix ConsistencyVectorIntegrationTest failure [kafka]

2023-12-01 Thread via GitHub
lucasbru opened a new pull request, #14895: URL: https://github.com/apache/kafka/pull/14895 #14735 changed the result for `KeyQuery` from `ValueAndTimestamp` to `V`, but forgot to update `ConsistencyVectorIntegrationTest` accordingly. ### Committer Checklist (excluded from commit m

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412156215 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/BackgroundEventHandler.java: ## @@ -49,4 +49,9 @@ public void add(BackgroundEvent event) {

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412158807 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1365,7 +1482,11 @@ public KafkaConsumerMetrics kafkaConsum

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412168452 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -666,12 +766,16 @@ public Map committed(final Sethttps://github.com

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412169763 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1365,7 +1482,11 @@ public KafkaConsumerMetrics kafkaConsumerMetrics

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412171456 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/events/BackgroundEventHandlerTest.java: ## @@ -51,91 +61,91 @@ public void tearDown() {

[jira] [Updated] (KAFKA-15891) Flaky test: testResetSinkConnectorOffsetsOverriddenConsumerGroupId – org.apache.kafka.connect.integration.OffsetsApiIntegrationTest

2023-12-01 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-15891: --- Labels: flaky-test (was: ) > Flaky test: testResetSinkConnectorOffsetsOverriddenConsumerGro

Re: [PR] KAFKA-14133: Migrate ProcessorStateManagerTest and StreamThreadTest to Mockito [kafka]

2023-12-01 Thread via GitHub
clolov commented on code in PR #13932: URL: https://github.com/apache/kafka/pull/13932#discussion_r1412177600 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamThreadTest.java: ## @@ -3494,32 +3323,27 @@ private StreamThread setUpThread(final Properties

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
lianetm commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1412178458 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -322,16 +314,14 @@ private void onErrorResponse(final Consumer

Re: [PR] KAFKA-14133: Migrate ProcessorStateManagerTest and StreamThreadTest to Mockito [kafka]

2023-12-01 Thread via GitHub
clolov commented on PR #13932: URL: https://github.com/apache/kafka/pull/13932#issuecomment-1836217374 Heya @cadonna! Many thanks for the review and apologies for the delay. Yes, it makes perfect sense, I also came across ``` I=0; while ./gradlew clients:test --tests RequestResponseTes

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412179127 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -666,12 +766,16 @@ public Map committed(final Set

Re: [PR] KAFKA-15890: Consumer.poll with long timeout unaware of assigned partitions [kafka]

2023-12-01 Thread via GitHub
lucasbru merged PR #14835: URL: https://github.com/apache/kafka/pull/14835 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-15890) Consumer.poll with long timeout unaware of assigned partitions

2023-12-01 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15890. -- Resolution: Fixed > Consumer.poll with long timeout unaware of assigned partitions > -

Re: [PR] KAFKA-15426: Process and persist directory assignments [kafka]

2023-12-01 Thread via GitHub
soarez commented on PR #14863: URL: https://github.com/apache/kafka/pull/14863#issuecomment-1836233799 Ready for review. Draft because it depends on https://github.com/apache/kafka/pull/14838 so ignore the first commit. @cmccabe @rondagostino @pprovenzano -- This is an automated m

[jira] [Created] (KAFKA-15954) Review minimal effort approach on consumer last heartbeat on unsubscribe

2023-12-01 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15954: -- Summary: Review minimal effort approach on consumer last heartbeat on unsubscribe Key: KAFKA-15954 URL: https://issues.apache.org/jira/browse/KAFKA-15954 Project:

[jira] [Created] (KAFKA-15953) Refactor polling delays

2023-12-01 Thread Andrew Schofield (Jira)
Andrew Schofield created KAFKA-15953: Summary: Refactor polling delays Key: KAFKA-15953 URL: https://issues.apache.org/jira/browse/KAFKA-15953 Project: Kafka Issue Type: Sub-task

[jira] [Assigned] (KAFKA-15954) Review minimal effort approach on consumer last heartbeat on unsubscribe

2023-12-01 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15954: -- Assignee: Lianet Magrans > Review minimal effort approach on consumer last heartbeat

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
lianetm commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1412204434 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -365,6 +366,16 @@ private void replaceUnresolvedAssignmentWithNew

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
lianetm commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1412204434 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -365,6 +366,16 @@ private void replaceUnresolvedAssignmentWithNew

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
dajac commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1836265424 @lianetm There are conflicts. Could you please rebase or merge trunk? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
dajac commented on code in PR #14857: URL: https://github.com/apache/kafka/pull/14857#discussion_r1412217786 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -365,6 +366,16 @@ private void replaceUnresolvedAssignmentWithNewAs

Re: [PR] KAFKA-14509: [2/N] Implement server side logic for ConsumerGroupDescribe API [kafka]

2023-12-01 Thread via GitHub
riedelmax commented on PR #14544: URL: https://github.com/apache/kafka/pull/14544#issuecomment-1836279426 Hey @dajac thanks for the review. I addressed all the points except for the ones we intent to do a follow up. -- This is an automated message from the Apache Git Service. To respo

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
lianetm commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1836283568 Done, latest changes merged @dajac, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-01 Thread via GitHub
lianetm commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1836284959 All test pass locally, I will keep an eye on the build here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[jira] [Created] (KAFKA-15955) Migrating ZK brokers send dir assignments

2023-12-01 Thread Igor Soarez (Jira)
Igor Soarez created KAFKA-15955: --- Summary: Migrating ZK brokers send dir assignments Key: KAFKA-15955 URL: https://issues.apache.org/jira/browse/KAFKA-15955 Project: Kafka Issue Type: Sub-task

[PR] MINOR: Add @Timeout annotation to consumer integration tests [kafka]

2023-12-01 Thread via GitHub
dajac opened a new pull request, #14896: URL: https://github.com/apache/kafka/pull/14896 In this [buid](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14826/11/pipeline/12/), the following test hang forever. ``` Gradle Test Run :core:test > Grad

[PR] KAFKA-15953: Refactor polling delays [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield opened a new pull request, #14897: URL: https://github.com/apache/kafka/pull/14897 Caches the maximum time to wait in the consumer network thread so the application thread is better isolated from the request managers. ### Committer Checklist (excluded from commit mess

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

2023-12-01 Thread via GitHub
vamossagar12 commented on code in PR #14882: URL: https://github.com/apache/kafka/pull/14882#discussion_r1412299630 ## core/src/test/scala/unit/kafka/server/ConsumerGroupHeartbeatRequestTest.scala: ## @@ -137,6 +137,98 @@ class ConsumerGroupHeartbeatRequestTest(cluster: Cluster

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on PR #14879: URL: https://github.com/apache/kafka/pull/14879#issuecomment-1836362310 > Can you elaborate on the direction to remove the background queue from the 'test builder' instead of using the one it constructed? I had issues with tests using the spy on the `As

[jira] [Assigned] (KAFKA-15955) Migrating ZK brokers send dir assignments

2023-12-01 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano reassigned KAFKA-15955: - Assignee: Proven Provenzano > Migrating ZK brokers send dir assignments > -

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
AndrewJSchofield commented on PR #14879: URL: https://github.com/apache/kafka/pull/14879#issuecomment-1836371437 > > Can you elaborate on the direction to remove the background queue from the 'test builder' instead of using the one it constructed? > > I had issues with tests using the

Re: [PR] MINOR: disable test_transactions with new group coordinator [kafka]

2023-12-01 Thread via GitHub
jolshan commented on PR #14884: URL: https://github.com/apache/kafka/pull/14884#issuecomment-1836445897 This is a python change so I will merge. These OutOfMemory issues are not good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] MINOR: disable test_transactions with new group coordinator [kafka]

2023-12-01 Thread via GitHub
jolshan merged PR #14884: URL: https://github.com/apache/kafka/pull/14884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-15281: Implement the groupMetadata Consumer API [kafka]

2023-12-01 Thread via GitHub
cadonna commented on code in PR #14879: URL: https://github.com/apache/kafka/pull/14879#discussion_r1412360413 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -849,9 +953,22 @@ public OptionalLong currentLag(TopicPartition top

Re: [PR] KAFKA-15953: Refactor polling delays [kafka]

2023-12-01 Thread via GitHub
lucasbru commented on code in PR #14897: URL: https://github.com/apache/kafka/pull/14897#discussion_r1412360054 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -51,7 +51,7 @@ */ public class ConsumerNetworkThread extends

Re: [PR] HOTFIX: fix ConsistencyVectorIntegrationTest failure [kafka]

2023-12-01 Thread via GitHub
hanyuzheng7 commented on PR #14895: URL: https://github.com/apache/kafka/pull/14895#issuecomment-1836500935 Thanks @lucasbru ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-01 Thread via GitHub
lianetm commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1412390835 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -181,11 +183,13 @@ private void process(final ListOffs

  1   2   >