Re: [PR] KAFKA-16659: KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER [kafka]

2024-05-03 Thread via GitHub
FrankYang0529 commented on code in PR #15853: URL: https://github.com/apache/kafka/pull/15853#discussion_r1589912438 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -906,6 +906,7 @@ public long position(TopicPartition partition

[jira] [Updated] (KAFKA-16666) Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`to tools module

2024-05-03 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-1: --- Description: `GroupMetadataMessageFormatter`[0], `OffsetsMessageFormatter`[1], and `Transac

[jira] [Updated] (KAFKA-16666) Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`to tools module

2024-05-03 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-1: --- Description: After https://github.com/apache/kafka/pull/15652, both `GroupMetadataMessageFo

[jira] [Updated] (KAFKA-16666) Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`to tools module

2024-05-03 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-1: --- Summary: Migrate `TransactionLogMessageFormatter`, GroupMetadataMessageFormatter` and `Offse

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on PR #15652: URL: https://github.com/apache/kafka/pull/15652#issuecomment-2094019717 > Those two classes are meant to be used with the console consumer. This is why they are not referenced anywhere. thanks for the reminder ! > I wonder if we should provide

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
dajac commented on PR #15652: URL: https://github.com/apache/kafka/pull/15652#issuecomment-2094014351 @chia7712 Those two classes are meant to be used with the console consumer. This is why they are not referenced anywhere. I wonder if we should provide an equivalent one in java. -- This

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on PR #15652: URL: https://github.com/apache/kafka/pull/15652#issuecomment-209382 > Yeah, I noticed those classes too. I am actually not sure what we kept them. It may be for backward compatibility. I need to dig a little more on it. It is anyway not related to this p

[jira] [Created] (KAFKA-16666) Remove unused class `GroupMetadataMessageFormatter` and `OffsetsMessageFormatter`

2024-05-03 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-1: -- Summary: Remove unused class `GroupMetadataMessageFormatter` and `OffsetsMessageFormatter` Key: KAFKA-1 URL: https://issues.apache.org/jira/browse/KAFKA-1

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on PR #15652: URL: https://github.com/apache/kafka/pull/15652#issuecomment-2093999641 There is a build having weird output, so I re-trigger QA -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16608: Honour interrupted thread state on KafkaConsumer.poll [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on PR #15803: URL: https://github.com/apache/kafka/pull/15803#issuecomment-2093997309 @lianetm any feedbacks? I'd like to merge it later :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] KAFKA-16027: MINOR Refactor MetadataTest#testUpdatePartitionLeadership (2) [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15857: URL: https://github.com/apache/kafka/pull/15857#discussion_r1589872138 ## clients/src/test/java/org/apache/kafka/clients/MetadataTest.java: ## @@ -590,76 +681,92 @@ public void testClusterCopy() { errors.put("topic3", Errors.INVA

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1589865773 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2992,6 +2992,11 @@ class KafkaApis(val requestChannel: RequestChannel, val preprocessingResponses = co

Re: [PR] KAFKA-15723: KRaft support in ListOffsetsRequestTest [kafka]

2024-05-03 Thread via GitHub
github-actions[bot] commented on PR #15047: URL: https://github.com/apache/kafka/pull/15047#issuecomment-2093988531 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] Window not honored? Why is X record not dropped? [kafka]

2024-05-03 Thread via GitHub
github-actions[bot] commented on PR #15314: URL: https://github.com/apache/kafka/pull/15314#issuecomment-2093988508 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] KAFKA-16484: Support to define per broker/controller property by ClusterConfigProperty [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15715: URL: https://github.com/apache/kafka/pull/15715#discussion_r1589861459 ## core/src/test/java/kafka/test/annotation/ClusterConfigProperty.java: ## @@ -27,6 +27,23 @@ @Target({ElementType.ANNOTATION_TYPE}) @Retention(RetentionPolicy.RUNTI

Re: [PR] KAFKA-16427: KafkaConsumer#position() does not respect timeout when group protocol is CONSUMER [kafka]

2024-05-03 Thread via GitHub
chia7712 merged PR #15843: URL: https://github.com/apache/kafka/pull/15843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15836: URL: https://github.com/apache/kafka/pull/15836#discussion_r1589821796 ## core/src/main/scala/kafka/server/FetchSession.scala: ## @@ -430,7 +438,10 @@ class FullFetchContext(private val time: Time, } cachedPartitions }

Re: [PR] KAFKA-16027: MINOR Refactor MetadataTest#testUpdatePartitionLeadership [kafka]

2024-05-03 Thread via GitHub
Alexander-Aghili commented on PR #15055: URL: https://github.com/apache/kafka/pull/15055#issuecomment-2093906904 > > I have the correct file still saved on my new forced update but I can't link it to this issue. I don't want to revert because I accidently pushed all the commits from the pas

[PR] KAFKA-16027: MINOR Refactor MetadataTest#testUpdatePartitionLeadership (2) [kafka]

2024-05-03 Thread via GitHub
Alexander-Aghili opened a new pull request, #15857: URL: https://github.com/apache/kafka/pull/15857 Previous PR had issues, this contains the up to date change: Link to previous PR: [KAFKA-16027: MINOR Refactor MetadataTest#testUpdatePartitionLeadership](https://github.com/apache/kafka/pu

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589807041 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589803572 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on PR #15766: URL: https://github.com/apache/kafka/pull/15766#issuecomment-2093737716 Thanks for the updates @frankvicky! Left some comments. Almost there... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589657729 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589705662 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589701677 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-15265: Add Remote Log Manager quota manager [kafka]

2024-05-03 Thread via GitHub
junrao commented on code in PR #15625: URL: https://github.com/apache/kafka/pull/15625#discussion_r1589688863 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -181,7 +189,8 @@ public RemoteLogManager(RemoteLogManagerConfig rlmConfig,

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589688305 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-03 Thread via GitHub
jsancio commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1589681992 ## raft/src/main/java/org/apache/kafka/snapshot/RecordsSnapshotWriter.java: ## @@ -216,4 +140,132 @@ private void appendBatches(List> batches) { batches.f

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589680017 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

[jira] [Resolved] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16474. Resolution: Fixed Ran the test several times, the client log also looks fine. > AsyncKafkaConsume

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589669265 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16207; KRaft's internal log listener to update voter set [kafka]

2024-05-03 Thread via GitHub
hachikuji commented on code in PR #15671: URL: https://github.com/apache/kafka/pull/15671#discussion_r1589667403 ## raft/src/main/java/org/apache/kafka/snapshot/RecordsSnapshotWriter.java: ## @@ -216,4 +140,132 @@ private void appendBatches(List> batches) { batches

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589652714 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589657729 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
lianetm commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589652714 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,381 @@ package org.apache.kafka.tools.consumer.group;

[jira] [Commented] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843326#comment-17843326 ] Philip Nee commented on KAFKA-16474: Yeah I think we still need to figure why networ

[jira] [Commented] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843324#comment-17843324 ] Lianet Magrans commented on KAFKA-16474: Yes, makes sense, that PR is addressing

Re: [PR] KAFKA-16027: MINOR Refactor MetadataTest#testUpdatePartitionLeadership [kafka]

2024-05-03 Thread via GitHub
Alexander-Aghili commented on PR #15055: URL: https://github.com/apache/kafka/pull/15055#issuecomment-2093622997 Ok will do. Ill link it here when done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Commented] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843320#comment-17843320 ] Philip Nee commented on KAFKA-16474: Hey [~lianetm]  - i'm planning to verify the fi

[jira] [Commented] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843318#comment-17843318 ] Lianet Magrans commented on KAFKA-16474: Hey [~pnee], this is the PR https://git

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15835: URL: https://github.com/apache/kafka/pull/15835#discussion_r1589614396 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorRuntimeMetrics.java: ## @@ -208,8 +209,8 @@ public void recordEventQueueTime

Re: [PR] KAFKA-14995: Automate asf.yaml collaborators refresh [kafka]

2024-05-03 Thread via GitHub
stevenbooke closed pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh URL: https://github.com/apache/kafka/pull/13842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[jira] [Assigned] (KAFKA-16474) AsyncKafkaConsumer might send out heartbeat request without waiting for its response

2024-05-03 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee reassigned KAFKA-16474: -- Assignee: Philip Nee (was: Lianet Magrans) > AsyncKafkaConsumer might send out heartbeat req

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-05-03 Thread via GitHub
mumrah commented on PR #15744: URL: https://github.com/apache/kafka/pull/15744#issuecomment-2093513532 So many conflicts 😭 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] KAFKA-16655: Deflaking ZKMigrationIntegrationTest.testDualWrite [kafka]

2024-05-03 Thread via GitHub
cmccabe merged PR #15845: URL: https://github.com/apache/kafka/pull/15845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-16655: Deflaking ZKMigrationIntegrationTest.testDualWrite [kafka]

2024-05-03 Thread via GitHub
ahuang98 commented on PR #15845: URL: https://github.com/apache/kafka/pull/15845#issuecomment-2093452127 test failures look unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-16615: JoinGroup API for upgrading ConsumerGroup [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15798: URL: https://github.com/apache/kafka/pull/15798#discussion_r1589476625 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -10921,6 +10822,1146 @@ public void testLastConsumerProtocolMe

Re: [PR] KAFKA-10199: Add remove operation with future to state updater [kafka]

2024-05-03 Thread via GitHub
cadonna commented on code in PR #15852: URL: https://github.com/apache/kafka/pull/15852#discussion_r1589452435 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -496,6 +504,103 @@ private void addTask(final Task task) {

Re: [PR] KAFKA-16665: Allow to initialize newly assigned partition's positions without allowing fetching while callback runs [kafka]

2024-05-03 Thread via GitHub
lianetm commented on PR #15856: URL: https://github.com/apache/kafka/pull/15856#issuecomment-2093364129 Hey @lucasbru , could you take a look? For more context, we initially decided not to allow initializing positions while the onPartitionsAssigned callback was running, concerned abo

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-03 Thread via GitHub
gaurav-narula commented on code in PR #15836: URL: https://github.com/apache/kafka/pull/15836#discussion_r1589418655 ## core/src/main/scala/kafka/server/FetchSession.scala: ## @@ -583,9 +595,13 @@ case class EvictableKey(privileged: Boolean, size: Int, id: Int) extends Compara

[jira] [Updated] (KAFKA-16665) Fail to get partition's position from within onPartitionsAssigned callback in new consumer

2024-05-03 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16665: --- Description: If we attempt to call consumer.position(tp) from within the onPartitionsAssign

[PR] KAFKA-15309: part 2-Add custom error handler for UnknownTopicOrPartitionException to Producer [kafka]

2024-05-03 Thread via GitHub
aliehsaeedii opened a new pull request, #15855: URL: https://github.com/apache/kafka/pull/15855 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] KAFKA-16507 Add raw record into RecordDeserialisationException [kafka]

2024-05-03 Thread via GitHub
fvaleri commented on code in PR #15691: URL: https://github.com/apache/kafka/pull/15691#discussion_r1589379241 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CompletedFetch.java: ## @@ -311,25 +311,35 @@ ConsumerRecord parseRecord(Deserializers deserialize

[jira] [Commented] (KAFKA-16511) Leaking tiered segments

2024-05-03 Thread Francois Visconte (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843253#comment-17843253 ] Francois Visconte commented on KAFKA-16511: --- [~ckamal][~showuon] I pulled the

[jira] [Comment Edited] (KAFKA-16511) Leaking tiered segments

2024-05-03 Thread Francois Visconte (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843253#comment-17843253 ] Francois Visconte edited comment on KAFKA-16511 at 5/3/24 3:39 PM: ---

[PR] MINOR: Remove `ConsumerGroupPartitionMetadataValue.Epoch` field [kafka]

2024-05-03 Thread via GitHub
dajac opened a new pull request, #15854: URL: https://github.com/apache/kafka/pull/15854 `ConsumerGroupPartitionMetadataValue.Epoch` is not used anywhere so we can remove it. Note that we already have non-backward compatible changes lined up for 3.8 so it is fine to do it. ### Commit

Re: [PR] KAFKA-16659: KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15853: URL: https://github.com/apache/kafka/pull/15853#discussion_r1589364098 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -906,6 +906,7 @@ public long position(TopicPartition partition, Dur

Re: [PR] KAFKA-16659: KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15853: URL: https://github.com/apache/kafka/pull/15853#discussion_r1589362449 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -906,6 +906,7 @@ public long position(TopicPartition partition, Dur

[jira] [Created] (KAFKA-16665) Fail to get partition's position from within onPartitionsAssigned callback in new consumer

2024-05-03 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16665: -- Summary: Fail to get partition's position from within onPartitionsAssigned callback in new consumer Key: KAFKA-16665 URL: https://issues.apache.org/jira/browse/KAFKA-16665

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-03 Thread via GitHub
AndrewJSchofield commented on code in PR #15836: URL: https://github.com/apache/kafka/pull/15836#discussion_r1589348685 ## core/src/main/scala/kafka/server/FetchSession.scala: ## @@ -395,19 +396,27 @@ object FullFetchContext { * The fetch context for a full fetch request.

Re: [PR] KAFKA-10199: Add remove operation with future to state updater [kafka]

2024-05-03 Thread via GitHub
cadonna commented on code in PR #15852: URL: https://github.com/apache/kafka/pull/15852#discussion_r1589354020 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StateUpdater.java: ## @@ -103,6 +149,17 @@ public String toString() { */ void remove(fi

Re: [PR] KAFKA-10199: Add remove operation with future to state updater [kafka]

2024-05-03 Thread via GitHub
cadonna commented on code in PR #15852: URL: https://github.com/apache/kafka/pull/15852#discussion_r1589353331 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -496,6 +504,103 @@ private void addTask(final Task task) {

Re: [PR] KAFKA-10199: Add remove operation with future to state updater [kafka]

2024-05-03 Thread via GitHub
cadonna commented on code in PR #15852: URL: https://github.com/apache/kafka/pull/15852#discussion_r1589352418 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -191,14 +193,20 @@ private void performActionsOnTasks() {

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-03 Thread via GitHub
jeffkbkim commented on code in PR #15835: URL: https://github.com/apache/kafka/pull/15835#discussion_r1589348895 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessor.java: ## @@ -126,9 +126,16 @@ private class EventProcessorTh

[jira] [Commented] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-05-03 Thread sanghyeok An (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843244#comment-17843244 ] sanghyeok An commented on KAFKA-16657: -- Thanks for your comments.  I totally under

[jira] [Created] (KAFKA-16664) Re-add EventAccumulator.take(timeout)

2024-05-03 Thread Jeff Kim (Jira)
Jeff Kim created KAFKA-16664: Summary: Re-add EventAccumulator.take(timeout) Key: KAFKA-16664 URL: https://issues.apache.org/jira/browse/KAFKA-16664 Project: Kafka Issue Type: Sub-task

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-03 Thread via GitHub
jeffkbkim commented on code in PR #15835: URL: https://github.com/apache/kafka/pull/15835#discussion_r1589342058 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorRuntimeMetrics.java: ## @@ -208,8 +209,8 @@ public void recordEventQueue

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-03 Thread via GitHub
jeffkbkim commented on code in PR #15835: URL: https://github.com/apache/kafka/pull/15835#discussion_r1589342890 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorRuntimeMetrics.java: ## @@ -208,8 +209,8 @@ public void recordEventQueue

Re: [PR] KAFKA-16307: fix coordinator thread idle ratio [kafka]

2024-05-03 Thread via GitHub
jeffkbkim commented on code in PR #15835: URL: https://github.com/apache/kafka/pull/15835#discussion_r1589342058 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorRuntimeMetrics.java: ## @@ -208,8 +209,8 @@ public void recordEventQueue

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-03 Thread via GitHub
gaurav-narula commented on code in PR #15836: URL: https://github.com/apache/kafka/pull/15836#discussion_r1589339685 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -541,9 +541,17 @@ class KafkaServer( }.toMap } -val fetchManager = ne

Re: [PR] KAFKA-16615: JoinGroup API for upgrading ConsumerGroup [kafka]

2024-05-03 Thread via GitHub
dongnuo123 commented on code in PR #15798: URL: https://github.com/apache/kafka/pull/15798#discussion_r1589336191 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -10921,6 +10822,1146 @@ public void testLastConsumerProto

Re: [PR] KAFKA-16615: JoinGroup API for upgrading ConsumerGroup [kafka]

2024-05-03 Thread via GitHub
dongnuo123 commented on code in PR #15798: URL: https://github.com/apache/kafka/pull/15798#discussion_r1589336191 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -10921,6 +10822,1146 @@ public void testLastConsumerProto

[jira] [Commented] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-05-03 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843236#comment-17843236 ] Andrew Schofield commented on KAFKA-16657: -- Ah, sorry for the jargon :) Apache

Re: [PR] KAFKA-15756: [3/3] Migrate testConsumptionWithBrokerFailures to run old protocol in new coordinator [kafka]

2024-05-03 Thread via GitHub
dongnuo123 closed pull request #15547: KAFKA-15756: [3/3] Migrate testConsumptionWithBrokerFailures to run old protocol in new coordinator URL: https://github.com/apache/kafka/pull/15547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
frankvicky commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589307637 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/DeleteConsumerGroupsTest.java: ## @@ -17,279 +17,382 @@ package org.apache.kafka.tools.consumer.group;

[jira] [Commented] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-05-03 Thread sanghyeok An (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843231#comment-17843231 ] sanghyeok An commented on KAFKA-16657: -- Thanks for conforming it!!  Sorry to say,

Re: [PR] KAFKA-9401: Reduce contention for Fetch requests [kafka]

2024-05-03 Thread via GitHub
gaurav-narula commented on code in PR #15836: URL: https://github.com/apache/kafka/pull/15836#discussion_r1589305094 ## core/src/main/scala/kafka/server/BrokerServer.scala: ## @@ -389,9 +389,17 @@ class BrokerServer( authorizer = config.createNewAuthorizer() author

Re: [PR] KAFKA-16593: Rewrite DeleteConsumerGroupsTest by ClusterTestExtensions [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15766: URL: https://github.com/apache/kafka/pull/15766#discussion_r1589298707 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupExecutor.java: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[jira] [Commented] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-05-03 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843227#comment-17843227 ] Andrew Schofield commented on KAFKA-16657: -- I just checked the KIP and I don't

Re: [PR] KAFKA-10199: Add remove operation with future to state updater [kafka]

2024-05-03 Thread via GitHub
lucasbru commented on code in PR #15852: URL: https://github.com/apache/kafka/pull/15852#discussion_r1589251704 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -191,14 +193,20 @@ private void performActionsOnTasks() {

[jira] [Updated] (KAFKA-16663) CoordinatorRuntime write timer tasks should be cancelled once HWM advances

2024-05-03 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-16663: - Description: Otherwise, we pile up the number of timer tasks which are no-ops if replication was success

[jira] [Commented] (KAFKA-14579) Move DumpLogSegments to tools

2024-05-03 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843219#comment-17843219 ] Chia-Ping Tsai commented on KAFKA-14579: After https://github.com/apache/kafka/p

[jira] [Created] (KAFKA-16663) CoordinatorRuntime write timer tasks should be cancelled once HWM advances

2024-05-03 Thread Jeff Kim (Jira)
Jeff Kim created KAFKA-16663: Summary: CoordinatorRuntime write timer tasks should be cancelled once HWM advances Key: KAFKA-16663 URL: https://issues.apache.org/jira/browse/KAFKA-16663 Project: Kafka

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15652: URL: https://github.com/apache/kafka/pull/15652#discussion_r1589206306 ## core/src/test/scala/unit/kafka/tools/DumpLogSegmentsTest.scala: ## @@ -399,6 +404,191 @@ class DumpLogSegmentsTest { assertEquals(partialBatches, partialBatchesCo

Re: [PR] KAFKA-16587: Add subscription model information to group state [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15785: URL: https://github.com/apache/kafka/pull/15785#discussion_r1589199642 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1307,17 +1305,25 @@ private CoordinatorResult consumerGr

Re: [PR] MINOR: ConsumerGroup#getOrMaybeCreateMember should not add the member to the group [kafka]

2024-05-03 Thread via GitHub
dajac merged PR #15847: URL: https://github.com/apache/kafka/pull/15847 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-16470; kafka-dump-log --offsets-decoder should support new records [kafka]

2024-05-03 Thread via GitHub
chia7712 commented on code in PR #15652: URL: https://github.com/apache/kafka/pull/15652#discussion_r1589164350 ## core/src/test/scala/unit/kafka/tools/DumpLogSegmentsTest.scala: ## @@ -399,6 +404,191 @@ class DumpLogSegmentsTest { assertEquals(partialBatches, partialBatche

[jira] [Commented] (KAFKA-16657) KIP-848 does not work well on Zookeeper Mode

2024-05-03 Thread sanghyeok An (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843212#comment-17843212 ] sanghyeok An commented on KAFKA-16657: -- [~schofielaj] Thanks for your comments.  I

Re: [PR] KAFKA-14509: [3/4] Add integration test for consumerGroupDescribe Api [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15727: URL: https://github.com/apache/kafka/pull/15727#discussion_r1589172884 ## core/src/test/scala/unit/kafka/server/ConsumerGroupDescribeRequestsTest.scala: ## @@ -0,0 +1,175 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-16399: Add JBOD support in tiered storage [kafka]

2024-05-03 Thread via GitHub
soarez commented on PR #15690: URL: https://github.com/apache/kafka/pull/15690#issuecomment-2092980307 Thanks for fixing the import. There are some failing tests, please have a look. I think we need to delete `kafka.server.KafkaConfigTest#testMultipleLogDirectoriesNotSupportedWithRem

Re: [PR] [1/N] ConfigCommandTest rewritten in java [kafka]

2024-05-03 Thread via GitHub
nizhikov commented on code in PR #15850: URL: https://github.com/apache/kafka/pull/15850#discussion_r1589172610 ## core/src/test/java/kafka/admin/JConfigCommandTest.java: ## @@ -0,0 +1,420 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contribu

[PR] KAFKA-16659: KafkaConsumer#position() does not respect wakup when group protocol is CONSUMER [kafka]

2024-05-03 Thread via GitHub
FrankYang0529 opened a new pull request, #15853: URL: https://github.com/apache/kafka/pull/15853 From KafkaConsumer#position comments, we should return WakeupException when KafkaConsumer#wakeup is called. https://github.com/apache/kafka/blob/240243b91d69c2b65b5e456065fdcce90c121b04/c

[jira] [Updated] (KAFKA-16595) Introduce ClusterTemplate in ClusterTests

2024-05-03 Thread Kuan Po Tseng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan Po Tseng updated KAFKA-16595: -- Summary: Introduce ClusterTemplate in ClusterTests (was: Introduce template in ClusterTests)

[jira] [Commented] (KAFKA-16511) Leaking tiered segments

2024-05-03 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17843204#comment-17843204 ] Luke Chen commented on KAFKA-16511: --- OK, I'll review it again. Thanks. > Leaking tier

[jira] [Assigned] (KAFKA-16174) Flaky test: testDescribeQuorumStatusSuccessful – org.apache.kafka.tools.MetadataQuorumCommandTest

2024-05-03 Thread Johnny Hsu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johnny Hsu reassigned KAFKA-16174: -- Assignee: Johnny Hsu > Flaky test: testDescribeQuorumStatusSuccessful – > org.apache.kafka.t

Re: [PR] KAFKA-16587: Add subscription model information to group state [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15785: URL: https://github.com/apache/kafka/pull/15785#discussion_r1589109858 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1307,17 +1305,25 @@ private CoordinatorResult consumerGr

[jira] [Assigned] (KAFKA-16661) add a lower `log.initial.task.delay.ms` value to integration test framework

2024-05-03 Thread Chia Chuan Yu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia Chuan Yu reassigned KAFKA-16661: - Assignee: Chia Chuan Yu > add a lower `log.initial.task.delay.ms` value to integration

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-03 Thread via GitHub
cadonna commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2092808466 > Is there any problem if we leave `awaitPendingAsyncCommitsAndExecuteCommitCallbacks` as is? It clearly needs the timer Yeah, I think that we should leave this as it is for now.

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-03 Thread via GitHub
cadonna commented on code in PR #15640: URL: https://github.com/apache/kafka/pull/15640#discussion_r1589007924 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/events/CompletableEventReaperTest.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] MINOR: ConsumerGroup#getOrMaybeCreateMember should not add the member to the group [kafka]

2024-05-03 Thread via GitHub
dajac commented on code in PR #15847: URL: https://github.com/apache/kafka/pull/15847#discussion_r1589072912 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilderTest.java: ## @@ -182,14 +172,16 @@ public TargetAssignmentBuilder.T

  1   2   >