Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1568482951 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/ServerSideAssignorBenchmark.java: ## @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568308820 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568325101 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568325101 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568325101 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-17 Thread via GitHub
jeqo commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1568385072 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/MultiFieldPaths.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-17 Thread via GitHub
jeqo commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1568385972 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/MultiFieldPaths.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568277518 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,20 @@ void enqueueMetadataChangeEvent( // Events

[PR] debug for 15679 [kafka]

2024-04-17 Thread via GitHub
chia7712 opened a new pull request, #15741: URL: https://github.com/apache/kafka/pull/15741 as title -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568311272 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1568327439 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2024-04-17 Thread via GitHub
vamossagar12 commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-2060631731 hey @ableegoldman .. i am interested in wrapping this up but i haven't looked at the streams codebase for some time and things seemed to have changed a bit. I am not aware of the

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1568383960 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/group_coordinator/ServerSideAssignorBenchmark.java: ## @@ -0,0 +1,185 @@ +package

Re: [PR] KAFKA-15588 ConfigCommandIntegrationTest rewritten in java [kafka]

2024-04-17 Thread via GitHub
nizhikov commented on code in PR #15645: URL: https://github.com/apache/kafka/pull/15645#discussion_r1568506632 ## core/src/test/java/kafka/admin/ConfigCommandIntegrationTest.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-16363: Storage tool crashes if dir is unavailable [kafka]

2024-04-17 Thread via GitHub
chia7712 merged PR #15733: URL: https://github.com/apache/kafka/pull/15733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: fix duplicated return and other streams docs typos [kafka]

2024-04-17 Thread via GitHub
AyoubOm commented on PR #15713: URL: https://github.com/apache/kafka/pull/15713#issuecomment-2060923661 @chia7712 please check this when you have time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568737120 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -775,6 +777,126 @@ public ClassicGroup classicGroup( }

Re: [PR] MINOR: Various cleanups in connect [kafka]

2024-04-17 Thread via GitHub
chia7712 merged PR #15734: URL: https://github.com/apache/kafka/pull/15734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16566: Fix consumer static membership system test with new protocol [kafka]

2024-04-17 Thread via GitHub
lucasbru commented on PR #15738: URL: https://github.com/apache/kafka/pull/15738#issuecomment-2060943337 @lianetm did you mean to closet the PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions [kafka]

2024-04-17 Thread via GitHub
mimaison commented on code in PR #15469: URL: https://github.com/apache/kafka/pull/15469#discussion_r1568692760 ## connect/api/src/main/java/org/apache/kafka/connect/data/Values.java: ## @@ -766,135 +852,23 @@ protected static boolean canParseSingleTokenLiteral(Parser parser,

[PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-17 Thread via GitHub
lucasbru opened a new pull request, #15742: URL: https://github.com/apache/kafka/pull/15742 When user-defined rebalance listeners fail with an exception, the expectation is that the error should be propagated to the user as a KafkaExpception and break the poll loop (behaviour in the legacy

Re: [PR] MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions [kafka]

2024-04-17 Thread via GitHub
gharris1727 commented on code in PR #15469: URL: https://github.com/apache/kafka/pull/15469#discussion_r1568679258 ## connect/api/src/main/java/org/apache/kafka/connect/data/Values.java: ## @@ -766,135 +852,23 @@ protected static boolean canParseSingleTokenLiteral(Parser

Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-17 Thread via GitHub
lucasbru commented on PR #15742: URL: https://github.com/apache/kafka/pull/15742#issuecomment-2060865659 @kirktrue @lianetm could you have a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR adds '-parameters' compiler option for :core tests [kafka]

2024-04-17 Thread via GitHub
chia7712 merged PR #15729: URL: https://github.com/apache/kafka/pull/15729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions [kafka]

2024-04-17 Thread via GitHub
mimaison commented on code in PR #15469: URL: https://github.com/apache/kafka/pull/15469#discussion_r1568541184 ## connect/api/src/main/java/org/apache/kafka/connect/data/Values.java: ## @@ -766,135 +852,23 @@ protected static boolean canParseSingleTokenLiteral(Parser parser,

Re: [PR] KAFKA-15853: Move consumer group and group coordinator configs out of core [kafka]

2024-04-17 Thread via GitHub
chia7712 merged PR #15684: URL: https://github.com/apache/kafka/pull/15684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (KAFKA-16298) Ensure user callbacks exceptions are propagated to the user on consumer poll

2024-04-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy reassigned KAFKA-16298: -- Assignee: Lucas Brutschy (was: Kirk True) > Ensure user callbacks exceptions are

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-17 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1568882032 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1176,6 +1176,42 @@ class LogManager(logDirs: Seq[File], } } + def

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1568904632 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,61 @@ class LogManager(logDirs: Seq[File], val sourceLog =

[PR] Fix typo [kafka]

2024-04-17 Thread via GitHub
birdoplank opened a new pull request, #15743: URL: https://github.com/apache/kafka/pull/15743 Testing potential security vulnerability in the pipeline to be reported as part of Apache vulnerability disclosure program: https://apache.org/security/#vulnerability-handling ###

[PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
mumrah opened a new pull request, #15744: URL: https://github.com/apache/kafka/pull/15744 This patch fixes two issues with IncrementalAlterConfigs and the ZK migration. First, it changes the handling of IncrementalAlterConfigs to check if the controller is ZK vs KRaft and only forward for

Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-17 Thread via GitHub
lianetm commented on code in PR #15742: URL: https://github.com/apache/kafka/pull/15742#discussion_r1568972567 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -1333,32 +1342,41 @@ public void

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dongnuo123 commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568994033 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -998,4 +1094,232 @@ public

Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]

2024-04-17 Thread via GitHub
OmniaGM commented on PR #15728: URL: https://github.com/apache/kafka/pull/15728#issuecomment-2061493764 Hi @aaron-ai thanks for the PR. Can you please have a look into the failing tests herehttps://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15728/1/#showFailuresLink I believe they

Re: [PR] KAFKA-16552: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests [kafka]

2024-04-17 Thread via GitHub
brandboat commented on code in PR #15719: URL: https://github.com/apache/kafka/pull/15719#discussion_r1569022628 ## core/src/test/scala/unit/kafka/log/LogManagerTest.scala: ## @@ -413,7 +413,7 @@ class LogManagerTest { assertEquals(numMessages * setSize / segmentBytes,

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-17 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1569131419 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,61 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-17 Thread via GitHub
lucasbru commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1568770827 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1388,6 +1393,31 @@ public void commitSync(Map offsets, Duration

Re: [PR] MINOR: Add test for PartitionMetadataFile [kafka]

2024-04-17 Thread via GitHub
KevinZTW commented on code in PR #15714: URL: https://github.com/apache/kafka/pull/15714#discussion_r1568968759 ## storage/src/test/java/org/apache/kafka/storage/internals/checkpoint/PartitionMetadataFileTest.java: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dongnuo123 commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568994033 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -998,4 +1094,232 @@ public

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
mumrah commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1569103206 ## core/src/main/scala/kafka/zk/KafkaZkClient.scala: ## @@ -467,13 +470,33 @@ class KafkaZkClient private[zk] (zooKeeperClient: ZooKeeperClient, isSecure: Boo *

[PR] KAFKA-16572: allow defining number of disks per broker in ClusterTest [kafka]

2024-04-17 Thread via GitHub
FrankYang0529 opened a new pull request, #15745: URL: https://github.com/apache/kafka/pull/15745 We introduced `disksPerBroker` in `TestKitNodes` from https://issues.apache.org/jira/browse/KAFKA-16559. We can support to config it in `ClusterTest`, so it's more convenient for integration

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-17 Thread via GitHub
mimaison commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1569073857 ## core/src/main/scala/kafka/server/ZkAdminManager.scala: ## @@ -79,10 +80,10 @@ class ZkAdminManager(val config: KafkaConfig, private val configHelper = new

Re: [PR] MINOR: Fix io-[wait-]ratio metrics description [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15722: URL: https://github.com/apache/kafka/pull/15722#discussion_r1568836486 ## clients/src/main/java/org/apache/kafka/common/network/Selector.java: ## @@ -1281,14 +1281,14 @@ private Meter createMeter(Metrics metrics, String groupName, Map

Re: [PR] MINOR: fix duplicated return and other streams docs typos [kafka]

2024-04-17 Thread via GitHub
chia7712 merged PR #15713: URL: https://github.com/apache/kafka/pull/15713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dongnuo123 commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568963034 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -775,6 +777,126 @@ public ClassicGroup classicGroup(

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dongnuo123 commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568962281 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -775,6 +777,126 @@ public ClassicGroup classicGroup(

Re: [PR] MINOR: Fix io-[wait-]ratio metrics description [kafka]

2024-04-17 Thread via GitHub
emitskevich-blp commented on code in PR #15722: URL: https://github.com/apache/kafka/pull/15722#discussion_r1569005969 ## clients/src/main/java/org/apache/kafka/common/network/Selector.java: ## @@ -1281,14 +1281,14 @@ private Meter createMeter(Metrics metrics, String

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
johnnychhsu commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1569005655 ## core/src/main/scala/kafka/zk/KafkaZkClient.scala: ## @@ -467,13 +470,33 @@ class KafkaZkClient private[zk] (zooKeeperClient: ZooKeeperClient, isSecure: Boo

Re: [PR] KAFKA-15853: Refactor org.apache.kafka.server.config.Defaults to follow Java config pattern [kafka]

2024-04-17 Thread via GitHub
OmniaGM closed pull request #15260: KAFKA-15853: Refactor org.apache.kafka.server.config.Defaults to follow Java config pattern URL: https://github.com/apache/kafka/pull/15260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1569051446 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -775,6 +777,126 @@ public ClassicGroup classicGroup( }

Re: [PR] MINOR: fix duplicated return and other streams docs typos [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on PR #15713: URL: https://github.com/apache/kafka/pull/15713#issuecomment-2061259436 @AyoubOm thanks for your contribution -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dongnuo123 commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1568962281 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -775,6 +777,126 @@ public ClassicGroup classicGroup(

Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-17 Thread via GitHub
lianetm commented on code in PR #15742: URL: https://github.com/apache/kafka/pull/15742#discussion_r1568991555 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -228,13 +228,16 @@ private void process(final ErrorEvent event) {

Re: [PR] KAFKA-15853: Refactor org.apache.kafka.server.config.Defaults to follow Java config pattern [kafka]

2024-04-17 Thread via GitHub
OmniaGM commented on PR #15260: URL: https://github.com/apache/kafka/pull/15260#issuecomment-2061522296 Am going to close this as we already started to move defaults out of Default and into the scoped config classes. -- This is an automated message from the Apache Git Service. To

Re: [PR] KAFKA-16424: remove truncated logs after alter dir [kafka]

2024-04-17 Thread via GitHub
FrankYang0529 commented on code in PR #15616: URL: https://github.com/apache/kafka/pull/15616#discussion_r1569024432 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -801,8 +803,23 @@ private Void deleteTypeIfExists(StorageAction delete,

[jira] [Commented] (KAFKA-16467) Add README to docs folder

2024-04-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838190#comment-17838190 ] ASF GitHub Bot commented on KAFKA-16467: FrankYang0529 commented on code in PR #596: URL:

Re: [PR] KAFKA-15853: Move KafkaConfig properties definition out of core [kafka]

2024-04-17 Thread via GitHub
OmniaGM commented on PR #15501: URL: https://github.com/apache/kafka/pull/15501#issuecomment-2061520938 Am going to close this one as it drifted away from the approach we took to break the configs out. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15853: Move KafkaConfig properties definition out of core [kafka]

2024-04-17 Thread via GitHub
OmniaGM closed pull request #15501: KAFKA-15853: Move KafkaConfig properties definition out of core URL: https://github.com/apache/kafka/pull/15501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1569077848 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -777,6 +778,59 @@ public ClassicGroup classicGroup( }

[jira] [Created] (KAFKA-16573) Streams does not specify where a Serde is needed

2024-04-17 Thread Ayoub Omari (Jira)
Ayoub Omari created KAFKA-16573: --- Summary: Streams does not specify where a Serde is needed Key: KAFKA-16573 URL: https://issues.apache.org/jira/browse/KAFKA-16573 Project: Kafka Issue Type:

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-17 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1568883111 ## core/src/test/java/kafka/testkit/BrokerNode.java: ## @@ -81,8 +82,7 @@ public BrokerNode build( logDataDirectories = Collections.

Re: [PR] KAFKA-16424: remove truncated logs after alter dir [kafka]

2024-04-17 Thread via GitHub
FrankYang0529 commented on code in PR #15616: URL: https://github.com/apache/kafka/pull/15616#discussion_r1568897953 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -801,8 +803,23 @@ private Void deleteTypeIfExists(StorageAction delete,

Re: [PR] KAFKA-15309: Add custom error handler to Producer [kafka]

2024-04-17 Thread via GitHub
OmniaGM commented on PR #15731: URL: https://github.com/apache/kafka/pull/15731#issuecomment-2061484219 Hi @aliehsaeedii thanks for the PoC and the KIP. Can you please mark this PoC as draft PR as the KIP still under discussion? -- This is an automated message from the Apache Git

Re: [PR] MINOR: Fix io-[wait-]ratio metrics description [kafka]

2024-04-17 Thread via GitHub
emitskevich-blp commented on code in PR #15722: URL: https://github.com/apache/kafka/pull/15722#discussion_r1569005969 ## clients/src/main/java/org/apache/kafka/common/network/Selector.java: ## @@ -1281,14 +1281,14 @@ private Meter createMeter(Metrics metrics, String

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-17 Thread via GitHub
dajac commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1569055496 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -998,4 +1094,232 @@ public

Re: [PR] KAFKA-16445: Add PATCH method for connector config [kafka]

2024-04-17 Thread via GitHub
C0urante commented on code in PR #6934: URL: https://github.com/apache/kafka/pull/6934#discussion_r1568959278 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/resources/ConnectorsResource.java: ## @@ -242,6 +242,19 @@ public Response

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
mumrah commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1569214270 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -3003,9 +3008,10 @@ class KafkaApis(val requestChannel: RequestChannel, } } +// Forwarding

Re: [PR] Kafka-16540: Update partitions if min isr config is changed. [kafka]

2024-04-17 Thread via GitHub
CalvinConfluent commented on code in PR #15702: URL: https://github.com/apache/kafka/pull/15702#discussion_r1569214320 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -2117,6 +2117,20 @@ ListPartitionReassignmentsResponseData

Re: [PR] MINOR: move topic configuration defaults [kafka]

2024-04-17 Thread via GitHub
cmccabe closed pull request #10589: MINOR: move topic configuration defaults URL: https://github.com/apache/kafka/pull/10589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-17 Thread via GitHub
kamalcph commented on code in PR #15634: URL: https://github.com/apache/kafka/pull/15634#discussion_r1563885907 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -282,15 +282,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** * Update high watermark

[jira] [Resolved] (KAFKA-16405) Mismatch assignment error when running consumer rolling upgrade system tests

2024-04-17 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-16405. --- Reviewer: Lucas Brutschy Resolution: Fixed > Mismatch assignment error when running consumer

Re: [PR] KAFKA-16572: allow defining number of disks per broker in ClusterTest [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on code in PR #15745: URL: https://github.com/apache/kafka/pull/15745#discussion_r1569310965 ## core/src/test/java/kafka/testkit/TestKitNodes.java: ## @@ -82,6 +82,10 @@ public Builder setNumBrokerNodes(int numBrokerNodes) { return

Re: [PR] KAFKA-16565: IncrementalAssignmentConsumerEventHandler throws error when attempting to remove a partition that isn't assigned [kafka]

2024-04-17 Thread via GitHub
lianetm commented on code in PR #15737: URL: https://github.com/apache/kafka/pull/15737#discussion_r1569396277 ## tests/kafkatest/services/verifiable_consumer.py: ## @@ -140,22 +150,32 @@ class IncrementalAssignmentConsumerEventHandler(ConsumerEventHandler): def

Re: [PR] KAFKA-16572: allow defining number of disks per broker in ClusterTest [kafka]

2024-04-17 Thread via GitHub
gaurav-narula commented on code in PR #15745: URL: https://github.com/apache/kafka/pull/15745#discussion_r1569160539 ## core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java: ## @@ -93,7 +93,7 @@ public List getAdditionalExtensions() {

[PR] KAFKA-16461: New consumer fails to consume records in security_test.py system test [kafka]

2024-04-17 Thread via GitHub
kirktrue opened a new pull request, #15746: URL: https://github.com/apache/kafka/pull/15746 The system test was failing because the `VerifiableConsumer` failed with a `NullPointerException` during startup. The reason for the NPE was an attempt to put a `null` as the value of

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-17 Thread via GitHub
kirktrue commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1569232314 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -616,6 +620,80 @@ public void

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-17 Thread via GitHub
kirktrue commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1569231774 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -616,6 +620,80 @@ public void

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-17 Thread via GitHub
kirktrue commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1569228945 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -616,6 +620,80 @@ public void

Re: [PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-17 Thread via GitHub
junrao commented on PR #15631: URL: https://github.com/apache/kafka/pull/15631#issuecomment-2061732009 @kamalcph : Thanks for triaging the failed tests. The PR LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-17 Thread via GitHub
junrao merged PR #15631: URL: https://github.com/apache/kafka/pull/15631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16483: migrate DeleteOffsetsConsumerGroupCommandIntegrationTest to use ClusterTestExtensions [kafka]

2024-04-17 Thread via GitHub
chia7712 commented on PR #15679: URL: https://github.com/apache/kafka/pull/15679#issuecomment-2061751961 @FrankYang0529 Could you reduce the partition number of offsets topic? It seems the timeout is caused by that coordinator is waiting for the offset partition, and our CI could be too

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2024-04-17 Thread via GitHub
ableegoldman commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-2061827689 No pressure! Just wanted to make sure you weren't still waiting on me for this. Or rather, just wanted to say, if you do want to try and get this in to 3.8 I will make sure to help

Re: [PR] KAFKA-14370: Simplify the ImageWriter API by adding freeze [kafka]

2024-04-17 Thread via GitHub
cmccabe closed pull request #12833: KAFKA-14370: Simplify the ImageWriter API by adding freeze URL: https://github.com/apache/kafka/pull/12833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] MINOR: Fix some flaky behavior in DeleteConsumerGroupsTest and clean up [kafka]

2024-04-17 Thread via GitHub
cmccabe closed pull request #8115: MINOR: Fix some flaky behavior in DeleteConsumerGroupsTest and clean up URL: https://github.com/apache/kafka/pull/8115 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (KAFKA-16003) The znode /config/topics is not updated during KRaft migration in "dual-write" mode

2024-04-17 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-16003: - Fix Version/s: 3.7.1 > The znode /config/topics is not updated during KRaft migration in >

Re: [PR] KAFKA-16480: Bump ListOffsets version, IBP version and mark last version of ListOffsets as unstable [kafka]

2024-04-17 Thread via GitHub
junrao commented on code in PR #15673: URL: https://github.com/apache/kafka/pull/15673#discussion_r1569221453 ## clients/src/main/java/org/apache/kafka/common/requests/ListOffsetsRequest.java: ## @@ -70,7 +70,7 @@ else if (isolationLevel == IsolationLevel.READ_COMMITTED)

[jira] [Commented] (KAFKA-16522) Admin client changes for adding and removing voters

2024-04-17 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838347#comment-17838347 ] José Armando García Sancio commented on KAFKA-16522: [~phong260702] thanks for

[jira] [Created] (KAFKA-16574) The metrics of LogCleaner disappear after reconfiguration

2024-04-17 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16574: -- Summary: The metrics of LogCleaner disappear after reconfiguration Key: KAFKA-16574 URL: https://issues.apache.org/jira/browse/KAFKA-16574 Project: Kafka

[PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-17 Thread via GitHub
kamalcph opened a new pull request, #15747: URL: https://github.com/apache/kafka/pull/15747 cherry-picked from: d092787487047178c674bd60a182129a8d997c4a Co-authored-by: hzh0425 <642256...@qq.com> ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[jira] [Created] (KAFKA-16575) Automatically remove KTable aggregation result when group becomes empty

2024-04-17 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-16575: --- Summary: Automatically remove KTable aggregation result when group becomes empty Key: KAFKA-16575 URL: https://issues.apache.org/jira/browse/KAFKA-16575

[PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-17 Thread via GitHub
kamalcph opened a new pull request, #15748: URL: https://github.com/apache/kafka/pull/15748 cherry-picked from: d092787487047178c674bd60a182129a8d997c4a Co-authored-by: hzh0425 <642256...@qq.com> ### Committer Checklist (excluded from commit message) - [ ] Verify design and

Re: [PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-17 Thread via GitHub
junrao commented on PR #15631: URL: https://github.com/apache/kafka/pull/15631#issuecomment-2061739168 @kamalcph : Do you want this PR to be in 3.7 and 3.6? If so, could you create a separate cherry-pick RP? Thanks. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
akhileshchg commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1569190991 ## core/src/main/scala/kafka/zk/KafkaZkClient.scala: ## @@ -467,13 +470,33 @@ class KafkaZkClient private[zk] (zooKeeperClient: ZooKeeperClient, isSecure: Boo

Re: [PR] KAFKA-16561: Disable `allow.auto.create.topics` in MirrorMaker2 Consumer Config [kafka]

2024-04-17 Thread via GitHub
C0urante commented on code in PR #15728: URL: https://github.com/apache/kafka/pull/15728#discussion_r1569197938 ## connect/mirror/src/main/java/org/apache/kafka/connect/mirror/MirrorConnectorConfig.java: ## @@ -169,6 +170,7 @@ static Map sourceConsumerConfig(Map props) {

Re: [PR] KAFKA-16539 Fix IncrementalAlterConfigs during ZK migration [kafka]

2024-04-17 Thread via GitHub
mumrah commented on code in PR #15744: URL: https://github.com/apache/kafka/pull/15744#discussion_r1569208039 ## core/src/main/scala/kafka/zk/KafkaZkClient.scala: ## @@ -467,13 +470,33 @@ class KafkaZkClient private[zk] (zooKeeperClient: ZooKeeperClient, isSecure: Boo *

[jira] [Updated] (KAFKA-16570) FenceProducers API returns "unexpected error" when successful

2024-04-17 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-16570: --- Description: When we want to fence a producer using the admin client, we send an

Re: [PR] Kafka-16540: Update partitions if min isr config is changed. [kafka]

2024-04-17 Thread via GitHub
CalvinConfluent commented on code in PR #15702: URL: https://github.com/apache/kafka/pull/15702#discussion_r1569210550 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -2360,4 +2363,9 @@ void setNewNextWriteOffset(long newNextWriteOffset) {

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-17 Thread via GitHub
kirktrue commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1569226961 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -793,9 +795,9 @@ public void commitAsync(Map offsets, OffsetCo

Re: [PR] Kafka-16540: Update partitions if min isr config is changed. [kafka]

2024-04-17 Thread via GitHub
splett2 commented on code in PR #15702: URL: https://github.com/apache/kafka/pull/15702#discussion_r1569269269 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -2117,6 +2117,20 @@ ListPartitionReassignmentsResponseData

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-17 Thread via GitHub
kamalcph commented on code in PR #15634: URL: https://github.com/apache/kafka/pull/15634#discussion_r1569328393 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -282,15 +282,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** * Update high watermark

  1   2   >