[jira] [Commented] (KAFKA-15709) KRaft support in ServerStartupTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838640#comment-17838640 ] Matthew de Detrich commented on KAFKA-15709: [~linzihao1999] are you still working on this

[jira] [Comment Edited] (KAFKA-15737) KRaft support in ConsumerBounceTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838639#comment-17838639 ] Matthew de Detrich edited comment on KAFKA-15737 at 4/18/24 12:39 PM:

[jira] [Commented] (KAFKA-15737) KRaft support in ConsumerBounceTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838639#comment-17838639 ] Matthew de Detrich commented on KAFKA-15737: Ill have a look at this > KRaft support in

[jira] [Commented] (KAFKA-14547) Be able to run kafka KRaft Server in tests without needing to run a storage setup script

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838637#comment-17838637 ] Matthew de Detrich commented on KAFKA-14547: Ill have a look into this > Be able to run

[jira] [Commented] (KAFKA-16378) Under tiered storage, deleting local logs does not free disk space

2024-04-18 Thread Ivan Yurchenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838635#comment-17838635 ] Ivan Yurchenko commented on KAFKA-16378: This was a problem in the RemoteStorageManager. I'll

[jira] [Closed] (KAFKA-16378) Under tiered storage, deleting local logs does not free disk space

2024-04-18 Thread Ivan Yurchenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Yurchenko closed KAFKA-16378. -- Was a problem in the remote storage manager plugin > Under tiered storage, deleting local logs

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on PR #15634: URL: https://github.com/apache/kafka/pull/15634#issuecomment-2063727744 Sorry that the story I mentioned above seems be another issue. Let me have the summary about my thought. 1. `log-start-offset-checkpoint` is missing and remote storage is enabled.

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1570596642 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-14569: Migrate Kafka Streams tests from Zookeeper to KRaft [kafka]

2024-04-18 Thread via GitHub
mdedetrich commented on PR #15341: URL: https://github.com/apache/kafka/pull/15341#issuecomment-2063692520 @OmniaGM So the structure of the PR/tests is now finalized, I ended up entirely deleting `KafkaEmbedded` and so now the tests are a lot closer to idioomatic testing with

[jira] [Created] (KAFKA-16585) No way to forward message from punctuation method in the FixedKeyProcessor

2024-04-18 Thread Stanislav Spiridonov (Jira)
Stanislav Spiridonov created KAFKA-16585: Summary: No way to forward message from punctuation method in the FixedKeyProcessor Key: KAFKA-16585 URL: https://issues.apache.org/jira/browse/KAFKA-16585

Re: [PR] MINOR:Add hint for `resumeDeletions` in `TopicDeletionManager` [kafka]

2024-04-18 Thread via GitHub
hudeqi commented on PR #15543: URL: https://github.com/apache/kafka/pull/15543#issuecomment-2063665249 Hi, @jlprat , could you help to review this pr? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] MINOR:Optimize the log output when cleaning up offsets for groups with a generation less than or equal to 0 [kafka]

2024-04-18 Thread via GitHub
hudeqi commented on PR #15726: URL: https://github.com/apache/kafka/pull/15726#issuecomment-2063663065 @dajac Hi, does this PR make sense for the group's offset cleanup log output? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Comment Edited] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao edited comment on KAFKA-16578 at 4/18/24 11:11 AM: - [~kirktrue], I

[jira] [Comment Edited] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao edited comment on KAFKA-16578 at 4/18/24 11:10 AM: - [~kirktrue], I

[jira] [Commented] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao commented on KAFKA-16578: --- [~kirktrue], I am working on modifying the the system tests for

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
mimaison commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063586815 Feel free to merge once CI completes. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
jlprat commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063587938 Thanks @mimaison. Will do! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-16552: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests [kafka]

2024-04-18 Thread via GitHub
brandboat commented on PR #15719: URL: https://github.com/apache/kafka/pull/15719#issuecomment-2063574931 Thanks for the reminder, soarez ! Already fix the error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1569131419 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,61 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570449270 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,63 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570449270 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,63 @@ class LogManager(logDirs: Seq[File], val sourceLog =

[jira] [Comment Edited] (KAFKA-8041) Flaky Test LogDirFailureTest#testIOExceptionDuringLogRoll

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838583#comment-17838583 ] Omnia Ibrahim edited comment on KAFKA-8041 at 4/18/24 10:14 AM: [~soarez]

[jira] [Commented] (KAFKA-8041) Flaky Test LogDirFailureTest#testIOExceptionDuringLogRoll

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838583#comment-17838583 ] Omnia Ibrahim commented on KAFKA-8041: -- [~soarez] I believe this should be fixed now after the merge

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
OmniaGM commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1570406354 ## server/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
jlprat commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063483812 @mimaison Rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-16552: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests [kafka]

2024-04-18 Thread via GitHub
soarez commented on PR #15719: URL: https://github.com/apache/kafka/pull/15719#issuecomment-2063477565 Thanks for the changes. You have some compilation errors: ``` [2024-04-17T15:27:16.786Z] > Task :jmh-benchmarks:compileJava [2024-04-17T15:27:16.786Z]

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
mimaison commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063460572 @jlprat Can you rebase to resolve the conflict? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570371982 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1178,6 +1178,33 @@ class LogManager(logDirs: Seq[File], } } + def

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
mimaison commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1570367941 ## server/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] debug for 15679 [kafka]

2024-04-18 Thread via GitHub
chia7712 closed pull request #15741: debug for 15679 URL: https://github.com/apache/kafka/pull/15741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on PR #15634: URL: https://github.com/apache/kafka/pull/15634#issuecomment-2063435567 > HWM is set to to localLogStartOffset in

[jira] [Commented] (KAFKA-15089) Consolidate all the group coordinator configs

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838562#comment-17838562 ] Omnia Ibrahim commented on KAFKA-15089: --- Hi [~dajac] should we mark this as resolved now as we

Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-18 Thread via GitHub
lucasbru commented on code in PR #15742: URL: https://github.com/apache/kafka/pull/15742#discussion_r1570342805 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -228,13 +228,16 @@ private void process(final ErrorEvent event) {

Re: [PR] KAFKA-16555: Consumer's RequestState has incorrect logic to determine if inflight [kafka]

2024-04-18 Thread via GitHub
lucasbru commented on PR #15723: URL: https://github.com/apache/kafka/pull/15723#issuecomment-2063300823 @lianetm did you want to make another pass or good to go? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-16073: Increment the local-log-start-offset before deleting segments in memory table [kafka]

2024-04-18 Thread via GitHub
showuon commented on PR #15748: URL: https://github.com/apache/kafka/pull/15748#issuecomment-2063302914 @kamalcph , the tests are still failing: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15748/ -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-18 Thread via GitHub
dajac commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1570226001 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/ServerSideAssignorBenchmark.java: ## @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1570226078 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +537,45 @@ public void

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1570217037 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +537,45 @@ public void

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1570111991 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +537,45 @@ public void

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1570217037 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +537,45 @@ public void

[PR] KAFKA-16583: fix PartitionRegistration#toRecord directory check under metadata version 3_7_IV2 [kafka]

2024-04-18 Thread via GitHub
superhx opened a new pull request, #15751: URL: https://github.com/apache/kafka/pull/15751 fix https://issues.apache.org/jira/browse/KAFKA-16583 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[jira] [Created] (KAFKA-16584) Make log processing summary configurable or debug

2024-04-18 Thread Andras Hatvani (Jira)
Andras Hatvani created KAFKA-16584: -- Summary: Make log processing summary configurable or debug Key: KAFKA-16584 URL: https://issues.apache.org/jira/browse/KAFKA-16584 Project: Kafka Issue

[jira] [Updated] (KAFKA-16584) Make log processing summary configurable or debug

2024-04-18 Thread Andras Hatvani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Hatvani updated KAFKA-16584: --- Description: Currently *every two minutes for every stream thread* statistics will be

[jira] [Assigned] (KAFKA-16574) The metrics of LogCleaner disappear after reconfiguration

2024-04-18 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16574: -- Assignee: Chia Chuan Yu (was: Chia-Ping Tsai) > The metrics of LogCleaner disappear

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-18 Thread via GitHub
rreddy-22 commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1570135556 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/TargetAssignmentBuilderBenchmark.java: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-18 Thread via GitHub
rreddy-22 commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1570135213 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/TargetAssignmentBuilderBenchmark.java: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-18 Thread via GitHub
rreddy-22 commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1570128341 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/TargetAssignmentBuilderBenchmark.java: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16568: JMH Benchmarks for Server Side Rebalances [kafka]

2024-04-18 Thread via GitHub
rreddy-22 commented on code in PR #15717: URL: https://github.com/apache/kafka/pull/15717#discussion_r1570122759 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/assignor/TargetAssignmentBuilderBenchmark.java: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-16583) Update from 3.4.0 to 3.7.0 image write failed in Kraft mode

2024-04-18 Thread HanXu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HanXu updated KAFKA-16583: -- Description: How to reproduce: 1. Launch a 3.4.0 controller and a 3.4.0 broker(BrokerA) in Kraft mode; 2.

[jira] [Created] (KAFKA-16583) Update from 3.4.0 to 3.7.0 image write failed in Kraft mode

2024-04-18 Thread HanXu (Jira)
HanXu created KAFKA-16583: - Summary: Update from 3.4.0 to 3.7.0 image write failed in Kraft mode Key: KAFKA-16583 URL: https://issues.apache.org/jira/browse/KAFKA-16583 Project: Kafka Issue Type:

Re: [PR] KAFKA-15396:Add a metric indicating the version of the current running kafka server [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #14284: KAFKA-15396:Add a metric indicating the version of the current running kafka server URL: https://github.com/apache/kafka/pull/14284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15134:Enrich the prompt reason in CommitFailedException [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #13930: KAFKA-15134:Enrich the prompt reason in CommitFailedException URL: https://github.com/apache/kafka/pull/13930 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-14824: ReplicaAlterLogDirsThread may cause serious disk growing in case of potential exception [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #13421: KAFKA-14824: ReplicaAlterLogDirsThread may cause serious disk growing in case of potential exception URL: https://github.com/apache/kafka/pull/13421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15086:Set a reasonable segment size upper limit for MM2 internal topics [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #13852: KAFKA-15086:Set a reasonable segment size upper limit for MM2 internal topics URL: https://github.com/apache/kafka/pull/13852 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-14906:Extract the coordinator service log from server log [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #13568: KAFKA-14906:Extract the coordinator service log from server log URL: https://github.com/apache/kafka/pull/13568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15119:Support incremental syncTopicAcls in MirrorSourceConnector [kafka]

2024-04-18 Thread via GitHub
hudeqi closed pull request #13913: KAFKA-15119:Support incremental syncTopicAcls in MirrorSourceConnector URL: https://github.com/apache/kafka/pull/13913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

<    1   2