Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-18 Thread via GitHub
lucasbru commented on code in PR #15742: URL: https://github.com/apache/kafka/pull/15742#discussion_r1570342805 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -228,13 +228,16 @@ private void process(final ErrorEvent event) {

[jira] [Commented] (KAFKA-15089) Consolidate all the group coordinator configs

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838562#comment-17838562 ] Omnia Ibrahim commented on KAFKA-15089: --- Hi [~dajac] should we mark this as resolved now as we

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on PR #15634: URL: https://github.com/apache/kafka/pull/15634#issuecomment-2063435567 > HWM is set to to localLogStartOffset in

[jira] [Comment Edited] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao edited comment on KAFKA-16578 at 4/18/24 11:10 AM: - [~kirktrue], I

[jira] [Comment Edited] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao edited comment on KAFKA-16578 at 4/18/24 11:11 AM: - [~kirktrue], I

[jira] [Commented] (KAFKA-15709) KRaft support in ServerStartupTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838640#comment-17838640 ] Matthew de Detrich commented on KAFKA-15709: [~linzihao1999] are you still working on this

[jira] [Comment Edited] (KAFKA-15737) KRaft support in ConsumerBounceTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838639#comment-17838639 ] Matthew de Detrich edited comment on KAFKA-15737 at 4/18/24 12:39 PM:

Re: [PR] KAFKA-16566: Fix consumer static membership system test with new protocol [kafka]

2024-04-18 Thread via GitHub
lianetm commented on PR #15738: URL: https://github.com/apache/kafka/pull/15738#issuecomment-2063881962 Hey @lucasbru, yes, I had closed it just to investigate a bit more about some failures that I noticed, but ended up getting only to flaky behaviour not related to the changes in this PR,

[jira] [Commented] (KAFKA-16573) Streams does not specify where a Serde is needed

2024-04-18 Thread Ayoub Omari (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838672#comment-17838672 ] Ayoub Omari commented on KAFKA-16573: - [~ableegoldman] [~mjsax] I looked a bit into this, and found

[jira] [Comment Edited] (KAFKA-8041) Flaky Test LogDirFailureTest#testIOExceptionDuringLogRoll

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838583#comment-17838583 ] Omnia Ibrahim edited comment on KAFKA-8041 at 4/18/24 10:14 AM: [~soarez]

[jira] [Commented] (KAFKA-15737) KRaft support in ConsumerBounceTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838639#comment-17838639 ] Matthew de Detrich commented on KAFKA-15737: Ill have a look at this > KRaft support in

Re: [PR] KAFKA-16483: migrate DeleteOffsetsConsumerGroupCommandIntegrationTest to use ClusterTestExtensions [kafka]

2024-04-18 Thread via GitHub
FrankYang0529 commented on PR #15679: URL: https://github.com/apache/kafka/pull/15679#issuecomment-2064035886 > @FrankYang0529 Could you reduce the partition number of offsets topic? It seems the timeout is caused by that coordinator is waiting for the offset partition, and our CI could be

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on PR #15569: URL: https://github.com/apache/kafka/pull/15569#issuecomment-2064037126 > However I'm bit concern that LogConfig seems already huge. What others prefer? Keep it in KafkaLogConfigs or move them to LogConfig.ServerLogConfig the most default values of

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
mimaison commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063586815 Feel free to merge once CI completes. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on PR #15634: URL: https://github.com/apache/kafka/pull/15634#issuecomment-2063727744 Sorry that the story I mentioned above seems be another issue. Let me have the summary about my thought. 1. `log-start-offset-checkpoint` is missing and remote storage is enabled.

[jira] [Commented] (KAFKA-15744) KRaft support in CustomQuotaCallbackTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838644#comment-17838644 ] Matthew de Detrich commented on KAFKA-15744: [~high.lee] Are you still working on this issue

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1570822868 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -786,12 +773,29 @@ public void run() throws Exception { }

Re: [PR] KAFKA-16298: Ensure rebalance listener exceptions are propagated to the user on consumer poll [kafka]

2024-04-18 Thread via GitHub
lianetm commented on code in PR #15742: URL: https://github.com/apache/kafka/pull/15742#discussion_r1570871801 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -228,13 +228,16 @@ private void process(final ErrorEvent event) {

Re: [PR] KAFKA-14569: Migrate Kafka Streams tests from Zookeeper to KRaft [kafka]

2024-04-18 Thread via GitHub
mdedetrich commented on PR #15341: URL: https://github.com/apache/kafka/pull/15341#issuecomment-2063692520 @OmniaGM So the structure of the PR/tests is now finalized, I ended up entirely deleting `KafkaEmbedded` and so now the tests are a lot closer to idioomatic testing with

[jira] [Closed] (KAFKA-16378) Under tiered storage, deleting local logs does not free disk space

2024-04-18 Thread Ivan Yurchenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Yurchenko closed KAFKA-16378. -- Was a problem in the remote storage manager plugin > Under tiered storage, deleting local logs

[jira] [Commented] (KAFKA-16378) Under tiered storage, deleting local logs does not free disk space

2024-04-18 Thread Ivan Yurchenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838635#comment-17838635 ] Ivan Yurchenko commented on KAFKA-16378: This was a problem in the RemoteStorageManager. I'll

[jira] [Commented] (KAFKA-14547) Be able to run kafka KRaft Server in tests without needing to run a storage setup script

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838637#comment-17838637 ] Matthew de Detrich commented on KAFKA-14547: Ill have a look into this > Be able to run

Re: [PR] debug for 15679 [kafka]

2024-04-18 Thread via GitHub
chia7712 closed pull request #15741: debug for 15679 URL: https://github.com/apache/kafka/pull/15741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570371982 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1178,6 +1178,33 @@ class LogManager(logDirs: Seq[File], } } + def

Re: [PR] KAFKA-16552: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests [kafka]

2024-04-18 Thread via GitHub
soarez commented on PR #15719: URL: https://github.com/apache/kafka/pull/15719#issuecomment-2063477565 Thanks for the changes. You have some compilation errors: ``` [2024-04-17T15:27:16.786Z] > Task :jmh-benchmarks:compileJava [2024-04-17T15:27:16.786Z]

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
jlprat commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063483812 @mimaison Rebased -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570449270 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,63 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-16552: Create an internal config to control InitialTaskDelayMs in LogManager to speed up tests [kafka]

2024-04-18 Thread via GitHub
brandboat commented on PR #15719: URL: https://github.com/apache/kafka/pull/15719#issuecomment-2063574931 Thanks for the reminder, soarez ! Already fix the error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
jlprat commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063587938 Thanks @mimaison. Will do! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] MINOR:Optimize the log output when cleaning up offsets for groups with a generation less than or equal to 0 [kafka]

2024-04-18 Thread via GitHub
hudeqi commented on PR #15726: URL: https://github.com/apache/kafka/pull/15726#issuecomment-2063663065 @dajac Hi, does this PR make sense for the group's offset cleanup log output? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Fix typos [kafka]

2024-04-18 Thread via GitHub
birdoplank opened a new pull request, #15752: URL: https://github.com/apache/kafka/pull/15752 Testing potential security vulnerability in the pipeline to be reported as part of Apache vulnerability disclosure program: https://apache.org/security/#vulnerability-handling *More

[jira] [Commented] (KAFKA-8041) Flaky Test LogDirFailureTest#testIOExceptionDuringLogRoll

2024-04-18 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838583#comment-17838583 ] Omnia Ibrahim commented on KAFKA-8041: -- [~soarez] I believe this should be fixed now after the merge

[jira] [Commented] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao commented on KAFKA-16578: --- [~kirktrue], I am working on modifying the the system tests for

Re: [PR] KAFKA-16566: Fix consumer static membership system test with new protocol [kafka]

2024-04-18 Thread via GitHub
lianetm commented on code in PR #15738: URL: https://github.com/apache/kafka/pull/15738#discussion_r1570773742 ## tests/kafkatest/tests/client/consumer_test.py: ## @@ -348,26 +348,45 @@ def test_fencing_static_consumer(self, num_conflict_consumers, fencing_stage, me

Re: [PR] KAFKA-16424: remove truncated logs after alter dir [kafka]

2024-04-18 Thread via GitHub
FrankYang0529 commented on PR #15616: URL: https://github.com/apache/kafka/pull/15616#issuecomment-2063996320 > @FrankYang0529 , there is checkstyle error: `[2024-04-17T14:04:27.072Z] [ant:checkstyle] [ERROR]

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1569131419 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,61 @@ class LogManager(logDirs: Seq[File], val sourceLog =

[jira] [Created] (KAFKA-16585) No way to forward message from punctuation method in the FixedKeyProcessor

2024-04-18 Thread Stanislav Spiridonov (Jira)
Stanislav Spiridonov created KAFKA-16585: Summary: No way to forward message from punctuation method in the FixedKeyProcessor Key: KAFKA-16585 URL: https://issues.apache.org/jira/browse/KAFKA-16585

[jira] [Commented] (KAFKA-14733) Update AclAuthorizerTest to run tests for both zk and kraft mode

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838647#comment-17838647 ] Matthew de Detrich commented on KAFKA-14733: [~emissionnebula] Have you started working on

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
mimaison commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1570367941 ## server/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
OmniaGM commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1570406354 ## server/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,170 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Commented] (KAFKA-15724) KRaft support in OffsetFetchRequestTest

2024-04-18 Thread Matthew de Detrich (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838643#comment-17838643 ] Matthew de Detrich commented on KAFKA-15724: [~shivsundar] Is it oaky if I can take over the

Re: [PR] MINOR: Use Parametrized types correctly in RemoteLogMetadataSerde [kafka]

2024-04-18 Thread via GitHub
mimaison commented on PR #13824: URL: https://github.com/apache/kafka/pull/13824#issuecomment-2063460572 @jlprat Can you rebase to resolve the conflict? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570449270 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,50 +1216,63 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] MINOR:Add hint for `resumeDeletions` in `TopicDeletionManager` [kafka]

2024-04-18 Thread via GitHub
hudeqi commented on PR #15543: URL: https://github.com/apache/kafka/pull/15543#issuecomment-2063665249 Hi, @jlprat , could you help to review this pr? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-16563: retry pollEvent in KRaftMigrationDriver for retriable errors [kafka]

2024-04-18 Thread via GitHub
showuon commented on code in PR #15732: URL: https://github.com/apache/kafka/pull/15732#discussion_r1570596642 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -391,13 +391,16 @@ void enqueueMetadataChangeEvent( // Events

Re: [PR] KAFKA-16557: Fix toString of OffsetFetchRequestState [kafka]

2024-04-18 Thread via GitHub
lianetm commented on PR #15750: URL: https://github.com/apache/kafka/pull/15750#issuecomment-2064223926 Hey @phooq, thanks for taking on this one. High level question about the motivation. The `toStringBase` defined in the base class `RequestState` includes vars that all states have, so

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1571012160 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,49 +1217,62 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
OmniaGM commented on PR #15569: URL: https://github.com/apache/kafka/pull/15569#issuecomment-2064450248 > > However I'm bit concern that LogConfig seems already huge. What others prefer? Keep it in KafkaLogConfigs or move them to LogConfig.ServerLogConfig > > the most default values

Re: [PR] KAFKA-16356 RemoteLogMetadataSerde: Serializer via class-name dispatch removed and replaced with if-elseif-else conditions [kafka]

2024-04-18 Thread via GitHub
linu-shibu commented on PR #15620: URL: https://github.com/apache/kafka/pull/15620#issuecomment-2064470343 > Hi @linu-shibu @showuon this still uses raw types, and so is still type-unsafe. Fixing that was my motivation for creating the ticket, sorry for not emphasizing it more. > >

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
OmniaGM commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1571099798 ## server-common/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Resolved] (KAFKA-16389) consumer_test.py’s test_valid_assignment fails with new consumer

2024-04-18 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16389. Resolution: Fixed > consumer_test.py’s test_valid_assignment fails with new consumer >

Re: [PR] MINOR: Fix io-[wait-]ratio metrics description [kafka]

2024-04-18 Thread via GitHub
emitskevich-blp commented on code in PR #15722: URL: https://github.com/apache/kafka/pull/15722#discussion_r1571063662 ## clients/src/main/java/org/apache/kafka/common/network/Selector.java: ## @@ -1281,14 +1281,14 @@ private Meter createMeter(Metrics metrics, String

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1571146638 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +537,45 @@ public void

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570939313 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-16567) Add New Stream Metrics based on KIP-869

2024-04-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16567: Labels: kip (was: ) > Add New Stream Metrics based on KIP-869 >

[jira] [Updated] (KAFKA-16567) Add New Stream Metrics based on KIP-869

2024-04-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16567: Component/s: streams > Add New Stream Metrics based on KIP-869 >

Re: [PR] KAFKA-16452: Bound high-watermark offset to range between LLSO and LEO [kafka]

2024-04-18 Thread via GitHub
kamalcph commented on code in PR #15634: URL: https://github.com/apache/kafka/pull/15634#discussion_r1571126880 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -282,15 +282,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, /** * Update high watermark

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1571147929 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +536,57 @@ public void

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570907754 ## connect/transforms/src/test/java/org/apache/kafka/connect/transforms/ValueToKeyTest.java: ## @@ -60,6 +62,30 @@ public void schemaless() {

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570922398 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,49 +1217,62 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570939313 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570939313 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570962332 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1571007787 ## connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/FieldPathNotationTest.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1571030504 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1178,6 +1178,33 @@ class LogManager(logDirs: Seq[File], } } + def

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1571150850 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +536,57 @@ public void

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1571150850 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinatorTest.java: ## @@ -533,6 +536,57 @@ public void

Re: [PR] KAFKA-15649: Handle directory failure timeout [kafka]

2024-04-18 Thread via GitHub
viktorsomogyi commented on code in PR #15697: URL: https://github.com/apache/kafka/pull/15697#discussion_r1570907841 ## core/src/main/scala/kafka/server/KafkaConfig.scala: ## @@ -528,6 +529,10 @@ object KafkaConfig { "If log.message.timestamp.type=CreateTime, the message

[jira] [Comment Edited] (KAFKA-16578) Revert changes to connect_distributed_test.py for the new async Consumer

2024-04-18 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838609#comment-17838609 ] Sagar Rao edited comment on KAFKA-16578 at 4/18/24 3:02 PM: [~kirktrue], I

Re: [PR] KAFKA-15649: Handle directory failure timeout [kafka]

2024-04-18 Thread via GitHub
viktorsomogyi commented on PR #15697: URL: https://github.com/apache/kafka/pull/15697#issuecomment-2064216189 Rebased on latest trunk as there were some conflicts. Addressed some of the comments but there are 2 things I need to investigate: * `LogDirFailureTest` fails in `@AfterAll`

Re: [PR] KAFKA-16528: Client HB timing fix [kafka]

2024-04-18 Thread via GitHub
lianetm commented on PR #15698: URL: https://github.com/apache/kafka/pull/15698#issuecomment-2064258399 Hey @cadonna, could you take a look when you have chance? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1571018390 ## connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/SingleFieldPathTest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-15585: DescribeTopicPartitions client side change. [kafka]

2024-04-18 Thread via GitHub
mumrah merged PR #15470: URL: https://github.com/apache/kafka/pull/15470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-16263) Add Kafka Streams docs about available listeners/callback

2024-04-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838727#comment-17838727 ] Matthias J. Sax commented on KAFKA-16263: - Yes, these are the handlers this ticket refers do. >

[jira] [Commented] (KAFKA-16585) No way to forward message from punctuation method in the FixedKeyProcessor

2024-04-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838730#comment-17838730 ] Matthias J. Sax commented on KAFKA-16585: - Thanks for raising this ticket. Wondering how we

Re: [PR] KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets [kafka]

2024-04-18 Thread via GitHub
jolshan closed pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets URL: https://github.com/apache/kafka/pull/9590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570920951 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1178,6 +1178,33 @@ class LogManager(logDirs: Seq[File], } } + def

Re: [PR] KAFKA-16555: Consumer's RequestState has incorrect logic to determine if inflight [kafka]

2024-04-18 Thread via GitHub
philipnee commented on code in PR #15723: URL: https://github.com/apache/kafka/pull/15723#discussion_r1570986831 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/RequestStateTest.java: ## @@ -48,4 +50,40 @@ public void testRequestStateSimple() {

[jira] [Commented] (KAFKA-16567) Add New Stream Metrics based on KIP-869

2024-04-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838728#comment-17838728 ] Matthias J. Sax commented on KAFKA-16567: - Why is this ticket marked as "blocker" for 4.0

Re: [PR] KAFKA-16554: Online downgrade triggering and group type conversion [kafka]

2024-04-18 Thread via GitHub
dajac commented on code in PR #15721: URL: https://github.com/apache/kafka/pull/15721#discussion_r1571079283 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -777,11 +778,78 @@ public ClassicGroup classicGroup( }

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1571093805 ## server-common/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (KAFKA-16386) NETWORK_EXCEPTIONs from transaction verification are not translated

2024-04-18 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838768#comment-17838768 ] Justine Olshan commented on KAFKA-16386: Note: For 3.6, this is only returned in the produce

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570960525 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-04-18 Thread via GitHub
gaurav-narula commented on code in PR #15136: URL: https://github.com/apache/kafka/pull/15136#discussion_r1570959086 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1189,49 +1217,62 @@ class LogManager(logDirs: Seq[File], val sourceLog =

Re: [PR] KAFKA-16555: Consumer's RequestState has incorrect logic to determine if inflight [kafka]

2024-04-18 Thread via GitHub
philipnee commented on code in PR #15723: URL: https://github.com/apache/kafka/pull/15723#discussion_r1570988398 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestState.java: ## @@ -98,12 +93,11 @@ public boolean canSendRequest(final long

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570998105 ## connect/transforms/src/test/java/org/apache/kafka/connect/transforms/field/FieldPathNotationTest.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-16386) NETWORK_EXCEPTIONs from transaction verification are not translated

2024-04-18 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-16386: --- Affects Version/s: 3.7.0 > NETWORK_EXCEPTIONs from transaction verification are not

Re: [PR] KAFKA-16272: Adding new coordinator related changes for connect_distributed.py [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on PR #15594: URL: https://github.com/apache/kafka/pull/15594#issuecomment-2064120764 hey @lucasbru , I ran the following test suite ``` my_test_suite: -

Re: [PR] MINOR: Fix io-[wait-]ratio metrics description [kafka]

2024-04-18 Thread via GitHub
emitskevich-blp commented on code in PR #15722: URL: https://github.com/apache/kafka/pull/15722#discussion_r1571063662 ## clients/src/main/java/org/apache/kafka/common/network/Selector.java: ## @@ -1281,14 +1281,14 @@ private Meter createMeter(Metrics metrics, String

[jira] [Updated] (KAFKA-16566) Update consumer static membership fencing system test to support new protocol

2024-04-18 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16566: --- Labels: kip-848-client-support system-tests (was: kip-848-client-support) > Update

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
OmniaGM commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1571119979 ## server-common/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-16386: Convert NETWORK_EXCEPTIONs from KIP-890 transaction verification [kafka]

2024-04-18 Thread via GitHub
jolshan commented on PR #15559: URL: https://github.com/apache/kafka/pull/15559#issuecomment-2064671000 I wonder if I should backport this to 3.7 as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-15853: Move KafkaConfig log properties and docs out of core [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15569: URL: https://github.com/apache/kafka/pull/15569#discussion_r1571142879 ## server-common/src/main/java/org/apache/kafka/server/config/KafkaLogConfigs.java: ## @@ -0,0 +1,188 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (KAFKA-15089) Consolidate all the group coordinator configs

2024-04-18 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838754#comment-17838754 ] David Jacot commented on KAFKA-15089: - The goal was to also define the “AbstractConfig” part here

Re: [PR] MINOR: Add test for PartitionMetadataFile [kafka]

2024-04-18 Thread via GitHub
chia7712 commented on code in PR #15714: URL: https://github.com/apache/kafka/pull/15714#discussion_r1570903125 ## storage/src/test/java/org/apache/kafka/storage/internals/checkpoint/PartitionMetadataFileTest.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16272: Adding new coordinator related changes for connect_distributed.py [kafka]

2024-04-18 Thread via GitHub
vamossagar12 commented on code in PR #15594: URL: https://github.com/apache/kafka/pull/15594#discussion_r1570933991 ## tests/kafkatest/services/connect.py: ## @@ -534,33 +535,40 @@ def received_messages(self): def start(self): self.logger.info("Creating

Re: [PR] KAFKA-14226: feat(connect:transform): Introduce FieldPath abstraction [kafka]

2024-04-18 Thread via GitHub
C0urante commented on code in PR #15379: URL: https://github.com/apache/kafka/pull/15379#discussion_r1570939313 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16103: AsyncConsumer should await pending async commits on commitSync and close [kafka]

2024-04-18 Thread via GitHub
cadonna commented on code in PR #15613: URL: https://github.com/apache/kafka/pull/15613#discussion_r1570932239 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerCommitTest.scala: ## @@ -304,6 +304,64 @@ class PlaintextConsumerCommitTest extends AbstractConsumerTest

[PR] KAFKA-16579: Revert Consumer Rolling Upgrade [kafka]

2024-04-18 Thread via GitHub
philipnee opened a new pull request, #15753: URL: https://github.com/apache/kafka/pull/15753 Consumer Rolling Upgrade is meant to test the protocol upgrade for the old protocol. Therefore, I am removing old changes. ```

Re: [PR] KAFKA-16579: Revert Consumer Rolling Upgrade [kafka]

2024-04-18 Thread via GitHub
philipnee commented on PR #15753: URL: https://github.com/apache/kafka/pull/15753#issuecomment-2064308513 @lucasbru - This is just to remove the consumer protocol from testing as it is not suited for this test. Much appreciated if you get a chance to look at this. -- This is an

  1   2   >