[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-11-01 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] MINOR: Enable kraft test in kafka.api and kafka.network [kafka]

2023-11-01 Thread via GitHub
dengziming commented on PR #14595: URL: https://github.com/apache/kafka/pull/14595#issuecomment-1790107126 Hello @cmccabe @jsancio , no flaky is found in the latest 4 builds, please take a look。 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] MINOR: Enable kraft test in kafka.api and kafka.network [kafka]

2023-11-01 Thread via GitHub
dengziming commented on PR #14595: URL: https://github.com/apache/kafka/pull/14595#issuecomment-1790105902 I triggered the CI 5 times, only `ConsumerBounceTest` is flaky and all other tests succeed, I have reverted `ConsumerBounceTest` and convert it in the future.

[jira] [Commented] (KAFKA-15536) dynamically resize remoteIndexCache

2023-11-01 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781999#comment-17781999 ] Henry Cai commented on KAFKA-15536: --- Can this be back ported to 3.6 branch?   > dynamically resize

[jira] [Commented] (KAFKA-15632) Drop the invalid remote log metadata events

2023-11-01 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781998#comment-17781998 ] Henry Cai commented on KAFKA-15632: --- Can this fix be back ported to 3.6 branch?  [~showuon] [~ckamal] 

[jira] [Comment Edited] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-11-01 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781751#comment-17781751 ] Arpit Goyal edited comment on KAFKA-15388 at 11/2/23 5:04 AM: --

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1379595660 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/SinglePointMetric.java: ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1379594061 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/LastValueTracker.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1379594061 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/LastValueTracker.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1374059672 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/KafkaMetricsCollector.java: ## @@ -0,0 +1,347 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Resolved] (KAFKA-15669) Implement telemetry naming strategy

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15669. - Fix Version/s: 3.7.0 Resolution: Fixed > Implement telemetry naming strategy >

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on PR #14618: URL: https://github.com/apache/kafka/pull/14618#issuecomment-1790065635 Thanks for all the review help @xvrl! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Resolved] (KAFKA-15668) Add Opentelemetry Proto library with shadowed classes

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15668. - Fix Version/s: 3.7.0 Resolution: Fixed > Add Opentelemetry Proto library with

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax merged PR #14618: URL: https://github.com/apache/kafka/pull/14618 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14618: URL: https://github.com/apache/kafka/pull/14618#discussion_r1379585726 ## build.gradle: ## @@ -295,7 +296,12 @@ subprojects { } publications { mavenJava(MavenPublication) { - from components.java +

Re: [PR] KAFKA-15614: [Minor] Renaming getter and correcting code comments [kafka]

2023-11-01 Thread via GitHub
mjsax commented on PR #14684: URL: https://github.com/apache/kafka/pull/14684#issuecomment-1790059069 Thanks for the follow up. Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15614: [Minor] Renaming getter and correcting code comments [kafka]

2023-11-01 Thread via GitHub
mjsax merged PR #14684: URL: https://github.com/apache/kafka/pull/14684 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-14957: Update-Description-String [kafka]

2023-11-01 Thread via GitHub
mjsax commented on PR #13909: URL: https://github.com/apache/kafka/pull/13909#issuecomment-1790049927 Seems I missed you last comment -- sorry about this. The bot comment just surfaced to me. > May I ask how do we want to change the default value? I mean, if we are to provide

[PR] KIP-909 [kafka]

2023-11-01 Thread via GitHub
philipnee opened a new pull request, #14691: URL: https://github.com/apache/kafka/pull/14691 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[jira] [Assigned] (KAFKA-15765) Remove task level metric "commit-latency"

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-15765: --- Assignee: Bruno Cadonna > Remove task level metric "commit-latency" >

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379562125 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379562125 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379562125 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379530923 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult query(

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379554731 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379550244 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379550064 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
ijuma commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379548300 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-11-01 Thread via GitHub
philipnee commented on PR #14639: URL: https://github.com/apache/kafka/pull/14639#issuecomment-1789989071 Test failures seem irrelevant, but retriggering the test as there's a failed build -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1378442872 ## streams/src/main/java/org/apache/kafka/streams/query/TimestampedRangeQuery.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1367565896 ## streams/src/main/java/org/apache/kafka/streams/query/TimestampedRangeQuery.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15110:Fix the fail start due to the existence of multiple version packages under libs [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13893: KAFKA-15110:Fix the fail start due to the existence of multiple version packages under libs URL: https://github.com/apache/kafka/pull/13893 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] KAFKA-15129;[9/N] Remove metrics in network package when broker shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #14043: KAFKA-15129;[9/N] Remove metrics in network package when broker shutdown URL: https://github.com/apache/kafka/pull/14043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15129;[6/N] Remove metrics in ControllerStats when broker shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13960: KAFKA-15129;[6/N] Remove metrics in ControllerStats when broker shutdown URL: https://github.com/apache/kafka/pull/13960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15129;[8/N] Remove metrics in KafkaRequestHandlerPool when broker shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #14015: KAFKA-15129;[8/N] Remove metrics in KafkaRequestHandlerPool when broker shutdown URL: https://github.com/apache/kafka/pull/14015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15129;[10/N] Remove metrics in log when broker shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #14058: KAFKA-15129;[10/N] Remove metrics in log when broker shutdown URL: https://github.com/apache/kafka/pull/14058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15129;[5/N] Remove metrics in ControllerChannelManager when broker shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13959: KAFKA-15129;[5/N] Remove metrics in ControllerChannelManager when broker shutdown URL: https://github.com/apache/kafka/pull/13959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15129;[4/N] Remove metrics in Partition when broker shutdown and stopPartitions [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13952: KAFKA-15129;[4/N] Remove metrics in Partition when broker shutdown and stopPartitions URL: https://github.com/apache/kafka/pull/13952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15129;[3/N] Remove metrics in AbstractFetcherManager when fetcher manager instance shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13929: KAFKA-15129;[3/N] Remove metrics in AbstractFetcherManager when fetcher manager instance shutdown URL: https://github.com/apache/kafka/pull/13929 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] KAFKA-15129;[2/N] Remove metrics in GroupMetadataManager when shutdown [kafka]

2023-11-01 Thread via GitHub
hudeqi closed pull request #13926: KAFKA-15129;[2/N] Remove metrics in GroupMetadataManager when shutdown URL: https://github.com/apache/kafka/pull/13926 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379524385 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult query(

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379523818 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult query(

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1379523619 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -127,14 +131,31 @@ public QueryResult query(

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1379511059 ## streams/src/main/java/org/apache/kafka/streams/state/VersionedRecord.java: ## @@ -27,6 +27,7 @@ public final class VersionedRecord { private final V value;

Re: [PR] KAFKA-15432:RLM Stop partitions should not be invoked for non-tiered storage topics [kafka]

2023-11-01 Thread via GitHub
hudeqi commented on PR #14667: URL: https://github.com/apache/kafka/pull/14667#issuecomment-1789956368 > > I am restarting the CI. There are large number of test failures including some TS ones. Let's wait to have a saner CI build. > > seems new CI failed tests are unrelated.

Re: [PR] KAFKA-15602: revert KAFKA-4852 [kafka]

2023-11-01 Thread via GitHub
mjsax commented on PR #14617: URL: https://github.com/apache/kafka/pull/14617#issuecomment-1789953849 Merged to `trunk` and cherry-picked to `3.6`, `3.5`, and `3.4` branches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14626: URL: https://github.com/apache/kafka/pull/14626#discussion_r1379469860 ## streams/src/main/java/org/apache/kafka/streams/query/MultiVersionedKeyQuery.java: ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
ableegoldman commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379493839 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG,

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
ijuma commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379486496 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
ijuma commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379485880 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14596: URL: https://github.com/apache/kafka/pull/14596#discussion_r1379482103 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredVersionedKeyValueStore.java: ## @@ -157,6 +213,30 @@ protected QueryResult runKeyQuery(final

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14596: URL: https://github.com/apache/kafka/pull/14596#discussion_r1379471204 ## streams/src/main/java/org/apache/kafka/streams/query/VersionedKeyQuery.java: ## @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Resolved] (KAFKA-15730) KRaft support in ProducerFailureHandlingTest

2023-11-01 Thread Sameer Tejani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Tejani resolved KAFKA-15730. --- Fix Version/s: 3.7.0 Resolution: Fixed > KRaft support in

[jira] [Resolved] (KAFKA-15731) KRaft support in ListOffsetsIntegrationTest

2023-11-01 Thread Sameer Tejani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Tejani resolved KAFKA-15731. --- Fix Version/s: 3.7.0 Resolution: Fixed > KRaft support in ListOffsetsIntegrationTest

[jira] [Resolved] (KAFKA-15732) KRaft support in LogAppendTimeTest

2023-11-01 Thread Sameer Tejani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Tejani resolved KAFKA-15732. --- Fix Version/s: 3.7.0 Resolution: Fixed > KRaft support in LogAppendTimeTest >

[jira] [Resolved] (KAFKA-15733) KRaft support in FetchRequestMaxBytesTest

2023-11-01 Thread Sameer Tejani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Tejani resolved KAFKA-15733. --- Fix Version/s: 3.7.0 Resolution: Fixed > KRaft support in FetchRequestMaxBytesTest >

[jira] [Commented] (KAFKA-15753) KRaft support in BrokerApiVersionsCommandTest

2023-11-01 Thread Sameer Tejani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781948#comment-17781948 ] Sameer Tejani commented on KAFKA-15753: --- [~linzihao1999] if these issues are simple enough (if you

[jira] [Commented] (KAFKA-14419) Failed SyncGroup leading to partitions lost due to processing during rebalances

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781946#comment-17781946 ] Matthias J. Sax commented on KAFKA-14419: - [~kirktrue] [~pnee] – will the new consumer threading

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379448271 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

Re: [PR] MINOR: Change the ProducerConfig constructor that allows disabling logging to "protected" [kafka]

2023-11-01 Thread via GitHub
mjsax commented on code in PR #14681: URL: https://github.com/apache/kafka/pull/14681#discussion_r1379448271 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -618,7 +618,7 @@ public ProducerConfig(Map props) { super(CONFIG, props);

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781935#comment-17781935 ] Matthias J. Sax commented on KAFKA-15602: - [~luke.kirby] – it seems unlikely that there will be

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-11-01 Thread via GitHub
apoorvmittal10 commented on PR #14618: URL: https://github.com/apache/kafka/pull/14618#issuecomment-1789838514 @wcarlson5 @mjsax 16 tests unrelated to the PR are failing. As we have +1 from @xvrl hence can we please merge this PR now. Thanks for the help. -- This is an automated message

Re: [PR] KAFKA-15468: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2023-11-01 Thread via GitHub
jolshan commented on PR #14489: URL: https://github.com/apache/kafka/pull/14489#issuecomment-1789834583 Newest test failures look unrelated: [Build / JDK 17 and Scala 2.13 /

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-11-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1379428164 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-01 Thread via GitHub
kirktrue commented on PR #14690: URL: https://github.com/apache/kafka/pull/14690#issuecomment-1789830091 BTW, although it's hard for a someone of really little brain like myself to follow at times, the chained `Future` mechanism is very interesting and useful here. My one concern is that

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-11-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1379426628 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -2619,6 +2619,9 @@ private ConfigEntry.ConfigSource

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-01 Thread via GitHub
kirktrue commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1379392210 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -729,6 +729,13 @@ protected MetadataRequest.Builder newMetadataRequestBuilderForNewTopics() {

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on PR #14387: URL: https://github.com/apache/kafka/pull/14387#issuecomment-1789827363 I took a first pass at the files today, and will take another look more closely at the metrics themselves tomorrow or so. -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379424990 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: Are we planning not to add any of the new sensor metrics

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-11-01 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1379425234 ## clients/src/main/java/org/apache/kafka/clients/admin/ConfigEntry.java: ## @@ -222,6 +222,7 @@ public enum ConfigSource {

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379424990 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: Are we planning not to add any of the new sensor metrics

Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]

2023-11-01 Thread via GitHub
lzyLuke commented on PR #14682: URL: https://github.com/apache/kafka/pull/14682#issuecomment-1789818408 > Thanks for the PR, @lzyLuke . > > Please remove these two lines from "The following features are not fully implemented in KRaft mode": > > > ``` > > Modifying certain

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jeffkbkim commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379419320 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: yeah, unfortunately we need this for the existing

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379418802 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: Or do we have this just to cover the existing generic

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1379416743 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This

Re: [PR] KAFKA-15763; Group Coordinator should not deliver new assignment before previous one is acknowledged [kafka]

2023-11-01 Thread via GitHub
jeffkbkim commented on code in PR #14673: URL: https://github.com/apache/kafka/pull/14673#discussion_r1379393118 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupMember.java: ## @@ -537,22 +533,34 @@ public Map>

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-11-01 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781922#comment-17781922 ] Philip Nee commented on KAFKA-15602: Hi [~luke.kirby] - You don't need to become a committer to

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379411500 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/CoordinatorMetricsShard.java: ## @@ -0,0 +1,82 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-11-01 Thread via GitHub
hachikuji commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1379404877 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This

Re: [PR] KAFKA-15665: Enforce min ISR when complete partition reassignment. [kafka]

2023-11-01 Thread via GitHub
CalvinConfluent commented on PR #14604: URL: https://github.com/apache/kafka/pull/14604#issuecomment-1789790905 Failed tests are irrelevant. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379402554 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -352,7 +374,14 @@ public CoordinatorResult commitOffset(

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379398861 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -503,6 +547,7 @@ public void onLoaded(MetadataImage newImage) {

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379398861 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -503,6 +547,7 @@ public void onLoaded(MetadataImage newImage) {

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379393439 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -140,7 +159,12 @@ public GroupCoordinatorShard build() {

Re: [PR] DO NOT MERGE: introduce internal StoreFactory [kafka]

2023-11-01 Thread via GitHub
ableegoldman commented on code in PR #14659: URL: https://github.com/apache/kafka/pull/14659#discussion_r1379390131 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java: ## @@ -611,33 +545,38 @@ public final void addProcessor(final

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379389203 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -121,6 +126,20 @@ public CoordinatorShardBuilder withTimer(

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-01 Thread via GitHub
jolshan commented on code in PR #14387: URL: https://github.com/apache/kafka/pull/14387#discussion_r1379388116 ## checkstyle/import-control.xml: ## @@ -240,6 +240,7 @@ + Review Comment: I recall we removed yammer metrics from the last pr, do

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-11-01 Thread Luke Kirby (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781911#comment-17781911 ] Luke Kirby commented on KAFKA-15602: Thanks so much team! Maybe not the right place to ask this, but

[jira] [Updated] (KAFKA-15770) org.apache.kafka.streams.integration.ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy is flaky

2023-11-01 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15770: Component/s: unit tests >

[PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-01 Thread via GitHub
lianetm opened a new pull request, #14690: URL: https://github.com/apache/kafka/pull/14690 Updates for the Client state machine, Heartbeat and Commit managers, to support required states and transitions as the member interacts with a consumer group, allowing to commit offsets, and get

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-11-01 Thread via GitHub
wcarlson5 merged PR #14685: URL: https://github.com/apache/kafka/pull/14685 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-11-01 Thread via GitHub
wcarlson5 commented on PR #14685: URL: https://github.com/apache/kafka/pull/14685#issuecomment-1789576997 I'm satisfied with comments and discussion on old PR https://github.com/apache/kafka/pull/14564 so I'm going to go a head and merge this -- This is an automated message from the

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-11-01 Thread via GitHub
wcarlson5 commented on code in PR #14685: URL: https://github.com/apache/kafka/pull/14685#discussion_r1379261920 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -350,6 +353,92 @@ public synchronized void update(int requestVersion, MetadataResponse

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-11-01 Thread via GitHub
philipnee commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1379231747 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,24 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]

2023-11-01 Thread via GitHub
cmccabe commented on PR #14682: URL: https://github.com/apache/kafka/pull/14682#issuecomment-1789520190 Thanks for the PR, @lzyLuke . Please remove these two lines from "The following features are not fully implemented in KRaft mode": > Modifying certain dynamic configurations

Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]

2023-11-01 Thread via GitHub
cmccabe commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379209724 ## docs/ops.html: ## @@ -3613,23 +3613,22 @@ https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - -

Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]

2023-11-01 Thread via GitHub
cmccabe commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379209314 ## docs/ops.html: ## @@ -3613,23 +3613,22 @@ https://issues.apache.org/jira/projects/KAFKA; target="_blank">project JIRA and the "kraft" component. - -

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-11-01 Thread via GitHub
AndrewJSchofield commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1379170868 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,24 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15311: Fix docs about reverting to ZooKeeper mode during KRaft migration [kafka]

2023-11-01 Thread via GitHub
lzyLuke commented on code in PR #14682: URL: https://github.com/apache/kafka/pull/14682#discussion_r1379152694 ## docs/ops.html: ## @@ -3787,7 +3786,7 @@ Migrating brokers to KRaft Reverting to ZooKeeper mode During the Migration While the cluster is still in migration

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-11-01 Thread via GitHub
junrao commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1379117057 ## clients/src/main/java/org/apache/kafka/clients/admin/ConfigEntry.java: ## @@ -222,6 +222,7 @@ public enum ConfigSource { DYNAMIC_BROKER_LOGGER_CONFIG, //

  1   2   3   >