[jira] [Updated] (KAFKA-15701) Allow use of user policy in CreateTopicPolicy

2023-10-26 Thread Jiao Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiao Zhang updated KAFKA-15701: --- Description: One use case of CreateTopicPolicy we have experienced is allow/reject topic creation

[jira] [Updated] (KAFKA-15701) Allow use of user policy in CreateTopicPolicy

2023-10-26 Thread Jiao Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiao Zhang updated KAFKA-15701: --- Description: One use case of CreateTopicPolicy we have experienced is allow/reject topic creation

[jira] [Updated] (KAFKA-15701) Allow use of user policy in CreateTopicPolicy

2023-10-26 Thread Jiao Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiao Zhang updated KAFKA-15701: --- Description: One use case of CreateTopicPolicy we have experienced is allow/reject topic creation

[jira] [Updated] (KAFKA-15701) Allow use of user policy in CreateTopicPolicy

2023-10-26 Thread Jiao Zhang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiao Zhang updated KAFKA-15701: --- Description: One use case of CreateTopicPolicy we have experienced is allow/reject topic creation

[jira] [Created] (KAFKA-15701) Allow use of user policy in CreateTopicPolicy

2023-10-26 Thread Jiao Zhang (Jira)
Jiao Zhang created KAFKA-15701: -- Summary: Allow use of user policy in CreateTopicPolicy Key: KAFKA-15701 URL: https://issues.apache.org/jira/browse/KAFKA-15701 Project: Kafka Issue Type:

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1374065471 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/LastValueTracker.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

2023-10-26 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780166#comment-17780166 ] Kamal Chandraprakash commented on KAFKA-15695: -- [~nikramakrishnan] I remember fixing this

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-26 Thread via GitHub
philipnee commented on PR #14639: URL: https://github.com/apache/kafka/pull/14639#issuecomment-1782275615 Hi @lucasbru - Thanks for taking the time to review my PR. I addressed all but 2 comments: 1. Mockito: Could you be more specific on how you expect to mock the response object?

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14639: URL: https://github.com/apache/kafka/pull/14639#discussion_r1374057359 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -127,7 +146,7 @@ public void maybeAutoCommit(final Map offsets

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1374013380 ## streams/src/main/java/org/apache/kafka/streams/state/internals/WrappedStateStore.java: ## @@ -34,6 +34,9 @@ public abstract class WrappedStateStore implements

Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14619: URL: https://github.com/apache/kafka/pull/14619#discussion_r1374019186 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Resolved] (KAFKA-15390) FetchResponse.preferredReplica may contains fenced replica in KRaft mode

2023-10-26 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming resolved KAFKA-15390. - Fix Version/s: 3.6.0 Resolution: Fixed > FetchResponse.preferredReplica may contains

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14639: URL: https://github.com/apache/kafka/pull/14639#discussion_r1374040965 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java: ## @@ -382,9 +522,49 @@ private ClientResponse

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1374037210 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricKey.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-26 Thread via GitHub
philipnee commented on PR #14642: URL: https://github.com/apache/kafka/pull/14642#issuecomment-1782236753 Hi @kirktrue Thanks for taking time reviewing my code. I made changes according to your suggestions. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374036176 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,24 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374035895 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -610,6 +628,18 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-26 Thread via GitHub
philipnee commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374035641 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -610,6 +628,18 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15445: [WIP] Add JVM Docker image [kafka]

2023-10-26 Thread via GitHub
sanjay-awatramani commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1374022606 ## docker/docker_release.py: ## @@ -0,0 +1,80 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-26 Thread via GitHub
ocadaruma commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1374014105 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -462,22 +462,32 @@ public Optional lastEntry(long producerId) {

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14618: URL: https://github.com/apache/kafka/pull/14618#discussion_r1374016565 ## build.gradle: ## @@ -295,7 +296,12 @@ subprojects { } publications { mavenJava(MavenPublication) { - from components.java +

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1374007941 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricKey.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-26 Thread via GitHub
ocadaruma commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1374014105 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -462,22 +462,32 @@ public Optional lastEntry(long producerId) {

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1374013380 ## streams/src/main/java/org/apache/kafka/streams/state/internals/WrappedStateStore.java: ## @@ -34,6 +34,9 @@ public abstract class WrappedStateStore implements

Re: [PR] KAFKA-15201: Allow git push to fail gracefully [kafka]

2023-10-26 Thread via GitHub
Owen-CH-Leung commented on code in PR #14645: URL: https://github.com/apache/kafka/pull/14645#discussion_r1374005528 ## release.py: ## @@ -730,7 +730,7 @@ def select_gpg_key(): fail("Ok, giving up") if not user_ok("Ok to push RC tag %s (y/n)?: " % rc_tag): fail("Ok,

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-26 Thread via GitHub
yyu1993 commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1373996821 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1216,15 +1228,21 @@ class LogManager(logDirs: Seq[File],

Re: [PR] KAFKA-15689: Logging skipped event when expected migration state is wrong [kafka]

2023-10-26 Thread via GitHub
showuon commented on PR #14646: URL: https://github.com/apache/kafka/pull/14646#issuecomment-1782174276 @ppatierno , please resolve the conflicts. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (KAFKA-15319) Upgrade rocksdb to fix CVE-2022-37434

2023-10-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15319: Fix Version/s: 3.5.2 > Upgrade rocksdb to fix CVE-2022-37434 >

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2023-10-26 Thread via GitHub
mjsax commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-1782162923 Cherry-picked this to `3.5` branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14596: URL: https://github.com/apache/kafka/pull/14596#discussion_r1373965049 ## streams/src/main/java/org/apache/kafka/streams/query/VersionedKeyQuery.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1373981478 ## streams/src/main/java/org/apache/kafka/streams/state/internals/WrappedStateStore.java: ## @@ -34,6 +34,9 @@ public abstract class WrappedStateStore implements

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1373977415 ## streams/src/main/java/org/apache/kafka/streams/state/internals/StoreQueryUtils.java: ## @@ -346,6 +346,23 @@ public static Function getDeserializeValue(final

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1373970782 ## streams/src/main/java/org/apache/kafka/streams/state/internals/StoreQueryUtils.java: ## @@ -346,6 +346,23 @@ public static Function getDeserializeValue(final

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1373970782 ## streams/src/main/java/org/apache/kafka/streams/state/internals/StoreQueryUtils.java: ## @@ -346,6 +346,23 @@ public static Function getDeserializeValue(final

Re: [PR] KAFKA-15527: Update Javadoc for range and reverseRange in ReadOnlyKeyValueStore and update upgrade-guide.html for kafka streams webpage [kafka]

2023-10-26 Thread via GitHub
mjsax merged PR #14600: URL: https://github.com/apache/kafka/pull/14600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-26 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1373953314 ## streams/src/main/java/org/apache/kafka/streams/state/internals/KeyValueToTimestampedKeyValueByteStoreAdapter.java: ## @@ -126,15 +128,22 @@ public QueryResult query(

[jira] [Updated] (KAFKA-15690) EosIntegrationTest is flaky.

2023-10-26 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15690: Component/s: streams unit tests > EosIntegrationTest is flaky. >

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-26 Thread via GitHub
kirktrue commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1373924481 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,24 @@ public class ConsumerConfig extends AbstractConfig {

[PR] [KIP-954] support custom DSL store providers [kafka]

2023-10-26 Thread via GitHub
agavra opened a new pull request, #14648: URL: https://github.com/apache/kafka/pull/14648 ### Overview Implementation for [KIP-954](https://cwiki.apache.org/confluence/display/KAFKA/KIP-954%3A+expand+default+DSL+store+configuration+to+custom+types) ### Testing Strategy *

Re: [PR] Kafka 15680 [kafka]

2023-10-26 Thread via GitHub
C0urante commented on PR #14630: URL: https://github.com/apache/kafka/pull/14630#issuecomment-1782075383 I can take a look next week. In the meantime, do you think you could add some unit tests for this change? -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-26 Thread via GitHub
jolshan commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1373924152 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -360,7 +360,9 @@ public short partitionRecordVersion() { } public

[jira] [Updated] (KAFKA-14419) Failed SyncGroup leading to partitions lost due to processing during rebalances

2023-10-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-14419: --- Summary: Failed SyncGroup leading to partitions lost due to processing

[jira] [Commented] (KAFKA-14419) Same message consumed again by the same stream task after partition is lost and reassigned

2023-10-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780119#comment-17780119 ] A. Sophie Blee-Goldman commented on KAFKA-14419: [~Carlstedt] first off, do you by any

[jira] [Comment Edited] (KAFKA-14419) Same message consumed again by the same stream task after partition is lost and reassigned

2023-10-26 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779731#comment-17779731 ] A. Sophie Blee-Goldman edited comment on KAFKA-14419 at 10/26/23 10:49 PM:

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-26 Thread via GitHub
ocadaruma commented on code in PR #14242: URL: https://github.com/apache/kafka/pull/14242#discussion_r1373898465 ## storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java: ## @@ -308,7 +308,14 @@ public void truncateFromEnd(long endOffset) {

[jira] [Updated] (KAFKA-15690) EosIntegrationTest is flaky.

2023-10-26 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu updated KAFKA-15690: --- Description: EosIntegrationTest

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-26 Thread via GitHub
CalvinConfluent commented on PR #14603: URL: https://github.com/apache/kafka/pull/14603#issuecomment-1781972169 https://issues.apache.org/jira/browse/KAFKA-15699 https://issues.apache.org/jira/browse/KAFKA-15700 -- This is an automated message from the Apache Git Service. To respond to

[jira] [Created] (KAFKA-15700) FetchFromFollowerIntegrationTest is flaky

2023-10-26 Thread Calvin Liu (Jira)
Calvin Liu created KAFKA-15700: -- Summary: FetchFromFollowerIntegrationTest is flaky Key: KAFKA-15700 URL: https://issues.apache.org/jira/browse/KAFKA-15700 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-15699) MirrorConnectorsIntegrationBaseTest is flaky

2023-10-26 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu updated KAFKA-15699: --- Description: It may relate to inappropriate test timeout testReplicateSourceDefault() {code:java}

[jira] [Updated] (KAFKA-15699) MirrorConnectorsIntegrationBaseTest is flaky

2023-10-26 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu updated KAFKA-15699: --- Description: May relate to inaproparate  testReplicateSourceDefault() *

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-26 Thread via GitHub
mumrah merged PR #14545: URL: https://github.com/apache/kafka/pull/14545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Updated] (KAFKA-15690) EosIntegrationTest is flaky.

2023-10-26 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Calvin Liu updated KAFKA-15690: --- Summary: EosIntegrationTest is flaky. (was: Flaky integration tests) > EosIntegrationTest is

[jira] [Created] (KAFKA-15699) MirrorConnectorsIntegrationBaseTest is flaky

2023-10-26 Thread Calvin Liu (Jira)
Calvin Liu created KAFKA-15699: -- Summary: MirrorConnectorsIntegrationBaseTest is flaky Key: KAFKA-15699 URL: https://issues.apache.org/jira/browse/KAFKA-15699 Project: Kafka Issue Type: Bug

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-26 Thread via GitHub
mumrah commented on PR #14545: URL: https://github.com/apache/kafka/pull/14545#issuecomment-1781961533 Test failures look unrelated. https://github.com/apache/kafka/assets/55116/b0eb9cec-76df-4d5e-ab73-dfd2ca4d5978;> -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-15605: Fix topic deletion handling during ZK migration [kafka]

2023-10-26 Thread via GitHub
mumrah commented on code in PR #14545: URL: https://github.com/apache/kafka/pull/14545#discussion_r1373863647 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1216,15 +1228,21 @@ class LogManager(logDirs: Seq[File],

[jira] [Created] (KAFKA-15698) KRaft mode brokers should clean up stray partitions from migration

2023-10-26 Thread David Arthur (Jira)
David Arthur created KAFKA-15698: Summary: KRaft mode brokers should clean up stray partitions from migration Key: KAFKA-15698 URL: https://issues.apache.org/jira/browse/KAFKA-15698 Project: Kafka

Re: [PR] KAFKA-15659: Fix shouldInvokeUserDefinedGlobalStateRestoreListener flaky test [kafka]

2023-10-26 Thread via GitHub
ableegoldman merged PR #14608: URL: https://github.com/apache/kafka/pull/14608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15659: Fix shouldInvokeUserDefinedGlobalStateRestoreListener flaky test [kafka]

2023-10-26 Thread via GitHub
ableegoldman commented on code in PR #14608: URL: https://github.com/apache/kafka/pull/14608#discussion_r1373854238 ## streams/src/test/java/org/apache/kafka/streams/integration/RestoreIntegrationTest.java: ## @@ -523,13 +527,13 @@ public void

Re: [PR] KAFKA-15659: Fix shouldInvokeUserDefinedGlobalStateRestoreListener flaky test [kafka]

2023-10-26 Thread via GitHub
ableegoldman commented on PR #14608: URL: https://github.com/apache/kafka/pull/14608#issuecomment-1781945353 Test failures are unrelated, and the relevant test passed in all cases, so let's merge this -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15659: Fix shouldInvokeUserDefinedGlobalStateRestoreListener flaky test [kafka]

2023-10-26 Thread via GitHub
ableegoldman commented on code in PR #14608: URL: https://github.com/apache/kafka/pull/14608#discussion_r1373854238 ## streams/src/test/java/org/apache/kafka/streams/integration/RestoreIntegrationTest.java: ## @@ -523,13 +527,13 @@ public void

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1373846331 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -4385,15 +4451,39 @@ class KafkaApisTest { .setGroupId("group-3")

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373822529 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -407,36 +414,41 @@ object StorageTool extends Logging { if (directories.isEmpty) { throw new

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1373847099 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -4385,15 +4451,39 @@ class KafkaApisTest { .setGroupId("group-3")

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1373846331 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -4385,15 +4451,39 @@ class KafkaApisTest { .setGroupId("group-3")

[jira] [Commented] (KAFKA-15316) CommitRequestManager not calling RequestState callbacks

2023-10-26 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17780100#comment-17780100 ] Philip Nee commented on KAFKA-15316: I think this is fixed. I will check > CommitRequestManager not

[jira] [Assigned] (KAFKA-15316) CommitRequestManager not calling RequestState callbacks

2023-10-26 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee reassigned KAFKA-15316: -- Assignee: Philip Nee > CommitRequestManager not calling RequestState callbacks >

[jira] [Updated] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15556: -- Labels: consumer-threading-refactor kip-848 (was: consumer-threading-refactor kip-848-preview) >

[jira] [Updated] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15556: -- Labels: consumer-threading-refactor (was: consumer-threading-refactor kip-848) > Remove

Re: [PR] KAFKA-15598 & KAFKA-15461: Add integration tests for DescribeGroups API, DeleteGroups API, OffsetDelete API and ListGroups API [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1373833449 ## core/src/test/scala/unit/kafka/server/OffsetDeleteRequestTest.scala: ## @@ -0,0 +1,165 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15583: Enforce HWM advance only if partition is not under-min-ISR [kafka]

2023-10-26 Thread via GitHub
CalvinConfluent commented on code in PR #14594: URL: https://github.com/apache/kafka/pull/14594#discussion_r1373832091 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1106,6 +1115,12 @@ class Partition(val topicPartition: TopicPartition, * @return true if the

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-26 Thread via GitHub
C0urante commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1373831428 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -528,72 +483,64 @@ public void

Re: [PR] KAFKA-15598 & KAFKA-15461: Add integration tests for DescribeGroups API, DeleteGroups API, OffsetDelete API and ListGroups API [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1373831315 ## core/src/test/scala/unit/kafka/server/OffsetDeleteRequestTest.scala: ## @@ -0,0 +1,165 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-26 Thread via GitHub
C0urante commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1373830704 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -528,72 +483,64 @@ public void

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373824387 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -267,35 +267,27 @@ class LogManager(logDirs: Seq[File], /** * Retrieves the Uuid for the directory,

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373824387 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -267,35 +267,27 @@ class LogManager(logDirs: Seq[File], /** * Retrieves the Uuid for the directory,

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373822529 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -407,36 +414,41 @@ object StorageTool extends Logging { if (directories.isEmpty) { throw new

[jira] [Updated] (KAFKA-15691) Upgrade existing and add new system tests to use new consumer

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15691: -- Summary: Upgrade existing and add new system tests to use new consumer (was: Upgrade existing and

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373820141 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -228,23 +226,26 @@ class KafkaServer( info(s"Cluster ID = $clusterId") /* load

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373816778 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogDirFailureChannel.java: ## @@ -57,7 +56,7 @@ public boolean hasOfflineLogDir(String logDir) {

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373817350 ## core/src/main/scala/kafka/server/KafkaRaftServer.scala: ## @@ -135,39 +136,52 @@ object KafkaRaftServer { * @return A tuple containing the loaded meta

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373817350 ## core/src/main/scala/kafka/server/KafkaRaftServer.scala: ## @@ -135,39 +136,52 @@ object KafkaRaftServer { * @return A tuple containing the loaded meta

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-26 Thread via GitHub
chb2ab commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1373815620 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -360,7 +360,9 @@ public short partitionRecordVersion() { } public

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373815484 ## metadata/src/main/java/org/apache/kafka/metadata/properties/MetaPropertiesVersion.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-26 Thread via GitHub
chb2ab commented on code in PR #14627: URL: https://github.com/apache/kafka/pull/14627#discussion_r1373814111 ## clients/src/main/resources/common/message/FetchRequest.json: ## @@ -53,7 +53,9 @@ // // Version 15 adds the ReplicaState which includes new field ReplicaEpoch

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373813203 ## core/src/test/java/kafka/testkit/KafkaClusterTestKit.java: ## @@ -227,8 +231,12 @@ public KafkaClusterTestKit build() throws Exception {

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373812595 ## metadata/src/main/java/org/apache/kafka/metadata/properties/MetaPropertiesEnsemble.java: ## @@ -0,0 +1,497 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373810872 ## metadata/src/main/java/org/apache/kafka/metadata/properties/PropertiesUtils.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-26 Thread via GitHub
cmccabe commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1373808533 ## metadata/src/main/java/org/apache/kafka/metadata/properties/MetaProperties.java: ## @@ -0,0 +1,266 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-26 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1373793336 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -864,6 +778,111 @@ class ReplicaManager(val config: KafkaConfig, } } + /* + * Note: This

Re: [PR] KAFKA-15598 & KAFKA-15461: Add integration tests for DescribeGroups API, DeleteGroups API, OffsetDelete API and ListGroups API [kafka]

2023-10-26 Thread via GitHub
dongnuo123 commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1373787330 ## core/src/test/scala/unit/kafka/server/ListGroupsRequestTest.scala: ## @@ -0,0 +1,236 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15583: Enforce HWM advance only if partition is not under-min-ISR With ELR [kafka]

2023-10-26 Thread via GitHub
artemlivshits commented on code in PR #14594: URL: https://github.com/apache/kafka/pull/14594#discussion_r1373758559 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1106,6 +1115,12 @@ class Partition(val topicPartition: TopicPartition, * @return true if the HW

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-26 Thread via GitHub
wcarlson5 merged PR #14575: URL: https://github.com/apache/kafka/pull/14575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-26 Thread via GitHub
wcarlson5 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1373750580 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/ClientTelemetrySender.java: ## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-14484: [1/N] Move PartitionMetadataFile to storage module [kafka]

2023-10-26 Thread via GitHub
alok123t commented on PR #14607: URL: https://github.com/apache/kafka/pull/14607#issuecomment-1781801313 @junrao I think the test failures should be unrelated to the PR from a quick look - not sure if these are known flaky tests, I will wait for another run from the latest commit --

Re: [PR] KAFKA-14484: [1/N] Move PartitionMetadataFile to storage module [kafka]

2023-10-26 Thread via GitHub
alok123t commented on code in PR #14607: URL: https://github.com/apache/kafka/pull/14607#discussion_r1373738648 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/PartitionMetadata.java: ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-26 Thread via GitHub
apoorvmittal10 commented on PR #14621: URL: https://github.com/apache/kafka/pull/14621#issuecomment-1781794070 > @apoorvmittal10 : Yes, we are gradually moving the codebase to java. Ideally, all new classes should be written in java. Thanks @junrao, I ll update the PR by tomorrow

[jira] [Updated] (KAFKA-15574) Update states and transitions for membership manager state machine

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15574: -- Description: This task is to update the state machine so that it correctly acts as the glue between

[jira] [Updated] (KAFKA-15574) Update states and transitions for membership manager state machine

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15574: -- Summary: Update states and transitions for membership manager state machine (was: Implement

[jira] [Updated] (KAFKA-15574) Implement heartbeat membership manager state machine

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15574: -- Description: This task includes: # Updating the states in the code as [defined in the diagrams on

[jira] [Updated] (KAFKA-15574) Implement heartbeat membership manager state machine

2023-10-26 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15574: -- Summary: Implement heartbeat membership manager state machine (was: Integrate partition assignment

  1   2   3   >