[jira] [Commented] (KAFKA-10337) Wait for pending async commits in commitSync() even if no offsets are specified

2023-07-19 Thread Tom Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744877#comment-17744877 ] Tom Lee commented on KAFKA-10337: - Thanks for pushing this through [~erikvanoosten], nice to see this

[GitHub] [kafka] ivanyu commented on pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-07-19 Thread via GitHub
ivanyu commented on PR #13984: URL: https://github.com/apache/kafka/pull/13984#issuecomment-1643169441 Conflicts resolved -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] jeffkbkim opened a new pull request, #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-19 Thread via GitHub
jeffkbkim opened a new pull request, #14056: URL: https://github.com/apache/kafka/pull/14056 Built on top of https://github.com/apache/kafka/pull/14017, this PR implements the existing Heartbeat API in the new Group Coordinator. ### Committer Checklist (excluded from commit message)

[GitHub] [kafka] satishd commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
satishd commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1268911909 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: [KAFKA-15221] Fix the race between fetch requests from a rebooted follower.

2023-07-19 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1268889660 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -858,13 +858,28 @@ class Partition(val topicPartition: TopicPartition, // No need to calculate low

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: [KAFKA-15221] Fix the race between fetch requests from a rebooted follower.

2023-07-19 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1268889660 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -858,13 +858,28 @@ class Partition(val topicPartition: TopicPartition, // No need to calculate low

[GitHub] [kafka] showuon commented on a diff in pull request #13999: KAFKA-15176: add tests for tiered storage metrics

2023-07-19 Thread via GitHub
showuon commented on code in PR #13999: URL: https://github.com/apache/kafka/pull/13999#discussion_r1268886912 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3379,6 +3396,246 @@ class ReplicaManagerTest {

[GitHub] [kafka] showuon commented on a diff in pull request #14051: KAFKA-15218: Avoid NPE thrown while deleting topic and fetch from follower concurrently

2023-07-19 Thread via GitHub
showuon commented on code in PR #14051: URL: https://github.com/apache/kafka/pull/14051#discussion_r1268862349 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -998,7 +998,13 @@ class Partition(val topicPartition: TopicPartition, // 3. Its metadata cached

[GitHub] [kafka] showuon commented on a diff in pull request #14045: refactor(storage): topic-based RLMM consumer-manager/task related improvements

2023-07-19 Thread via GitHub
showuon commented on code in PR #14045: URL: https://github.com/apache/kafka/pull/14045#discussion_r1268858531 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerManager.java: ## @@ -87,36 +96,41 @@ public void

[GitHub] [kafka] bmscomp commented on pull request #14032: MINOR: Upgrade Gradle wrapper version to 8.2.1

2023-07-19 Thread via GitHub
bmscomp commented on PR #14032: URL: https://github.com/apache/kafka/pull/14032#issuecomment-1642924837 @divijvaidya In the `retry_zinc` file bash script in the kafka root directory there is something about this issue but related to an older version of gradle ``` # Hacky

[GitHub] [kafka] gharris1727 commented on pull request #14055: KAFKA-15031: Add plugin.discovery to Connect worker configuration (KIP-898)

2023-07-19 Thread via GitHub
gharris1727 commented on PR #14055: URL: https://github.com/apache/kafka/pull/14055#issuecomment-1642919694 Here's an example error log output when in HYBRID_FAIL mode (I shortened the listing for brevity; the actual prints include every non-migrated plugin) ``` Plugins are missing

[GitHub] [kafka] gharris1727 opened a new pull request, #14055: KAFKA-15031: Add configurable scanning modes to Connect worker config (KIP-898)

2023-07-19 Thread via GitHub
gharris1727 opened a new pull request, #14055: URL: https://github.com/apache/kafka/pull/14055 This is the primary feature for KIP-898, which allows users to reconfigure the Connect worker among the different scanning modes. The different modes and their functionality and intended use are

[GitHub] [kafka] CalvinConfluent commented on a diff in pull request #14053: [KAFKA-15221] Fix the race between fetch requests from a rebooted follower.

2023-07-19 Thread via GitHub
CalvinConfluent commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1268783217 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1366,6 +1376,17 @@ class Partition(val topicPartition: TopicPartition,

[GitHub] [kafka] philipnee opened a new pull request, #14054: KAFKA-14648: Moving bootstrap to NetworkClient Poll.

2023-07-19 Thread via GitHub
philipnee opened a new pull request, #14054: URL: https://github.com/apache/kafka/pull/14054 Motivation: Instantiating a new client may result in a fatal failure if the bootstrap server cannot be resolved due to misconfiguration or transient network issues such as slow DNS. This is

[GitHub] [kafka] gharris1727 merged pull request #13971: KAFKA-15150: Add ServiceLoaderScanner implementation

2023-07-19 Thread via GitHub
gharris1727 merged PR #13971: URL: https://github.com/apache/kafka/pull/13971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] gharris1727 commented on pull request #13971: KAFKA-15150: Add ServiceLoaderScanner implementation

2023-07-19 Thread via GitHub
gharris1727 commented on PR #13971: URL: https://github.com/apache/kafka/pull/13971#issuecomment-1642887109 Flaky test failures appear unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Created] (KAFKA-15221) Potential race condition between requests from rebooted followers

2023-07-19 Thread Calvin Liu (Jira)
Calvin Liu created KAFKA-15221: -- Summary: Potential race condition between requests from rebooted followers Key: KAFKA-15221 URL: https://issues.apache.org/jira/browse/KAFKA-15221 Project: Kafka

[GitHub] [kafka] kirktrue commented on pull request #13990: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-07-19 Thread via GitHub
kirktrue commented on PR #13990: URL: https://github.com/apache/kafka/pull/13990#issuecomment-1642861657 @junrao The tests ran again with (no code change in between) and now there are only four test failures in (seemingly) unrelated areas of the code. -- This is an automated message from

[GitHub] [kafka] ijuma commented on pull request #13949: KAFKA-15141: init logger statically on hot codepaths

2023-07-19 Thread via GitHub
ijuma commented on PR #13949: URL: https://github.com/apache/kafka/pull/13949#issuecomment-1642838445 I merged the PR. Thanks for the contribution! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] cmccabe commented on a diff in pull request #14010: KAFKA-15183: Add more controller, loader, snapshot emitter metrics

2023-07-19 Thread via GitHub
cmccabe commented on code in PR #14010: URL: https://github.com/apache/kafka/pull/14010#discussion_r1268717074 ## metadata/src/main/java/org/apache/kafka/image/publisher/metrics/SnapshotEmitterMetrics.java: ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14010: KAFKA-15183: Add more controller, loader, snapshot emitter metrics

2023-07-19 Thread via GitHub
cmccabe commented on code in PR #14010: URL: https://github.com/apache/kafka/pull/14010#discussion_r1268716163 ## metadata/src/main/java/org/apache/kafka/controller/metrics/QuorumControllerMetrics.java: ## @@ -54,6 +54,14 @@ public class QuorumControllerMetrics implements

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: Fence follower fetch with stale broker epoch

2023-07-19 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1268702825 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -1366,6 +1376,17 @@ class Partition(val topicPartition: TopicPartition, fetchParams.replicaId,

[GitHub] [kafka] CalvinConfluent opened a new pull request, #14053: Fence follower fetch with stale broker epoch

2023-07-19 Thread via GitHub
CalvinConfluent opened a new pull request, #14053: URL: https://github.com/apache/kafka/pull/14053 There are two changes here: 1. Acquire read lock when updating the follower fetch state. This can prevent a race case with fetch requests from a rebooted follower. 2. Fence the follower

[jira] [Updated] (KAFKA-10579) Flaky test connect.integration.InternalTopicsIntegrationTest.testStartWhenInternalTopicsCreatedManuallyWithCompactForBrokersDefaultCleanupPolicy

2023-07-19 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-10579: Fix Version/s: 3.6.0 > Flaky test >

[jira] [Resolved] (KAFKA-12842) Failing test: org.apache.kafka.connect.integration.ConnectWorkerIntegrationTest.testSourceTaskNotBlockedOnShutdownWithNonExistentTopic

2023-07-19 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-12842. - Fix Version/s: 3.6.0 Resolution: Fixed > Failing test: >

[jira] [Resolved] (KAFKA-8690) Flakey test ConnectWorkerIntegrationTest#testAddAndRemoveWorke

2023-07-19 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-8690. Fix Version/s: 3.6.0 Resolution: Fixed > Flakey test

[jira] [Resolved] (KAFKA-10579) Flaky test connect.integration.InternalTopicsIntegrationTest.testStartWhenInternalTopicsCreatedManuallyWithCompactForBrokersDefaultCleanupPolicy

2023-07-19 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-10579. - Resolution: Fixed > Flaky test >

[GitHub] [kafka] gharris1727 merged pull request #14029: KAFKA-10579: Upgrade reflections from 0.9.12 to 0.10.2

2023-07-19 Thread via GitHub
gharris1727 merged PR #14029: URL: https://github.com/apache/kafka/pull/14029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] gharris1727 commented on pull request #14029: KAFKA-10579: Upgrade reflections from 0.9.12 to 0.10.2

2023-07-19 Thread via GitHub
gharris1727 commented on PR #14029: URL: https://github.com/apache/kafka/pull/14029#issuecomment-1642744154 Test failures appear unrelated, and this has passed unit tests, system tests, and stress testing already. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14005: KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors

2023-07-19 Thread via GitHub
gharris1727 commented on code in PR #14005: URL: https://github.com/apache/kafka/pull/14005#discussion_r1268647699 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationExactlyOnceTest.java: ## @@ -46,4 +51,49 @@ public void

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14005: KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors

2023-07-19 Thread via GitHub
gharris1727 commented on code in PR #14005: URL: https://github.com/apache/kafka/pull/14005#discussion_r1268624984 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationBaseTest.java: ## @@ -933,12 +938,94 @@ protected static

[GitHub] [kafka] divijvaidya commented on pull request #14032: MINOR: Upgrade Gradle wrapper version to 8.2.1

2023-07-19 Thread via GitHub
divijvaidya commented on PR #14032: URL: https://github.com/apache/kafka/pull/14032#issuecomment-1642721299 We have had a large number of failures [1] since this was merged with `Timeout waiting to lock * cache (/home/jenkins/.gradle/caches/*/* It is currently in use by another Gradle

[GitHub] [kafka] jolshan commented on pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
jolshan commented on PR #14046: URL: https://github.com/apache/kafka/pull/14046#issuecomment-1642696317 Thanks for the fix @dajac let's just confirm the build and we should be good :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] philipnee opened a new pull request, #14052: KAFKA-14960: [Part I]TopicMetadataRequestManager Implementation (#7)

2023-07-19 Thread via GitHub
philipnee opened a new pull request, #14052: URL: https://github.com/apache/kafka/pull/14052 TopicMetadataRequestManager implementation - *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so

[GitHub] [kafka] gharris1727 commented on pull request #13977: KAFKA-15162: Reflectively find plugins in parent ClassLoaders that aren't on the classpath

2023-07-19 Thread via GitHub
gharris1727 commented on PR #13977: URL: https://github.com/apache/kafka/pull/13977#issuecomment-1642662589 Test failures in CI appear unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] dajac commented on a diff in pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
dajac commented on code in PR #14046: URL: https://github.com/apache/kafka/pull/14046#discussion_r1268592972 ## clients/src/main/java/org/apache/kafka/common/requests/OffsetCommitRequest.java: ## @@ -89,33 +88,29 @@ public Map offsets() { return offsets; } -

[GitHub] [kafka] dajac commented on pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
dajac commented on PR #14046: URL: https://github.com/apache/kafka/pull/14046#issuecomment-1642653815 > Something strange is going on with AuthorizerIntegrationTest in the build, but that might be unrelated. I will look into that. @jolshan I found the issue related to

[GitHub] [kafka] ijuma merged pull request #13949: KAFKA-15141: init logger statically on hot codepaths

2023-07-19 Thread via GitHub
ijuma merged PR #13949: URL: https://github.com/apache/kafka/pull/13949 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] C0urante commented on a diff in pull request #13801: KAFKA-15018: Failing offset flush for EOS when secondary offset store writes fails for tombstone records

2023-07-19 Thread via GitHub
C0urante commented on code in PR #13801: URL: https://github.com/apache/kafka/pull/13801#discussion_r1268520485 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/ConnectorOffsetBackingStore.java: ## @@ -279,10 +280,33 @@ public Future set(Map values, Callback

[GitHub] [kafka] jolshan commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
jolshan commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268535712 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetAndMetadata.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] jolshan commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
jolshan commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268526325 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -386,6 +386,18 @@ public short groupMetadataValueVersion() { }

[GitHub] [kafka] jolshan commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
jolshan commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268525603 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -386,6 +386,18 @@ public short groupMetadataValueVersion() { }

[GitHub] [kafka] yashmayya commented on a diff in pull request #14005: KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors

2023-07-19 Thread via GitHub
yashmayya commented on code in PR #14005: URL: https://github.com/apache/kafka/pull/14005#discussion_r1268520364 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationExactlyOnceTest.java: ## @@ -46,4 +51,49 @@ public void

[GitHub] [kafka] yashmayya commented on pull request #14044: KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers

2023-07-19 Thread via GitHub
yashmayya commented on PR #14044: URL: https://github.com/apache/kafka/pull/14044#issuecomment-1642566534 > The change from private to protected technically counts as a change to public interface, so we'd need a KIP for that Ah, I did wonder about this but wasn't entirely certain,

[GitHub] [kafka] jolshan commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
jolshan commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268514084 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetAndMetadata.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] yashmayya commented on pull request #14036: MINOR: Fix Javadocs for SourceTaskContext::transactionContext and SinkTaskContext::errantRecordReporter to use NoSuchMethodError instead of

2023-07-19 Thread via GitHub
yashmayya commented on PR #14036: URL: https://github.com/apache/kafka/pull/14036#issuecomment-1642557203 Thanks Greg! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] philipnee commented on pull request #13797: KAFKA-14950: implement assign() and assignment()

2023-07-19 Thread via GitHub
philipnee commented on PR #13797: URL: https://github.com/apache/kafka/pull/13797#issuecomment-1642545962 @junrao - thanks for the review. I've addressed the two issues you pointed out. Would you be able to take another look at it? -- This is an automated message from the Apache Git

[GitHub] [kafka] vamossagar12 opened a new pull request, #14051: KAFKA-15218: Avoid NPE thrown while deleting topic and fetch from follower concurrently

2023-07-19 Thread via GitHub
vamossagar12 opened a new pull request, #14051: URL: https://github.com/apache/kafka/pull/14051 When deleting topics, we'll first clear all the remoteReplicaMap when stopPartitions

[GitHub] [kafka] kamalcph commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
kamalcph commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1268420993 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] kamalcph commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
kamalcph commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1268399339 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[jira] [Commented] (KAFKA-15190) Allow configuring a streams process ID

2023-07-19 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744726#comment-17744726 ] Matthias J. Sax commented on KAFKA-15190: - {quote}but although {{StreamsPartitionAssignor}} 

[GitHub] [kafka] vamossagar12 commented on pull request #14000: [MINOR] Fixing comment with IncrementalCooperativeAssignor#handleLostAssignments

2023-07-19 Thread via GitHub
vamossagar12 commented on PR #14000: URL: https://github.com/apache/kafka/pull/14000#issuecomment-1642481356 Thanks @gharris1727 . hmm the meaning of the variable and it's usage in the comment is slightly off in this case. `revokedInPrevious` being true doesn't just signify successive

[GitHub] [kafka] kamalcph commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
kamalcph commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1268399339 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] gharris1727 merged pull request #14036: MINOR: Fix Javadocs for SourceTaskContext::transactionContext and SinkTaskContext::errantRecordReporter to use NoSuchMethodError instead of NoS

2023-07-19 Thread via GitHub
gharris1727 merged PR #14036: URL: https://github.com/apache/kafka/pull/14036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] satishd commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
satishd commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1268373435 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] jolshan commented on pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
jolshan commented on PR #14046: URL: https://github.com/apache/kafka/pull/14046#issuecomment-1642437560 Something strange is going on with AuthorizerIntegrationTest in the build, but that might be unrelated. I will look into that. -- This is an automated message from the Apache Git

[GitHub] [kafka] splett2 opened a new pull request, #14050: KAFKA-15220: Do not returned fenced brokers from getAliveBrokerNode

2023-07-19 Thread via GitHub
splett2 opened a new pull request, #14050: URL: https://github.com/apache/kafka/pull/14050 `getAliveBrokerNode` returns fenced brokers as alive which is inconsistent with methods like getAliveBrokerNodes. Add a filter to not return fenced brokers and adds a test to validate that

[GitHub] [kafka] gharris1727 commented on pull request #14000: [MINOR] Fixing comment with IncrementalCooperativeAssignor#handleLostAssignments

2023-07-19 Thread via GitHub
gharris1727 commented on PR #14000: URL: https://github.com/apache/kafka/pull/14000#issuecomment-1642429559 Hey @vamossagar12 I don't think that this comment is incorrect or confusing enough to warrant stand-alone PR. If you have other substantive changes in this area, then we can

[jira] [Created] (KAFKA-15220) KRaftMetadataCache returns fenced brokers from getAliveBrokerNode

2023-07-19 Thread David Mao (Jira)
David Mao created KAFKA-15220: - Summary: KRaftMetadataCache returns fenced brokers from getAliveBrokerNode Key: KAFKA-15220 URL: https://issues.apache.org/jira/browse/KAFKA-15220 Project: Kafka

[GitHub] [kafka] jolshan commented on a diff in pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
jolshan commented on code in PR #14046: URL: https://github.com/apache/kafka/pull/14046#discussion_r1268350437 ## clients/src/main/java/org/apache/kafka/common/requests/OffsetCommitRequest.java: ## @@ -89,33 +88,29 @@ public Map offsets() { return offsets; } -

[GitHub] [kafka] jolshan commented on pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
jolshan commented on PR #14046: URL: https://github.com/apache/kafka/pull/14046#issuecomment-1642424251 > Note the use of "latestVersionUnstable": true in the request schema. This means that this new version is not available yet unless activated. This also leaves us room to add the

[GitHub] [kafka] dajac commented on pull request #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
dajac commented on PR #14046: URL: https://github.com/apache/kafka/pull/14046#issuecomment-1642425069 > > Note the use of "latestVersionUnstable": true in the request schema. This means that this new version is not available yet unless activated. > > This also leaves us room to add

[GitHub] [kafka] gharris1727 commented on pull request #14020: KAFKA-10579: Make Reflections thread safe to resolve flaky NPE scanning failure

2023-07-19 Thread via GitHub
gharris1727 commented on PR #14020: URL: https://github.com/apache/kafka/pull/14020#issuecomment-1642410512 Closing this in favor of upgrading the library. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] gharris1727 closed pull request #14020: KAFKA-10579: Make Reflections thread safe to resolve flaky NPE scanning failure

2023-07-19 Thread via GitHub
gharris1727 closed pull request #14020: KAFKA-10579: Make Reflections thread safe to resolve flaky NPE scanning failure URL: https://github.com/apache/kafka/pull/14020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Commented] (KAFKA-15219) Support delegation tokens in KRaft

2023-07-19 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744700#comment-17744700 ] Proven Provenzano commented on KAFKA-15219: --- I'm sorry, I seem to have forgotten to create the

[jira] [Resolved] (KAFKA-15211) DistributedConfigTest#shouldFailWithInvalidKeySize fails when run after TestSslUtils#generate

2023-07-19 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15211. - Fix Version/s: 3.6.0 Resolution: Fixed >

[GitHub] [kafka] nizhikov commented on pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on PR #13278: URL: https://github.com/apache/kafka/pull/13278#issuecomment-1642362522 @mimaison all your comments addressed except the one with the Tuple. Please, review. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] gharris1727 merged pull request #14039: KAFKA-15211: Mock InvalidParameterException in DistributedConfigTest

2023-07-19 Thread via GitHub
gharris1727 merged PR #14039: URL: https://github.com/apache/kafka/pull/14039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] nizhikov commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268293020 ## tools/src/main/java/org/apache/kafka/tools/DeleteRecordsCommand.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] gharris1727 commented on pull request #14039: KAFKA-15211: Mock InvalidParameterException in DistributedConfigTest

2023-07-19 Thread via GitHub
gharris1727 commented on PR #14039: URL: https://github.com/apache/kafka/pull/14039#issuecomment-1642357913 Test failures look unrelated, and tests pass locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] nizhikov commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268286710 ## tools/src/main/java/org/apache/kafka/tools/DeleteRecordsCommand.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Updated] (KAFKA-14094) KIP-853: KRaft controller memebership changes

2023-07-19 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio updated KAFKA-14094: --- Summary: KIP-853: KRaft controller memebership changes (was:

[jira] [Updated] (KAFKA-14094) KIP-853: KRaft Voters Change

2023-07-19 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-14094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] José Armando García Sancio updated KAFKA-14094: --- Fix Version/s: (was: 3.6.0) > KIP-853: KRaft Voters Change >

[jira] [Updated] (KAFKA-15219) Support delegation tokens in KRaft

2023-07-19 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass updated KAFKA-15219: Description: Delegation tokens have been created in KIP-48 and improved in

[jira] [Created] (KAFKA-15219) Support delegation tokens in KRaft

2023-07-19 Thread Viktor Somogyi-Vass (Jira)
Viktor Somogyi-Vass created KAFKA-15219: --- Summary: Support delegation tokens in KRaft Key: KAFKA-15219 URL: https://issues.apache.org/jira/browse/KAFKA-15219 Project: Kafka Issue Type:

[GitHub] [kafka] nizhikov commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268278161 ## tools/src/test/java/org/apache/kafka/tools/DeleteRecordsCommandTest.java: ## @@ -0,0 +1,191 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] nizhikov commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268270865 ## tools/src/test/java/org/apache/kafka/tools/CoreUtilsTest.java: ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[GitHub] [kafka] nizhikov commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
nizhikov commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268252459 ## tools/src/main/java/org/apache/kafka/tools/CoreUtils.java: ## @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[GitHub] [kafka] mimaison commented on pull request #13260: KAFKA-14661: Upgrade Zookeeper to 3.8.1

2023-07-19 Thread via GitHub
mimaison commented on PR #13260: URL: https://github.com/apache/kafka/pull/13260#issuecomment-1642274928 My point is that this really contradicts the KIP. In the KIP we say "it does not work" but then we have a test that relies on the upgrade working. This is a bit counter intuitive. --

[GitHub] [kafka] C0urante commented on pull request #13853: KAFKA-15088: Fixing Incorrect Reference Usage in Connector State Changes

2023-07-19 Thread via GitHub
C0urante commented on PR #13853: URL: https://github.com/apache/kafka/pull/13853#issuecomment-1642271686 The refactor that @gharris1727 is suggesting here would also be nice since it would allow us to establish states that are only applicable to `Connector` or `Task` instances. There's a

[GitHub] [kafka] C0urante commented on pull request #14044: KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers

2023-07-19 Thread via GitHub
C0urante commented on PR #14044: URL: https://github.com/apache/kafka/pull/14044#issuecomment-1642230322 The change from `private` to `protected` technically counts as a change to public interface, so we'd need a KIP for that. I'm also a little hesitant to upgrade the visibility of these

[GitHub] [kafka] fvaleri commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
fvaleri commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268183773 ## tools/src/main/java/org/apache/kafka/tools/DeleteRecordsCommand.java: ## @@ -0,0 +1,217 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] fvaleri commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
fvaleri commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268183325 ## tools/src/main/java/org/apache/kafka/tools/CoreUtils.java: ## @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[jira] [Commented] (KAFKA-8128) Dynamic delegation token change possibility for consumer/producer

2023-07-19 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17744640#comment-17744640 ] Viktor Somogyi-Vass commented on KAFKA-8128: So after 4 years (yes) I think I'll pick this

[jira] [Assigned] (KAFKA-8128) Dynamic delegation token change possibility for consumer/producer

2023-07-19 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass reassigned KAFKA-8128: -- Assignee: Viktor Somogyi-Vass > Dynamic delegation token change possibility

[GitHub] [kafka] mimaison commented on a diff in pull request #13278: KAFKA-14591 DeleteRecordsCommand moved to tools

2023-07-19 Thread via GitHub
mimaison commented on code in PR #13278: URL: https://github.com/apache/kafka/pull/13278#discussion_r1268090551 ## tools/src/main/java/org/apache/kafka/tools/CoreUtils.java: ## @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

[GitHub] [kafka] viktorsomogyi commented on pull request #13975: KAFKA-15161: Fix InvalidReplicationFactorException at connect startup

2023-07-19 Thread via GitHub
viktorsomogyi commented on PR #13975: URL: https://github.com/apache/kafka/pull/13975#issuecomment-1642093548 Captured two screen recordings to demonstrate the reproduction of the problem. This is the bug:

[GitHub] [kafka] clolov opened a new pull request, #14049: KAFKA-14038: Optimise calculation of size for log in remote tier

2023-07-19 Thread via GitHub
clolov opened a new pull request, #14049: URL: https://github.com/apache/kafka/pull/14049 ### Summary This pull request aims to address sub-points 1 and 2 from

[jira] [Updated] (KAFKA-15129) Clean up all metrics that were forgotten to be closed

2023-07-19 Thread hudeqi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hudeqi updated KAFKA-15129: --- Description: In the current kafka code, there are still many module metrics that are forgotten to be

[GitHub] [kafka] msn-tldr opened a new pull request, #14048: On metadata-update, mark paritions for immediate retry

2023-07-19 Thread via GitHub
msn-tldr opened a new pull request, #14048: URL: https://github.com/apache/kafka/pull/14048 Alternative POC -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [kafka] dajac commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
dajac commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268008501 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordHelpers.java: ## @@ -467,6 +471,72 @@ public static Record newEmptyGroupMetadataRecord(

[GitHub] [kafka] dajac commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
dajac commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268008967 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/RecordHelpers.java: ## @@ -467,6 +471,72 @@ public static Record newEmptyGroupMetadataRecord(

[GitHub] [kafka] dajac commented on a diff in pull request #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
dajac commented on code in PR #14047: URL: https://github.com/apache/kafka/pull/14047#discussion_r1268008079 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -386,6 +386,18 @@ public short groupMetadataValueVersion() { } }

[GitHub] [kafka] dajac opened a new pull request, #14047: KAFKA-14499: [2/N] Add OffsetCommit record & related

2023-07-19 Thread via GitHub
dajac opened a new pull request, #14047: URL: https://github.com/apache/kafka/pull/14047 This patch does a few things: 1) It introduces the `OffsetAndMetadata` class which hold the committed offsets in the group coordinator. 2) It adds methods to deal with OffsetCommit records to

[GitHub] [kafka] dajac opened a new pull request, #14046: KAFKA-14499: [1/N] Introduce OffsetCommit API version 9 and add new StaleMemberEpochException error

2023-07-19 Thread via GitHub
dajac opened a new pull request, #14046: URL: https://github.com/apache/kafka/pull/14046 This patch does a few things: 1) It introduces version 9 of the OffsetCommit API. This new version has no schema changes but it can return a StaleMemberEpochException if the new consumer group

[GitHub] [kafka] Nickstery commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
Nickstery commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1267948092 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] Nickstery commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
Nickstery commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1267954353 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] Nickstery commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
Nickstery commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1267954353 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] Nickstery commented on a diff in pull request #14004: KAFKA-15168: Handle overlapping remote log segments in RemoteLogMetadata cache

2023-07-19 Thread via GitHub
Nickstery commented on code in PR #14004: URL: https://github.com/apache/kafka/pull/14004#discussion_r1267954353 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogLeaderEpochState.java: ## @@ -100,17 +100,29 @@ void

[GitHub] [kafka] clolov commented on pull request #13260: KAFKA-14661: Upgrade Zookeeper to 3.8.1

2023-07-19 Thread via GitHub
clolov commented on PR #13260: URL: https://github.com/apache/kafka/pull/13260#issuecomment-1641957263 Hey @forlack! Thank you for the suggestion, I have upgraded the dependency so that tests can run. Heya @mimaison, I have addressed your comment about adding the latest version to

  1   2   >