[GitHub] [kafka] ableegoldman opened a new pull request, #12585: HOTFIX: fix PriorityQueue iteration to assign warmups in priority order

2022-09-01 Thread GitBox
ableegoldman opened a new pull request, #12585: URL: https://github.com/apache/kafka/pull/12585 Based on a patch submitted to the `confluentinc` fork & then abandoned. Needed some updates and minor expansion but more or less just re-applied the changes proposed in https://github.com/conflue

[jira] [Created] (KAFKA-14196) Flaky OffsetValidationTest seems to indicate potential duplication issue during rebalance

2022-09-01 Thread Philip Nee (Jira)
Philip Nee created KAFKA-14196: -- Summary: Flaky OffsetValidationTest seems to indicate potential duplication issue during rebalance Key: KAFKA-14196 URL: https://issues.apache.org/jira/browse/KAFKA-14196

[GitHub] [kafka] mghosh4 commented on pull request #10910: KAFKA-12965 - Graceful clean up of task error metrics

2022-09-01 Thread GitBox
mghosh4 commented on PR #10910: URL: https://github.com/apache/kafka/pull/10910#issuecomment-1234907832 Any update on when this diff can be landed? Incase you need some help, I am happy to help. This is a problem that we have had to deal with too. -- This is an automated message from the

[GitHub] [kafka] philipnee closed pull request #12580: MINOR: Fix flaky OffsetValidationTest

2022-09-01 Thread GitBox
philipnee closed pull request #12580: MINOR: Fix flaky OffsetValidationTest URL: https://github.com/apache/kafka/pull/12580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [kafka] cmccabe merged pull request #12578: KAFKA-14195: Fix KRaft AlterConfig policy usage for Legacy/Full case

2022-09-01 Thread GitBox
cmccabe merged PR #12578: URL: https://github.com/apache/kafka/pull/12578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] vvcephei merged pull request #12582: MINOR: Demystify rebalance schedule log

2022-09-01 Thread GitBox
vvcephei merged PR #12582: URL: https://github.com/apache/kafka/pull/12582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] vvcephei commented on pull request #12582: MINOR: Demystify rebalance schedule log

2022-09-01 Thread GitBox
vvcephei commented on PR #12582: URL: https://github.com/apache/kafka/pull/12582#issuecomment-1234803264 The java 17 build failed with this unrelated exception: ``` 10:00:43 Execution failed for task ':core:unitTest'. 10:00:43 > Process 'Gradle Test Executor 50' finished with n

[GitHub] [kafka] wcarlson5 commented on a diff in pull request #12583: KAFKA-10199: Separate state updater from old restore

2022-09-01 Thread GitBox
wcarlson5 commented on code in PR #12583: URL: https://github.com/apache/kafka/pull/12583#discussion_r960982217 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java: ## @@ -396,8 +397,7 @@ public static StreamThread create(final TopologyMetadat

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r961032133 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,16 +272,20 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r961012010 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,16 +272,20 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
guozhangwang commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r961002924 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] guozhangwang commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
guozhangwang commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960983275 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,16 +272,20 @@ private void maybeAddNodeForOptimizationMe

[jira] [Created] (KAFKA-14195) Fix KRaft AlterConfig policy usage for Legacy/Full case

2022-09-01 Thread Ron Dagostino (Jira)
Ron Dagostino created KAFKA-14195: - Summary: Fix KRaft AlterConfig policy usage for Legacy/Full case Key: KAFKA-14195 URL: https://issues.apache.org/jira/browse/KAFKA-14195 Project: Kafka Iss

[GitHub] [kafka] rondagostino commented on a diff in pull request #12578: KAFKA-14039: Additional compatibility fix for KRaft Alter Configs

2022-09-01 Thread GitBox
rondagostino commented on code in PR #12578: URL: https://github.com/apache/kafka/pull/12578#discussion_r960958343 ## metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java: ## @@ -248,33 +249,45 @@ private void incrementalAlterConfigResource(Config

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960956896 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] jsancio commented on a diff in pull request #12578: KAFKA-14039: Additional compatibility fix for KRaft Alter Configs

2022-09-01 Thread GitBox
jsancio commented on code in PR #12578: URL: https://github.com/apache/kafka/pull/12578#discussion_r960954024 ## metadata/src/main/java/org/apache/kafka/controller/ConfigurationControlManager.java: ## @@ -248,33 +249,45 @@ private void incrementalAlterConfigResource(ConfigResou

[GitHub] [kafka] ashmeet13 commented on pull request #12414: KAFKA-14073 Logging the reason for Snapshot

2022-09-01 Thread GitBox
ashmeet13 commented on PR #12414: URL: https://github.com/apache/kafka/pull/12414#issuecomment-1234551652 Thank you @jsancio for the review. Have made the changes. For the following comment - >It looks like this method is only used by tests. I think it is okay for this to have an "

[GitHub] [kafka] ashmeet13 commented on a diff in pull request #12414: KAFKA-14073 Logging the reason for Snapshot

2022-09-01 Thread GitBox
ashmeet13 commented on code in PR #12414: URL: https://github.com/apache/kafka/pull/12414#discussion_r960902453 ## core/src/main/scala/kafka/server/metadata/MetadataSnapshotter.scala: ## @@ -31,5 +32,5 @@ trait MetadataSnapshotter { * * @return True

[GitHub] [kafka] ashmeet13 commented on a diff in pull request #12414: KAFKA-14073 Logging the reason for Snapshot

2022-09-01 Thread GitBox
ashmeet13 commented on code in PR #12414: URL: https://github.com/apache/kafka/pull/12414#discussion_r960901993 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataSnapshotter.scala: ## @@ -59,11 +60,11 @@ class BrokerMetadataSnapshotter( val writer = writerBuilder

[GitHub] [kafka] ashmeet13 commented on a diff in pull request #12414: KAFKA-14073 Logging the reason for Snapshot

2022-09-01 Thread GitBox
ashmeet13 commented on code in PR #12414: URL: https://github.com/apache/kafka/pull/12414#discussion_r960901730 ## core/src/main/scala/kafka/server/metadata/BrokerMetadataListener.scala: ## @@ -119,16 +120,29 @@ class BrokerMetadataListener( } _bytesSinceLastSnap

[GitHub] [kafka] ndrwdn opened a new pull request, #12584: KAFKA-14194 Fix NPE in Cluster.nodeIfOnline

2022-09-01 Thread GitBox
ndrwdn opened a new pull request, #12584: URL: https://github.com/apache/kafka/pull/12584 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[jira] [Updated] (KAFKA-14194) NPE in Cluster.nodeIfOnline

2022-09-01 Thread Andrew Dean (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Dean updated KAFKA-14194: Component/s: clients > NPE in Cluster.nodeIfOnline > --- > >

[jira] [Updated] (KAFKA-14194) NPE in Cluster.nodeIfOnline

2022-09-01 Thread Andrew Dean (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Dean updated KAFKA-14194: Affects Version/s: 3.2.1 > NPE in Cluster.nodeIfOnline > --- > >

[jira] [Created] (KAFKA-14194) NPE in Cluster.nodeIfOnline

2022-09-01 Thread Andrew Dean (Jira)
Andrew Dean created KAFKA-14194: --- Summary: NPE in Cluster.nodeIfOnline Key: KAFKA-14194 URL: https://issues.apache.org/jira/browse/KAFKA-14194 Project: Kafka Issue Type: Bug Reporte

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960845326 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960845326 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] cadonna commented on pull request #12519: KAFKA-10199: Handle exceptions from state updater

2022-09-01 Thread GitBox
cadonna commented on PR #12519: URL: https://github.com/apache/kafka/pull/12519#issuecomment-1234414581 I am sorry, one of my merged PRs has introduced conflicts here. 🙁 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] mimaison commented on pull request #11783: KAFKA-14143: Exactly-once source connector system tests (KIP-618)

2022-09-01 Thread GitBox
mimaison commented on PR #11783: URL: https://github.com/apache/kafka/pull/11783#issuecomment-1234403651 Yes if the system tests pass, I'm in favor of merging this in 3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[jira] [Comment Edited] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599002#comment-17599002 ] John Gray edited comment on KAFKA-14172 at 9/1/22 2:56 PM: --- I

[GitHub] [kafka] cadonna commented on a diff in pull request #12583: KAFKA-10199: Separate state updater from old restore

2022-09-01 Thread GitBox
cadonna commented on code in PR #12583: URL: https://github.com/apache/kafka/pull/12583#discussion_r960754245 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -280,25 +280,17 @@ public void shouldClassifyExistingTasksWithStateUpda

[GitHub] [kafka] cadonna commented on pull request #12583: KAFKA-10199: Separate state updater from old restore

2022-09-01 Thread GitBox
cadonna commented on PR #12583: URL: https://github.com/apache/kafka/pull/12583#issuecomment-1234388852 Call for review: @wcarlson5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [kafka] cadonna opened a new pull request, #12583: KAFKA-10199: Separate state updater from old restore

2022-09-01 Thread GitBox
cadonna opened a new pull request, #12583: URL: https://github.com/apache/kafka/pull/12583 Separates the code path for the new state updater from the code path of the old restoration. Ensures that with the state updater tasks are processed before all tasks are running. ###

[jira] [Comment Edited] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599002#comment-17599002 ] John Gray edited comment on KAFKA-14172 at 9/1/22 2:44 PM: --- I

[jira] [Comment Edited] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598956#comment-17598956 ] John Gray edited comment on KAFKA-14172 at 9/1/22 2:41 PM: --- My

[jira] [Commented] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599002#comment-17599002 ] John Gray commented on KAFKA-14172: --- I know next to nothing about the internal working

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960708055 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[jira] [Comment Edited] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598956#comment-17598956 ] John Gray edited comment on KAFKA-14172 at 9/1/22 1:29 PM: --- My

[GitHub] [kafka] C0urante commented on pull request #11783: KAFKA-14143: Exactly-once source connector system tests (KIP-618)

2022-09-01 Thread GitBox
C0urante commented on PR #11783: URL: https://github.com/apache/kafka/pull/11783#issuecomment-1234282325 @showuon @mimaison @jsancio are we okay with adding this to the 3.3 branch? There's very little additional risk; the only changes to non-testing code are an additional branch [here](htt

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960654203 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960647123 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -356,6 +381,95 @@ private void mergeDuplicateSourceNodes() {

[jira] [Comment Edited] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598956#comment-17598956 ] John Gray edited comment on KAFKA-14172 at 9/1/22 1:12 PM: --- My

[jira] [Commented] (KAFKA-14172) bug: State stores lose state when tasks are reassigned under EOS wit…

2022-09-01 Thread John Gray (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598956#comment-17598956 ] John Gray commented on KAFKA-14172: --- My stateful/EOS Kafka apps also seem to be strugg

[GitHub] [kafka] vvcephei opened a new pull request, #12582: MINOR: Demystify rebalance schedule log

2022-09-01 Thread GitBox
vvcephei opened a new pull request, #12582: URL: https://github.com/apache/kafka/pull/12582 People have a hard time understanding what's going on with probing rebalances, partly due to cryptic messages like this: ``` 2022-08-31T12:03:29.197+09:00 . Requested to schedule probing

[GitHub] [kafka] cadonna merged pull request #12573: KAFKA-10199: Remove changelog unregister from state updater

2022-09-01 Thread GitBox
cadonna merged PR #12573: URL: https://github.com/apache/kafka/pull/12573 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] cadonna commented on pull request #12573: KAFKA-10199: Remove changelog unregister from state updater

2022-09-01 Thread GitBox
cadonna commented on PR #12573: URL: https://github.com/apache/kafka/pull/12573#issuecomment-1234212740 Build failures are not related: ``` Build / JDK 11 and Scala 2.13 / org.apache.kafka.trogdor.coordinator.CoordinatorTest.testTaskRequestWithOldStartMsGetsUpdated() Build / JDK 17

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960474042 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,17 +276,36 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960457146 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,16 +272,20 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] vpapavas commented on a diff in pull request #12555: Optimize self-join

2022-09-01 Thread GitBox
vpapavas commented on code in PR #12555: URL: https://github.com/apache/kafka/pull/12555#discussion_r960457146 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -270,16 +272,20 @@ private void maybeAddNodeForOptimizationMetada

[GitHub] [kafka] yashmayya commented on a diff in pull request #12478: KAFKA-13952 fix RetryWithToleranceOperator to respect infinite retries configuration

2022-09-01 Thread GitBox
yashmayya commented on code in PR #12478: URL: https://github.com/apache/kafka/pull/12478#discussion_r960439387 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperatorTest.java: ## @@ -199,112 +202,156 @@ private RetryWithToleranceOper

[GitHub] [kafka] yashmayya commented on a diff in pull request #12478: KAFKA-13952 fix RetryWithToleranceOperator to respect infinite retries configuration

2022-09-01 Thread GitBox
yashmayya commented on code in PR #12478: URL: https://github.com/apache/kafka/pull/12478#discussion_r960436353 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperator.java: ## @@ -175,9 +179,8 @@ protected V execAndRetry(Operation op

[GitHub] [kafka] yashmayya commented on a diff in pull request #12478: KAFKA-13952 fix RetryWithToleranceOperator to respect infinite retries configuration

2022-09-01 Thread GitBox
yashmayya commented on code in PR #12478: URL: https://github.com/apache/kafka/pull/12478#discussion_r960435913 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/errors/RetryWithToleranceOperator.java: ## @@ -269,7 +289,7 @@ void backoff(int attempt, long deadlin

[GitHub] [kafka] jokeryin432 opened a new pull request, #12581: corporate.xml

2022-09-01 Thread GitBox
jokeryin432 opened a new pull request, #12581: URL: https://github.com/apache/kafka/pull/12581 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of tes

[GitHub] [kafka] nizhikov commented on pull request #12574: KAFKA-13908 Rethrow ExecutionException to preserve original cause

2022-09-01 Thread GitBox
nizhikov commented on PR #12574: URL: https://github.com/apache/kafka/pull/12574#issuecomment-1233908096 Hello @showuon. Can you, please, take a look at my changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[jira] [Commented] (KAFKA-13077) Replication failing after unclean shutdown of ZK and all brokers

2022-09-01 Thread Sergey Ivanov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598790#comment-17598790 ] Sergey Ivanov commented on KAFKA-13077: --- [~ivanyu], ohh, we continued investigatio