[jira] [Updated] (KAFKA-13804) Log broker shutdown reason during startup at the end of log output

2022-04-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-13804: -- Description: Currently, when the broker hit exception during startup, we will first log the exception

[jira] [Updated] (KAFKA-13804) Log broker shutdown reason during startup at the end of log output

2022-04-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-13804: -- Summary: Log broker shutdown reason during startup at the end of log output (was: Log broker shutdown

[GitHub] [kafka] mjsax commented on a diff in pull request #12030: KAFKA-13785: [5/N][emit final] cache for time ordered window store

2022-04-11 Thread GitBox
mjsax commented on code in PR #12030: URL: https://github.com/apache/kafka/pull/12030#discussion_r847995239 ## streams/src/main/java/org/apache/kafka/streams/state/internals/PrefixedWindowKeySchemas.java: ## @@ -33,8 +33,8 @@ public class PrefixedWindowKeySchemas { priva

[GitHub] [kafka] showuon commented on pull request #12028: KAFKA-13804: output the reason why broker exit unexpectedly during startup

2022-04-11 Thread GitBox
showuon commented on PR #12028: URL: https://github.com/apache/kafka/pull/12028#issuecomment-1096122509 @mimaison @dajac , please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] cmccabe opened a new pull request, #12033: KAFKA-13807: Fix incrementalAlterConfig and refactor some things

2022-04-11 Thread GitBox
cmccabe opened a new pull request, #12033: URL: https://github.com/apache/kafka/pull/12033 Ensure that we can set log.flush.interval.ms at the broker or cluster level via IncrementalAlterConfigs. THis was broken by KAFKA-13749, which added log.flush.interval.ms as the second synonym r

[GitHub] [kafka] hachikuji commented on a diff in pull request #12032: MINOR: Change validation `AlterPartition` validation order in `KafkaController`

2022-04-11 Thread GitBox
hachikuji commented on code in PR #12032: URL: https://github.com/apache/kafka/pull/12032#discussion_r847812399 ## core/src/main/scala/kafka/controller/KafkaController.scala: ## @@ -2348,7 +2348,20 @@ class KafkaController(val config: KafkaConfig, controllerContext.pa

[GitHub] [kafka] jsancio commented on a diff in pull request #12032: MINOR: Change validation `AlterPartition` validation order in `KafkaController`

2022-04-11 Thread GitBox
jsancio commented on code in PR #12032: URL: https://github.com/apache/kafka/pull/12032#discussion_r847810681 ## core/src/main/scala/kafka/controller/KafkaController.scala: ## @@ -2348,7 +2348,20 @@ class KafkaController(val config: KafkaConfig, controllerContext.part

[jira] [Updated] (KAFKA-13816) Downgrading Connect rebalancing protocol from incremental to eager causes duplicate task instances

2022-04-11 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-13816: -- Description: The rebalancing protocol for a Kafka Connect cluster can be downgraded from incr

[GitHub] [kafka] hachikuji opened a new pull request, #12032: MINOR: Change validation `AlterPartition` validation order in `KafkaController`

2022-04-11 Thread GitBox
hachikuji opened a new pull request, #12032: URL: https://github.com/apache/kafka/pull/12032 Currently we validate recovery state before checking leader epoch. It seems more intuitive to validate leader epoch first since the leader might be working with stale state. This patch fixes this an

[GitHub] [kafka] vvcephei commented on a diff in pull request #11993: KAFKA-13654: Extend KStream process with new Processor API

2022-04-11 Thread GitBox
vvcephei commented on code in PR #11993: URL: https://github.com/apache/kafka/pull/11993#discussion_r84395 ## streams/src/test/java/org/apache/kafka/test/MockProcessorNode.java: ## @@ -60,7 +61,7 @@ public void init(final InternalProcessorContext context) { @Override

[GitHub] [kafka] vvcephei merged pull request #11999: [MINOR] fix(streams): align variable names

2022-04-11 Thread GitBox
vvcephei merged PR #11999: URL: https://github.com/apache/kafka/pull/11999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] jsancio merged pull request #12021: MINOR: Re-use counter in mocking of LogSegment.size

2022-04-11 Thread GitBox
jsancio merged PR #12021: URL: https://github.com/apache/kafka/pull/12021 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Comment Edited] (KAFKA-8318) Session Window Aggregations generate an extra tombstone

2022-04-11 Thread padavan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520787#comment-17520787 ] padavan edited comment on KAFKA-8318 at 4/11/22 7:51 PM: - any new

[jira] [Commented] (KAFKA-8318) Session Window Aggregations generate an extra tombstone

2022-04-11 Thread padavan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520787#comment-17520787 ] padavan commented on KAFKA-8318: any news?  > Session Window Aggregations generate an ex

[GitHub] [kafka] lihaosky commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-11 Thread GitBox
lihaosky commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r847621808 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregate.java: ## @@ -80,22 +109,54 @@ public void enableSendingOldValues() {

[jira] [Updated] (KAFKA-13819) Add application.server to Streams assignor logs when set

2022-04-11 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-13819: - Component/s: streams > Add application.server to Streams assignor logs when set > --

[jira] [Created] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13820: Summary: Add debug-level logs to explain why a store is filtered out during interactive query Key: KAFKA-13820 URL: https://issues.apache.org/jira/browse/KAFKA-13820

[jira] [Updated] (KAFKA-13818) Add generation to consumer assignor logs

2022-04-11 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-13818: - Component/s: consumer > Add generation to consumer assignor logs > -

[jira] [Updated] (KAFKA-13820) Add debug-level logs to explain why a store is filtered out during interactive query

2022-04-11 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-13820: - Component/s: streams > Add debug-level logs to explain why a store is filtered out during > int

[GitHub] [kafka] lihaosky commented on a diff in pull request #11896: KAFKA-13785: [6/N][Emit final] emit final for TimeWindowedKStreamImpl

2022-04-11 Thread GitBox
lihaosky commented on code in PR #11896: URL: https://github.com/apache/kafka/pull/11896#discussion_r847582057 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/TimeWindowedKStreamImpl.java: ## @@ -232,11 +247,19 @@ );

[jira] [Created] (KAFKA-13819) Add application.server to Streams assignor logs when set

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13819: Summary: Add application.server to Streams assignor logs when set Key: KAFKA-13819 URL: https://issues.apache.org/jira/browse/KAFKA-13819 Project: Kafka Issu

[jira] [Created] (KAFKA-13818) Add generation to consumer assignor logs

2022-04-11 Thread John Roesler (Jira)
John Roesler created KAFKA-13818: Summary: Add generation to consumer assignor logs Key: KAFKA-13818 URL: https://issues.apache.org/jira/browse/KAFKA-13818 Project: Kafka Issue Type: Improvem

[jira] [Assigned] (KAFKA-13818) Add generation to consumer assignor logs

2022-04-11 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler reassigned KAFKA-13818: Assignee: John Roesler > Add generation to consumer assignor logs > -

[GitHub] [kafka] hachikuji opened a new pull request, #12031: KAFKA-13651; Add audit logging to `StandardAuthorizer`

2022-04-11 Thread GitBox
hachikuji opened a new pull request, #12031: URL: https://github.com/apache/kafka/pull/12031 This patch adds audit support through the `kafka.authorizer.logger` logger to `StandardAuthorizer`. It follows the same conventions in `AclAuthorizer`. When `logIfAllowed` is set in the `Action`, t

[jira] [Created] (KAFKA-13817) Schedule nextTimeToEmit to system time every time instead of just once

2022-04-11 Thread Hao Li (Jira)
Hao Li created KAFKA-13817: -- Summary: Schedule nextTimeToEmit to system time every time instead of just once Key: KAFKA-13817 URL: https://issues.apache.org/jira/browse/KAFKA-13817 Project: Kafka I

[GitHub] [kafka] vvcephei commented on a diff in pull request #12020: MINOR: Add generation to consumer assignor logs

2022-04-11 Thread GitBox
vvcephei commented on code in PR #12020: URL: https://github.com/apache/kafka/pull/12020#discussion_r847561327 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -86,10 +87,23 @@ import static org.apache.kafka.streams.pro

[GitHub] [kafka] ccding commented on a diff in pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-11 Thread GitBox
ccding commented on code in PR #12029: URL: https://github.com/apache/kafka/pull/12029#discussion_r847548177 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -678,7 +678,7 @@ class UnifiedLog(@volatile var logStartOffset: Long, * * @throws KafkaStorageException i

[GitHub] [kafka] lihaosky opened a new pull request, #12030: cache for time ordered window store

2022-04-11 Thread GitBox
lihaosky opened a new pull request, #12030: URL: https://github.com/apache/kafka/pull/12030 ### Description A new cache for `RocksDBTimeOrderedWindowStore`. Need this because `RocksDBTimeOrderedWindowStore`'s key ordering if different from `CachingWindowStore` which has issues for `Merge

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-11 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long ensureLea

[GitHub] [kafka] hachikuji commented on a diff in pull request #12027: MINOR: Fix TestDowngrade.test_upgrade_and_downgrade

2022-04-11 Thread GitBox
hachikuji commented on code in PR #12027: URL: https://github.com/apache/kafka/pull/12027#discussion_r847524509 ## tests/kafkatest/tests/core/downgrade_test.py: ## @@ -139,13 +139,13 @@ def test_upgrade_and_downgrade(self, version, compression_types, security_protoc

[GitHub] [kafka] gitlw commented on pull request #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-11 Thread GitBox
gitlw commented on PR #12029: URL: https://github.com/apache/kafka/pull/12029#issuecomment-1095270481 @dajac @junrao This PR is related to https://github.com/apache/kafka/pull/8672. Can you please take a look? Thank you! -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-11 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long ensureLea

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-11 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long ensureLea

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-11 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long ensureLea

[GitHub] [kafka] YeonCheolGit commented on a diff in pull request #12019: KAFKA-13764: Improve balancing algorithm for Connect incremental rebalancing

2022-04-11 Thread GitBox
YeonCheolGit commented on code in PR #12019: URL: https://github.com/apache/kafka/pull/12019#discussion_r847446537 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -200,283 +229,149 @@ private Long ensureLea

[GitHub] [kafka] gitlw opened a new pull request, #12029: KAFKA-13815: Avoid reinitialization for a replica that is being deleted

2022-04-11 Thread GitBox
gitlw opened a new pull request, #12029: URL: https://github.com/apache/kafka/pull/12029 This PR tries to avoid the reinitialization of the leader epoch cache and the partition metadata if the corresponding replica is being deleted. With this change, the asyncDelete method can run more

[jira] [Commented] (KAFKA-13816) Downgrading Connect rebalancing protocol from incremental to eager causes duplicate task instances

2022-04-11 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520572#comment-17520572 ] Chris Egerton commented on KAFKA-13816: --- I won't have time to work on this but I h

[GitHub] [kafka] dengziming commented on pull request #12024: MINOR: Move some integration tests to unit tests

2022-04-11 Thread GitBox
dengziming commented on PR #12024: URL: https://github.com/apache/kafka/pull/12024#issuecomment-1095046509 Hello @dajac , Maybe you can take a look at this because those tests in `TopicCommandIntegrationTest` are created by you. -- This is an automated message from the Apache Git Service.

[jira] [Created] (KAFKA-13816) Downgrading Connect rebalancing protocol from incremental to eager causes duplicate task instances

2022-04-11 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-13816: - Summary: Downgrading Connect rebalancing protocol from incremental to eager causes duplicate task instances Key: KAFKA-13816 URL: https://issues.apache.org/jira/browse/KAFKA-138

[GitHub] [kafka] showuon commented on pull request #10367: KAFKA-12495: allow consecutive revoke in incremental cooperative assignor in connector

2022-04-11 Thread GitBox
showuon commented on PR #10367: URL: https://github.com/apache/kafka/pull/10367#issuecomment-1095006324 > @showuon @C0urante @kkonstantine What is the status of this PR? As far as I understand this PR might resolve [KAFKA-8391](https://issues.apache.org/jira/browse/KAFKA-8391), [KAFKA-1228

[jira] [Assigned] (KAFKA-13815) Avoid reinitialization for a replica that is being deleted

2022-04-11 Thread Lucas Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Wang reassigned KAFKA-13815: -- Assignee: Lucas Wang > Avoid reinitialization for a replica that is being deleted > -

[jira] [Created] (KAFKA-13815) Avoid reinitialization for a replica that is being deleted

2022-04-11 Thread Lucas Wang (Jira)
Lucas Wang created KAFKA-13815: -- Summary: Avoid reinitialization for a replica that is being deleted Key: KAFKA-13815 URL: https://issues.apache.org/jira/browse/KAFKA-13815 Project: Kafka Issue

[GitHub] [kafka] cadonna commented on pull request #10367: KAFKA-12495: allow consecutive revoke in incremental cooperative assignor in connector

2022-04-11 Thread GitBox
cadonna commented on PR #10367: URL: https://github.com/apache/kafka/pull/10367#issuecomment-1094942596 @showuon @C0urante @kkonstantine What is the status of this PR? As far as I understand this PR might resolve [KAFKA-8391](https://issues.apache.org/jira/browse/KAFKA-8391), [KAFKA-12283]

[GitHub] [kafka] showuon opened a new pull request, #12028: KAFKA-13804: output the reason why broker exit unexpectedly during startup

2022-04-11 Thread GitBox
showuon opened a new pull request, #12028: URL: https://github.com/apache/kafka/pull/12028 JIRA: [KAFKA-13804](https://issues.apache.org/jira/browse/KAFKA-13804) improve the log readability by adding the exception why broker exit during startup. ### Committer Checklist (exclude

[jira] [Updated] (KAFKA-13804) Log broker shutdown reason at the end of log output

2022-04-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-13804: -- Description: Currently, when the broker hit exception, or SIGINT signal received, we will first log t

[jira] [Updated] (KAFKA-13804) Log broker shutdown reason at the end of log output

2022-04-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-13804: -- Description: Currently, when the broker hit exception, or SIGINT signal received, we will first log t

[jira] [Commented] (KAFKA-13810) Document behavior of KafkaProducer.flush() w.r.t callbacks

2022-04-11 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520477#comment-17520477 ] Luke Chen commented on KAFKA-13810: --- Yes, after flush, all the requests are completed,

[GitHub] [kafka] dajac commented on pull request #12027: MINOR: Fix TestDowngrade.test_upgrade_and_downgrade

2022-04-11 Thread GitBox
dajac commented on PR #12027: URL: https://github.com/apache/kafka/pull/12027#issuecomment-1094821441 Got a successful run here: http://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/2022-04-11--001.system-test-kafka-branch-builder--1649665531--dajac--minor-fix

[jira] [Commented] (KAFKA-13810) Document behavior of KafkaProducer.flush() w.r.t callbacks

2022-04-11 Thread Karsten Spang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520435#comment-17520435 ] Karsten Spang commented on KAFKA-13810: --- [~showuon] I assume that all callbacks ar

[GitHub] [kafka] Gerrrr commented on pull request #11945: KAFKA-13769: Explicitly route FK join results to correct partitions

2022-04-11 Thread GitBox
Ge commented on PR #11945: URL: https://github.com/apache/kafka/pull/11945#issuecomment-1094706378 @mjsax Can you please take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] dajac opened a new pull request, #12027: MINOR: Fix TestDowngrade.test_upgrade_and_downgrade

2022-04-11 Thread GitBox
dajac opened a new pull request, #12027: URL: https://github.com/apache/kafka/pull/12027 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade