[GitHub] [kafka] ijuma commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ijuma commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714467779 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding times

[jira] [Commented] (KAFKA-13070) LogManager shutdown races with periodic work scheduled by the instance

2021-09-22 Thread Cong Ding (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418918#comment-17418918 ] Cong Ding commented on KAFKA-13070: --- I created a PR for this: https://github.com/apach

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714439705 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding time

[GitHub] [kafka] ccding commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-09-22 Thread GitBox
ccding commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-925468033 @kowshik Thanks for the code review. Updated your email in the PR description as well as in the commit message. -- This is an automated message from the Apache Git Service. To

[jira] [Assigned] (KAFKA-13070) LogManager shutdown races with periodic work scheduled by the instance

2021-09-22 Thread Cong Ding (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cong Ding reassigned KAFKA-13070: - Assignee: Cong Ding (was: Manasvi Gupta) > LogManager shutdown races with periodic work schedu

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714410283 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -103,11 +103,14 @@ class KafkaScheduler(val threads: Int, schedule(name, fun,

[GitHub] [kafka] kowshik commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
kowshik commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714401442 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -103,11 +103,14 @@ class KafkaScheduler(val threads: Int, schedule(name, fun

[jira] [Created] (KAFKA-13319) Do not send AddOffsetsToTxn/TxnOffsetCommit if offsets map is empty

2021-09-22 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-13319: --- Summary: Do not send AddOffsetsToTxn/TxnOffsetCommit if offsets map is empty Key: KAFKA-13319 URL: https://issues.apache.org/jira/browse/KAFKA-13319 Project: Ka

[GitHub] [kafka] ijuma commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ijuma commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714391884 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding times

[GitHub] [kafka] hachikuji opened a new pull request #11354: MINOR: Print lastTimestamp when dumping producer snapshots

2021-09-22 Thread GitBox
hachikuji opened a new pull request #11354: URL: https://github.com/apache/kafka/pull/11354 The `LastTimestamp` field is useful because its value is present even when there are no data batches written by a given producerId. ### Committer Checklist (excluded from commit message) -

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714377435 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding time

[GitHub] [kafka] ijuma commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ijuma commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714375625 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding times

[jira] [Commented] (KAFKA-13255) Mirrormaker config property config.properties.exclude is not working as expected

2021-09-22 Thread Anamika Nadkarni (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418867#comment-17418867 ] Anamika Nadkarni commented on KAFKA-13255: -- [~ryannedolan] How much time usuall

[GitHub] [kafka] ccding commented on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-925334750 I pushed twice with the two commits. It appears Jenkins is currently running on the first commit, and the second one is pending. Unfortunately, I don't have permission to stop it

[GitHub] [kafka] junrao commented on a change in pull request #11327: KAFKA-13305: fix NullPointerException in LogCleanerManager "uncleanable-bytes" gauge

2021-09-22 Thread GitBox
junrao commented on a change in pull request #11327: URL: https://github.com/apache/kafka/pull/11327#discussion_r714305801 ## File path: core/src/main/scala/kafka/log/LogCleanerManager.scala ## @@ -512,6 +514,27 @@ private[log] class LogCleanerManager(val logDirs: Seq[File],

[GitHub] [kafka] jolshan commented on pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on pull request #11170: URL: https://github.com/apache/kafka/pull/11170#issuecomment-925320746 I was thinking a bit about this and the downgrade path and I realized that we never remove the topic ID from the state. I think this is ok because a downgrade will no longer use

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714300664 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala ## @@ -929,6 +929,29 @@ class AbstractFetcherThreadTest { fetcher

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714299871 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherThreadTest.scala ## @@ -929,6 +929,29 @@ class AbstractFetcherThreadTest { fetcher

[GitHub] [kafka] junrao commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
junrao commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714283329 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -53,7 +53,7 @@ trait Scheduler { * @param unit The unit for the preceding time

[GitHub] [kafka] cmccabe merged pull request #11310: KAFKA-13279: KRaft: add CreateTopicsPolicy + AlterConfigsPolicy

2021-09-22 Thread GitBox
cmccabe merged pull request #11310: URL: https://github.com/apache/kafka/pull/11310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] stephan-mohn edited a comment on pull request #11254: KAFKA-2424: Introduce Scalafix linter

2021-09-22 Thread GitBox
stephan-mohn edited a comment on pull request #11254: URL: https://github.com/apache/kafka/pull/11254#issuecomment-925266540 Aiven Inc - https://aiven.io/ is misusing of Apache Kafka trademark as 'Aiven Kafka' [1] and violates Apache trademarks. See - [1] https://help.aiven.io/en/?q=%22aiv

[GitHub] [kafka] stephan-mohn edited a comment on pull request #11254: KAFKA-2424: Introduce Scalafix linter

2021-09-22 Thread GitBox
stephan-mohn edited a comment on pull request #11254: URL: https://github.com/apache/kafka/pull/11254#issuecomment-925266540 Aiven Inc - https://aiven.io/ is misusing of Apache Kafka trademark as 'Aiven Kafka' [1] and violates Apache trademarks. See - [1] https://help.aiven.io/en/?q=%22aiv

[GitHub] [kafka] stephan-mohn edited a comment on pull request #11254: KAFKA-2424: Introduce Scalafix linter

2021-09-22 Thread GitBox
stephan-mohn edited a comment on pull request #11254: URL: https://github.com/apache/kafka/pull/11254#issuecomment-925266540 Aiven Inc - https://aiven.io/ is misusing of Apache Kafka trademark as 'Aiven Kafka' [1] and violates Apache trademarks. See - [1] https://help.aiven.io/en/?q=%22aiv

[GitHub] [kafka] stephan-mohn commented on pull request #11254: KAFKA-2424: Introduce Scalafix linter

2021-09-22 Thread GitBox
stephan-mohn commented on pull request #11254: URL: https://github.com/apache/kafka/pull/11254#issuecomment-925266540 Aiven Inc - https://aiven.io/ is misusing of Apache Kafka trademark as 'Aiven Kafka' [1] and violates Apache trademarks. See - [1] https://help.aiven.io/en/?q=%22aiven+kafk

[GitHub] [kafka] vvcephei merged pull request #11316: KAFKA-10544: Migrate KTable aggregate and reduce

2021-09-22 Thread GitBox
vvcephei merged pull request #11316: URL: https://github.com/apache/kafka/pull/11316 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] vvcephei commented on pull request #11316: KAFKA-10544: Migrate KTable aggregate and reduce

2021-09-22 Thread GitBox
vvcephei commented on pull request #11316: URL: https://github.com/apache/kafka/pull/11316#issuecomment-925249710 I just merged in trunk and re-ran `./gradlew clean :streams:testAll` and it passed. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [kafka] vvcephei commented on pull request #11336: DRAFT: Add currentPosition and times to ProcessorContext

2021-09-22 Thread GitBox
vvcephei commented on pull request #11336: URL: https://github.com/apache/kafka/pull/11336#issuecomment-925238816 Thanks for taking a look @guozhangwang and @mjsax ! You're both right about the missing time methods. I'll just send an update to that KIP and send a separate PR. -- T

[GitHub] [kafka] vvcephei commented on a change in pull request #11336: DRAFT: Add currentPosition and times to ProcessorContext

2021-09-22 Thread GitBox
vvcephei commented on a change in pull request #11336: URL: https://github.com/apache/kafka/pull/11336#discussion_r714237174 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/GlobalProcessorContextImpl.java ## @@ -110,6 +112,11 @@ public long cur

[GitHub] [kafka] jeqo commented on a change in pull request #11315: KAFKA-10540: Migrate KStream aggregate operations

2021-09-22 Thread GitBox
jeqo commented on a change in pull request #11315: URL: https://github.com/apache/kafka/pull/11315#discussion_r714236064 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregate.java ## @@ -74,92 +79,118 @@ public void enable

[GitHub] [kafka] vvcephei commented on a change in pull request #11336: DRAFT: Add currentPosition and times to ProcessorContext

2021-09-22 Thread GitBox
vvcephei commented on a change in pull request #11336: URL: https://github.com/apache/kafka/pull/11336#discussion_r714232901 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/api/ProcessorContext.java ## @@ -247,4 +251,45 @@ Cancellable schedule(final Dura

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714226694 ## File path: core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala ## @@ -2810,4 +2811,132 @@ class ReplicaManagerTest { Replicas.NON

[GitHub] [kafka] ccding commented on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-925205714 Addressed all comments above. PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] vvcephei commented on a change in pull request #11315: KAFKA-10540: Migrate KStream aggregate operations

2021-09-22 Thread GitBox
vvcephei commented on a change in pull request #11315: URL: https://github.com/apache/kafka/pull/11315#discussion_r714207540 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamAggregate.java ## @@ -77,43 +83,51 @@ public void init(final org.

[GitHub] [kafka] ccding edited a comment on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding edited a comment on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-925201730 @junrao for the side effect, I am saying ``` sortedNewSegments.foreach(_.changeFileSuffixes(SwapFileSuffix, "")) ``` will execute. #10763 mainly targets the cas

[GitHub] [kafka] ccding commented on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-925201730 @junrao for the side effect, I am mainly saying ``` sortedNewSegments.foreach(_.changeFileSuffixes(SwapFileSuffix, "")) ``` will execute. #10763 mainly targets the cas

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714216176 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -499,12 +499,13 @@ class BrokerServer( if (clientToControllerChannelManager

[GitHub] [kafka] junrao commented on pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
junrao commented on pull request #11351: URL: https://github.com/apache/kafka/pull/11351#issuecomment-925172265 @ccding : For the side effect that you mentioned, a segment to be renamed to .delete, but won't be physically deleted. This seems fine since on broker restart, we have the logic

[GitHub] [kafka] junrao commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
junrao commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714186270 ## File path: core/src/main/scala/kafka/utils/KafkaScheduler.scala ## @@ -107,7 +107,10 @@ class KafkaScheduler(val threads: Int, debug("Scheduling ta

[GitHub] [kafka] junrao commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
junrao commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714188175 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -482,6 +482,20 @@ class BrokerServer( } metadataSnapshotter.foreach(s

[GitHub] [kafka] junrao commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
junrao commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714188175 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -482,6 +482,20 @@ class BrokerServer( } metadataSnapshotter.foreach(s

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714195204 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -491,6 +491,21 @@ abstract class AbstractFetcherThread(name: String,

[GitHub] [kafka] ccding closed pull request #11270: MINOR: print debug log for InvalidRequestException

2021-09-22 Thread GitBox
ccding closed pull request #11270: URL: https://github.com/apache/kafka/pull/11270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] ccding closed pull request #11269: MINOR: rename variable sTime to currentTimeMs to follow convention in the codebase

2021-09-22 Thread GitBox
ccding closed pull request #11269: URL: https://github.com/apache/kafka/pull/11269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714153185 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherManagerTest.scala ## @@ -133,4 +137,75 @@ class AbstractFetcherManagerTest { asse

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714152192 ## File path: core/src/test/scala/unit/kafka/server/AbstractFetcherManagerTest.scala ## @@ -133,4 +137,75 @@ class AbstractFetcherManagerTest { asse

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714140053 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1739,6 +1758,37 @@ class ReplicaManager(val config: KafkaConfig, partition

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714134701 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1424,6 +1428,21 @@ class ReplicaManager(val config: KafkaConfig, val

[GitHub] [kafka] jolshan commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
jolshan commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r714133543 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -491,6 +491,21 @@ abstract class AbstractFetcherThread(name: String,

[jira] [Comment Edited] (KAFKA-13239) Use RocksDB.ingestExternalFile for restoration

2021-09-22 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418705#comment-17418705 ] Guozhang Wang edited comment on KAFKA-13239 at 9/22/21, 4:39 PM: -

[GitHub] [kafka] ccding commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714121176 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -499,12 +499,13 @@ class BrokerServer( if (clientToControllerChannelManager

[jira] [Commented] (KAFKA-13239) Use RocksDB.ingestExternalFile for restoration

2021-09-22 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418705#comment-17418705 ] Guozhang Wang commented on KAFKA-13239: --- {quote} What does this mean/how do memtab

[GitHub] [kafka] cmccabe commented on pull request #11311: KAFKA-13280: Avoid O(N) behavior in KRaftMetadataCache#topicNamesToIds

2021-09-22 Thread GitBox
cmccabe commented on pull request #11311: URL: https://github.com/apache/kafka/pull/11311#issuecomment-925086878 I filed https://issues.apache.org/jira/browse/KAFKA-13318 to create JMH benchmarks for this and other cache operations. For now, I think we should commit this to fix the O(N) be

[jira] [Created] (KAFKA-13318) Add jmh benchmarks for MetadataCache#topicNamesToIds

2021-09-22 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-13318: Summary: Add jmh benchmarks for MetadataCache#topicNamesToIds Key: KAFKA-13318 URL: https://issues.apache.org/jira/browse/KAFKA-13318 Project: Kafka Issue Ty

[GitHub] [kafka] guozhangwang commented on pull request #11227: KAFKA-13211: add support for infinite range query for WindowStore

2021-09-22 Thread GitBox
guozhangwang commented on pull request #11227: URL: https://github.com/apache/kafka/pull/11227#issuecomment-925079657 LGTM! Merged to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [kafka] guozhangwang merged pull request #11227: KAFKA-13211: add support for infinite range query for WindowStore

2021-09-22 Thread GitBox
guozhangwang merged pull request #11227: URL: https://github.com/apache/kafka/pull/11227 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsu

[GitHub] [kafka] ijuma commented on a change in pull request #11351: KAFKA-13315: log layer exception during shutdown that caused an unclean shutdown

2021-09-22 Thread GitBox
ijuma commented on a change in pull request #11351: URL: https://github.com/apache/kafka/pull/11351#discussion_r714082956 ## File path: core/src/main/scala/kafka/server/BrokerServer.scala ## @@ -499,12 +499,13 @@ class BrokerServer( if (clientToControllerChannelManager !

[GitHub] [kafka] ccding commented on a change in pull request #11058: KAFKA-12802 Added a file based cache for consumed remote log metadata for each partition to avoid consuming again incase of broker

2021-09-22 Thread GitBox
ccding commented on a change in pull request #11058: URL: https://github.com/apache/kafka/pull/11058#discussion_r708590530 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/CommittedOffsetsFile.java ## @@ -0,0 +1,80 @@ +/* + * Licensed to

[jira] [Created] (KAFKA-13317) Kafka brokers can't communicate with each other, reports Error sending fetch request , java.io.IOException: Connection to 2 was disconnected before the response was read

2021-09-22 Thread priya Vijay (Jira)
priya Vijay created KAFKA-13317: --- Summary: Kafka brokers can't communicate with each other, reports Error sending fetch request , java.io.IOException: Connection to 2 was disconnected before the response was read Key: KAFKA-13317

[GitHub] [kafka] mprusakov opened a new pull request #11353: Reducing amount of garbage that gets generated during a poll operation

2021-09-22 Thread GitBox
mprusakov opened a new pull request #11353: URL: https://github.com/apache/kafka/pull/11353 Presently KafkaConsumer creates an enourmous amount of garbage during a poll. Polls are generaly very frequent and so allocations during a poll decrease efficiency of the overall solution. This comm

[GitHub] [kafka] vijaykriishna removed a comment on pull request #10873: KAFKA-7360 Fixed code snippet

2021-09-22 Thread GitBox
vijaykriishna removed a comment on pull request #10873: URL: https://github.com/apache/kafka/pull/10873#issuecomment-922391928 @junrao Please review the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [kafka] vijaykriishna commented on pull request #10873: KAFKA-7360 Fixed code snippet

2021-09-22 Thread GitBox
vijaykriishna commented on pull request #10873: URL: https://github.com/apache/kafka/pull/10873#issuecomment-924982174 @junrao Please review the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] ccding commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-09-22 Thread GitBox
ccding commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-924981747 Updated the Test section -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [kafka] vijaykriishna edited a comment on pull request #10873: KAFKA-7360 Fixed code snippet

2021-09-22 Thread GitBox
vijaykriishna edited a comment on pull request #10873: URL: https://github.com/apache/kafka/pull/10873#issuecomment-922391928 @junrao Please review the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] scholzj opened a new pull request #11352: Add DirectoryConfigProvider to the service provider list

2021-09-22 Thread GitBox
scholzj opened a new pull request #11352: URL: https://github.com/apache/kafka/pull/11352 Following the discussion in confluentinc/kafka-images#102, it seems that the `DirectoryConfigProvider` should be also listed in the list of the service providers. CC: @C0urante ### Commi

[jira] [Created] (KAFKA-13316) Convert CreateTopicsRequestWithPolicyTest to use ClusterTest

2021-09-22 Thread dengziming (Jira)
dengziming created KAFKA-13316: -- Summary: Convert CreateTopicsRequestWithPolicyTest to use ClusterTest Key: KAFKA-13316 URL: https://issues.apache.org/jira/browse/KAFKA-13316 Project: Kafka Iss

[jira] [Updated] (KAFKA-13293) Support client reload of JKS/PEM certificates

2021-09-22 Thread Elliot West (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot West updated KAFKA-13293: Description: Producer/Consumer clients do not currently automatically reload certificates when th

[jira] [Updated] (KAFKA-13293) Support client reload of JKS/PEM certificates

2021-09-22 Thread Elliot West (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliot West updated KAFKA-13293: Summary: Support client reload of JKS/PEM certificates (was: Support client reload of PEM certifi

[GitHub] [kafka] dajac commented on a change in pull request #11170: KAFKA-13102: Topic IDs not propagated to metadata cache quickly enough for Fetch path

2021-09-22 Thread GitBox
dajac commented on a change in pull request #11170: URL: https://github.com/apache/kafka/pull/11170#discussion_r713916798 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1424,6 +1428,21 @@ class ReplicaManager(val config: KafkaConfig, val f

[jira] [Commented] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read.

2021-09-22 Thread Amar Bogari (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418584#comment-17418584 ] Amar Bogari commented on KAFKA-7870: We are running kafka 2.4.0 and see similar issue

[GitHub] [kafka] ijuma commented on pull request #11345: Allow empty last segment to have missing offset index during recovery

2021-09-22 Thread GitBox
ijuma commented on pull request #11345: URL: https://github.com/apache/kafka/pull/11345#issuecomment-924899752 @ccding the PR does have unit tests though, so we should update that part of the PR message. -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] dengziming commented on pull request #11339: MINOR: ClientQuotaRecord is not handled in KRaft MetadataShell

2021-09-22 Thread GitBox
dengziming commented on pull request #11339: URL: https://github.com/apache/kafka/pull/11339#issuecomment-924890352 Hello, @cmccabe @mumrah PTAL. thank you 🤝. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713800603 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -505,7 +497,7 @@ public boolean poll(Tim

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713800603 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -505,7 +497,7 @@ public boolean poll(Tim

[jira] [Resolved] (KAFKA-13302) [IEP-59] Support not default page size

2021-09-22 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov resolved KAFKA-13302. - Resolution: Invalid Sorry for any inconvinience. This issue should go to Ignite project.

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713843300 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713843300 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713843300 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-924807334 @showuon The latest code changes have been committed, if you have time, please review it. Thanks. -- This is an automated message from the Apache Git Service. To resp

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713805122 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713805122 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-924798121 Thanks for your review @showuon I will re-commit code later. Hope you can continue to give advice -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713805122 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -670,10 +662,11 @@ private void validate

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713800603 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -505,7 +497,7 @@ public boolean poll(Tim

[GitHub] [kafka] RivenSun2 commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
RivenSun2 commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713793616 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -66,15 +66,7 @@ import org.slf4j.Logger

[GitHub] [kafka] showuon commented on a change in pull request #11340: fix issue : KafkaConsumer cannot jump out of the poll method, and the…

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11340: URL: https://github.com/apache/kafka/pull/11340#discussion_r713737164 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java ## @@ -66,15 +66,7 @@ import org.slf4j.Logger;

[GitHub] [kafka] showuon commented on a change in pull request #11347: KAFKA-13296: warn if previous assignment has duplicate partitions

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11347: URL: https://github.com/apache/kafka/pull/11347#discussion_r713725768 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -351,6 +351,17 @@ public GroupAssign

[GitHub] [kafka] showuon commented on a change in pull request #11347: KAFKA-13296: warn if previous assignment has duplicate partitions

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11347: URL: https://github.com/apache/kafka/pull/11347#discussion_r713720965 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -351,6 +351,17 @@ public GroupAssign

[GitHub] [kafka] showuon commented on a change in pull request #11347: KAFKA-13296: warn if previous assignment has duplicate partitions

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11347: URL: https://github.com/apache/kafka/pull/11347#discussion_r713720965 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -351,6 +351,17 @@ public GroupAssign

[GitHub] [kafka] showuon commented on a change in pull request #11227: KAFKA-13211: add support for infinite range query for WindowStore

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11227: URL: https://github.com/apache/kafka/pull/11227#discussion_r713675956 ## File path: streams/src/main/java/org/apache/kafka/streams/state/internals/InMemoryWindowStore.java ## @@ -462,14 +460,34 @@ public boolean hasNext() {

[GitHub] [kafka] showuon commented on a change in pull request #11227: KAFKA-13211: add support for infinite range query for WindowStore

2021-09-22 Thread GitBox
showuon commented on a change in pull request #11227: URL: https://github.com/apache/kafka/pull/11227#discussion_r713675491 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/RangeQueryForWindowStoreIntegrationTest.java ## @@ -0,0 +1,338 @@ +/* + * Licens