[GitHub] [kafka] chia7712 commented on a change in pull request #10096: KAFKA-12268: Make early poll return opt-in

2021-02-09 Thread GitBox
chia7712 commented on a change in pull request #10096: URL: https://github.com/apache/kafka/pull/10096#discussion_r573463351 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java ## @@ -2480,4 +2508,38 @@ String getClientId() { boolean u

[GitHub] [kafka] chia7712 commented on a change in pull request #10096: KAFKA-12268: Make early poll return opt-in

2021-02-09 Thread GitBox
chia7712 commented on a change in pull request #10096: URL: https://github.com/apache/kafka/pull/10096#discussion_r573459714 ## File path: streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java ## @@ -1201,6 +1201,9 @@ private void verifyMaxInFlightRequestPerConnecti

[GitHub] [kafka] ewencp commented on pull request #10056: KAFKA-12293: remove JCenter and Bintray repositories mentions out of build gradle scripts (sunset is announced for those repositories)

2021-02-09 Thread GitBox
ewencp commented on pull request #10056: URL: https://github.com/apache/kafka/pull/10056#issuecomment-776454131 I actually don't watch this much anymore, but what is the replacement? References were added intentionally because some dependencies weren't available in maven central. Has every

[GitHub] [kafka] vvcephei commented on pull request #10096: KAFKA-12268: Make early poll return opt-in

2021-02-09 Thread GitBox
vvcephei commented on pull request #10096: URL: https://github.com/apache/kafka/pull/10096#issuecomment-776446769 Hey @rajinisivaram and @chia7712 , Here is my proposed fix for https://issues.apache.org/jira/browse/KAFKA-12268 , based on our discussion in https://github.com/apache/k

[GitHub] [kafka] vvcephei opened a new pull request #10096: KAFKA-12268: Make early poll return opt-in

2021-02-09 Thread GitBox
vvcephei opened a new pull request #10096: URL: https://github.com/apache/kafka/pull/10096 * Revert the default Consumer#poll behavior back to early return on records only * Add config to enable early return on metadata or records * Set the return-on-metadata config in Streams to supp

[jira] [Commented] (KAFKA-12314) Leverage custom comparator for optimized range scans on RocksDB

2021-02-09 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282215#comment-17282215 ] Guozhang Wang commented on KAFKA-12314: --- cc [~vcrfxia] a PR from 3 years ago on us

[GitHub] [kafka] yeralin commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-02-09 Thread GitBox
yeralin commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-776422927 Any updates on this? :) This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [kafka] vcrfxia commented on pull request #10091: KAFKA-9524: increase retention time for window and grace periods longer than one day

2021-02-09 Thread GitBox
vcrfxia commented on pull request #10091: URL: https://github.com/apache/kafka/pull/10091#issuecomment-776412900 Rather than having the `grace(...)` method update `maintainDurationMs`, would it be better to update `maintainMs()` to return `Math.max(maintainDurationMs, sizeMs + graceMs)` ra

[GitHub] [kafka] chia7712 commented on a change in pull request #10080: MINOR: KafkaBroker.brokerState should be volatile instead of AtomicReference

2021-02-09 Thread GitBox
chia7712 commented on a change in pull request #10080: URL: https://github.com/apache/kafka/pull/10080#discussion_r573418590 ## File path: core/src/main/scala/kafka/server/KafkaBroker.scala ## @@ -90,7 +92,7 @@ trait KafkaBroker extends KafkaMetricsGroup { explicitMetricNa

[GitHub] [kafka] chia7712 merged pull request #10035: MINOR: Remove unused parameters in functions.

2021-02-09 Thread GitBox
chia7712 merged pull request #10035: URL: https://github.com/apache/kafka/pull/10035 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-12314) Leverage custom comparator for optimized range scans on RocksDB

2021-02-09 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282165#comment-17282165 ] A. Sophie Blee-Goldman commented on KAFKA-12314: Hey Sagar, one thing I

[jira] [Commented] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2021-02-09 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282162#comment-17282162 ] A. Sophie Blee-Goldman commented on KAFKA-5146: --- I agree, let's definitely

[jira] [Commented] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

2021-02-09 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282158#comment-17282158 ] Randall Hauch commented on KAFKA-10021: --- Including in 2.6.2 due to reports of reba

[jira] [Updated] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

2021-02-09 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-10021: -- Fix Version/s: (was: 2.6.3) 2.6.2 > When reading to the end of the conf

[jira] [Updated] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

2021-02-09 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-10021: -- Fix Version/s: (was: 2.6.2) > When reading to the end of the config log, check if fetch.ma

[jira] [Updated] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

2021-02-09 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-10021: -- Fix Version/s: 2.6.3 > When reading to the end of the config log, check if fetch.max.wait.ms i

[jira] [Updated] (KAFKA-10021) When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

2021-02-09 Thread Randall Hauch (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-10021: -- Fix Version/s: 2.6.2 2.7.1 2.8.0 2.5.2

[GitHub] [kafka] cmccabe opened a new pull request #10095: MINOR: Introduce the KIP-500 Broker lifecycle manager

2021-02-09 Thread GitBox
cmccabe opened a new pull request #10095: URL: https://github.com/apache/kafka/pull/10095 The broker lifecycle manager handles registering the broker and sending periodic heartbeats, as described in KIP-631. Based on the responses it receives from the controller, it transitions the broker

[jira] [Updated] (KAFKA-12319) Flaky test ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit()

2021-02-09 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-12319: --- Description: I've seen this test fail a few times locally. But recently I saw it fail on a

[GitHub] [kafka] jolshan commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776344485 @junrao On the previous commit JDK8 built and only MirrorConnectorsIntegrationSSLTest.testOneWayReplicationWithAutoOffsetSync() failed. (The only difference between this commit

[GitHub] [kafka] cmccabe commented on a change in pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10093: URL: https://github.com/apache/kafka/pull/10093#discussion_r573350073 ## File path: tests/kafkatest/services/kafka/kafka.py ## @@ -171,35 +347,79 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfi

[GitHub] [kafka] cmccabe commented on a change in pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10093: URL: https://github.com/apache/kafka/pull/10093#discussion_r573348782 ## File path: config/raft-controller.properties ## @@ -0,0 +1,124 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] [kafka] cmccabe commented on a change in pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10093: URL: https://github.com/apache/kafka/pull/10093#discussion_r573348737 ## File path: config/raft-controller.properties ## @@ -0,0 +1,124 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor

[GitHub] [kafka] cmccabe commented on a change in pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10093: URL: https://github.com/apache/kafka/pull/10093#discussion_r573348327 ## File path: config/raft-broker.properties ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lice

[GitHub] [kafka] cmccabe commented on a change in pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10093: URL: https://github.com/apache/kafka/pull/10093#discussion_r573348107 ## File path: config/raft-broker.properties ## @@ -0,0 +1,125 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lice

[GitHub] [kafka] junrao commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
junrao commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776328567 @jolshan : Thanks. Were the JDK 8 tests ok too? This is an automated message from the Apache Git Service. To resp

[GitHub] [kafka] jolshan commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776322170 Looks like one was already created. https://issues.apache.org/jira/browse/KAFKA-12284 https://issues.apache.org/jira/browse/KAFKA-12319 ---

[jira] [Created] (KAFKA-12319) Flaky test ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit()

2021-02-09 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-12319: -- Summary: Flaky test ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit() Key: KAFKA-12319 URL: https://issues.apache.org/jira/browse/KAFKA-12319

[GitHub] [kafka] cmccabe commented on pull request #10069: MINOR: Add RaftReplicaManager

2021-02-09 Thread GitBox
cmccabe commented on pull request #10069: URL: https://github.com/apache/kafka/pull/10069#issuecomment-776317812 Thanks @rondagostino . I left two small comments... LGTM after those are addressed. This is an automated messa

[GitHub] [kafka] cmccabe commented on a change in pull request #10069: MINOR: Add RaftReplicaManager

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10069: URL: https://github.com/apache/kafka/pull/10069#discussion_r573328630 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -185,7 +185,9 @@ object HostedPartition { * This state only applies to broke

[GitHub] [kafka] cmccabe commented on a change in pull request #10069: MINOR: Add RaftReplicaManager

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10069: URL: https://github.com/apache/kafka/pull/10069#discussion_r573326890 ## File path: core/src/main/scala/kafka/server/RaftReplicaManager.scala ## @@ -0,0 +1,397 @@ +/** + * Licensed to the Apache Software Foundation (ASF) und

[GitHub] [kafka] cmccabe opened a new pull request #10094: MINOR: Add the KIP-500 metadata shell

2021-02-09 Thread GitBox
cmccabe opened a new pull request #10094: URL: https://github.com/apache/kafka/pull/10094 The Kafka Metadata shell is a new command which allows users to interactively examine the metadata stored in a KIP-500 cluster. It can read the metadata from the controllers directly, by conne

[GitHub] [kafka] jolshan commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776313326 I had 2 JDK 11 failures previously. I also noticed JDK failures on other PRs. I'll check JIRA for these issues -

[GitHub] [kafka] rondagostino opened a new pull request #10093: MINOR: Support Raft-based metadata quorums in system tests

2021-02-09 Thread GitBox
rondagostino opened a new pull request #10093: URL: https://github.com/apache/kafka/pull/10093 We need to be able to run system tests with Raft-based metadata quorums -- both co-located brokers and controllers as well as remote controllers -- in addition to the ZooKepeer-based mode we run

[GitHub] [kafka] rohitrmd commented on pull request #9744: KAFKA-10062: Add a method to retrieve the current timestamp as known by the Streams app

2021-02-09 Thread GitBox
rohitrmd commented on pull request #9744: URL: https://github.com/apache/kafka/pull/9744#issuecomment-776306678 @mjsax 2 1. tests failing are not related to my changes. not sure how to fix them. 2. Are there kip updates i need to do? ---

[GitHub] [kafka] junrao commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
junrao commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776303090 @jolshan : Thanks. For those transient failures, could you file a jira if it's not tracked already? Are the JDK 8 failures also transient? ---

[jira] [Commented] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2021-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17282099#comment-17282099 ] Matthias J. Sax commented on KAFKA-5146: Any comments from [~guozhang] [~vvcephei

[jira] [Updated] (KAFKA-12314) Leverage custom comparator for optimized range scans on RocksDB

2021-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12314: Component/s: streams > Leverage custom comparator for optimized range scans on RocksDB > -

[jira] [Updated] (KAFKA-12313) Consider deprecating the default.windowed.serde.inner.class configs

2021-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12313: Component/s: streams > Consider deprecating the default.windowed.serde.inner.class configs

[jira] [Updated] (KAFKA-12313) Consider deprecating the default.windowed.serde.inner.class configs

2021-02-09 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12313: Labels: needs-kip (was: ) > Consider deprecating the default.windowed.serde.inner.class c

[GitHub] [kafka] jolshan edited a comment on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan edited a comment on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776291259 `MirrorConnectorsIntegrationSSLTest.testOneWayReplicationWithAutoOffsetSync()` seems to be flaky (is failing on other open PRs) It passed when I ran locally. `Connect

[GitHub] [kafka] jolshan commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776291259 `MirrorConnectorsIntegrationSSLTest.testOneWayReplicationWithAutoOffsetSync()` seems to be flaky (is failing on other open PRs) `ConnectionQuotasTest.testListenerConnection

[GitHub] [kafka] dejan2609 commented on pull request #10056: KAFKA-12293: remove JCenter and Bintray repositories mentions out of build gradle scripts (sunset is announced for those repositories)

2021-02-09 Thread GitBox
dejan2609 commented on pull request #10056: URL: https://github.com/apache/kafka/pull/10056#issuecomment-776291112 (note: I just force-pushed once more in order to remove phrase `[Work in progress]` out of the git commit message... **_now_** PR is ready ready for a review... for real 😃)

[GitHub] [kafka] dejan2609 commented on pull request #10056: KAFKA-12293: remove JCenter and Bintray repositories mentions out of build gradle scripts (sunset is announced for those repositories)

2021-02-09 Thread GitBox
dejan2609 commented on pull request #10056: URL: https://github.com/apache/kafka/pull/10056#issuecomment-776288230 This PR is ready for a review. FYI @ewencp and @ijuma This is an automated message from the Apache Git Ser

[jira] [Assigned] (KAFKA-12318) system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

2021-02-09 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-12318: -- Assignee: Justine Olshan > system tests need to fetch Topic IDs via Admin Client inst

[jira] [Created] (KAFKA-12318) system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper

2021-02-09 Thread Ron Dagostino (Jira)
Ron Dagostino created KAFKA-12318: - Summary: system tests need to fetch Topic IDs via Admin Client instead of via ZooKeeper Key: KAFKA-12318 URL: https://issues.apache.org/jira/browse/KAFKA-12318 Proj

[GitHub] [kafka] jolshan commented on pull request #10041: MINOR: Prevent creating partition.metadata until ID can be written

2021-02-09 Thread GitBox
jolshan commented on pull request #10041: URL: https://github.com/apache/kafka/pull/10041#issuecomment-776280666 Yes. I will want this on the 2.8 branch. I'll check the failed tests. I've also had trouble with building at least one of the three JDKs, but it seems like it is not the same on

[GitHub] [kafka] MarcoLotz commented on a change in pull request #10042: KAFKA-9527: fix NPE when using time-based argument for Stream Resetter

2021-02-09 Thread GitBox
MarcoLotz commented on a change in pull request #10042: URL: https://github.com/apache/kafka/pull/10042#discussion_r573277614 ## File path: streams/src/test/java/org/apache/kafka/streams/tools/StreamsResetterTest.java ## @@ -247,6 +268,26 @@ public void shouldDetermineInterna

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573256853 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573256612 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573256321 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573254808 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] rondagostino commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
rondagostino commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573240204 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] rondagostino commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
rondagostino commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573239810 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] jeqo commented on a change in pull request #10042: KAFKA-9527: fix NPE when using time-based argument for Stream Resetter

2021-02-09 Thread GitBox
jeqo commented on a change in pull request #10042: URL: https://github.com/apache/kafka/pull/10042#discussion_r573239173 ## File path: core/src/main/scala/kafka/tools/StreamsResetter.java ## @@ -503,7 +506,16 @@ private void resetToDatetime(final Consumer client, final

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573238990 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573235373 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573236986 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573236986 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573235373 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573235167 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573234880 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jeqo opened a new pull request #10092: KAFKA-12287: Add WARN logging on consumer-groups when reset-offsets by timestamp or duration can't find an offset and defaults to latest

2021-02-09 Thread GitBox
jeqo opened a new pull request #10092: URL: https://github.com/apache/kafka/pull/10092 Similar to #10042, this PR introduces a WARN logging when no offset is found in a topic partition. Test strategy: - Adding a test case for a topic without records. ### Committer Check

[GitHub] [kafka] cmccabe commented on a change in pull request #10084: MINOR: Rename DecommissionBrokers to UnregisterBrokers

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10084: URL: https://github.com/apache/kafka/pull/10084#discussion_r573229531 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -217,6 +217,7 @@ class KafkaApis(val requestChannel: RequestChannel, case Ap

[GitHub] [kafka] cmccabe commented on a change in pull request #10084: MINOR: Rename DecommissionBrokers to UnregisterBrokers

2021-02-09 Thread GitBox
cmccabe commented on a change in pull request #10084: URL: https://github.com/apache/kafka/pull/10084#discussion_r573229531 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -217,6 +217,7 @@ class KafkaApis(val requestChannel: RequestChannel, case Ap

[GitHub] [kafka] rondagostino commented on a change in pull request #10069: MINOR: Add RaftReplicaManager

2021-02-09 Thread GitBox
rondagostino commented on a change in pull request #10069: URL: https://github.com/apache/kafka/pull/10069#discussion_r573201310 ## File path: core/src/main/scala/kafka/server/metadata/MetadataPartitions.scala ## @@ -39,11 +41,30 @@ object MetadataPartition { record.isr(

[GitHub] [kafka] dajac commented on a change in pull request #9430: KAFKA-5235: GetOffsetShell: support for multiple topics and consumer configuration override

2021-02-09 Thread GitBox
dajac commented on a change in pull request #9430: URL: https://github.com/apache/kafka/pull/9430#discussion_r573201470 ## File path: core/src/main/scala/kafka/tools/GetOffsetShell.scala ## @@ -20,102 +20,131 @@ package kafka.tools import java.util.Properties import joptsim

[GitHub] [kafka] MarcoLotz opened a new pull request #10091: KAFKA-9524: increase retention time for window and grace periods longer than one day

2021-02-09 Thread GitBox
MarcoLotz opened a new pull request #10091: URL: https://github.com/apache/kafka/pull/10091 **Reproducing:** The bug can be easily reproduced for any TimeWindow where window + grace period > 1 day. Changing any test in TimeWindowedKStreamImplTest.java for this condition will reproduce t

[GitHub] [kafka] dajac commented on pull request #9430: KAFKA-5235: GetOffsetShell: support for multiple topics and consumer configuration override

2021-02-09 Thread GitBox
dajac commented on pull request #9430: URL: https://github.com/apache/kafka/pull/9430#issuecomment-776211759 Launched system tests: https://jenkins.confluent.io/job/system-test-kafka-branch-builder/4362/ This is an automated

[GitHub] [kafka] dajac commented on a change in pull request #9430: KAFKA-5235: GetOffsetShell: support for multiple topics and consumer configuration override

2021-02-09 Thread GitBox
dajac commented on a change in pull request #9430: URL: https://github.com/apache/kafka/pull/9430#discussion_r573192881 ## File path: core/src/main/scala/kafka/tools/GetOffsetShell.scala ## @@ -20,102 +20,120 @@ package kafka.tools import java.util.Properties import joptsim

[GitHub] [kafka] mumrah opened a new pull request #10090: Upstream BrokerMetadataListener and related classes

2021-02-09 Thread GitBox
mumrah opened a new pull request #10090: URL: https://github.com/apache/kafka/pull/10090 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Comment Edited] (KAFKA-9524) Default window retention does not consider grace period

2021-02-09 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281834#comment-17281834 ] Marco Lotz edited comment on KAFKA-9524 at 2/9/21, 7:30 PM: I

[jira] [Comment Edited] (KAFKA-9524) Default window retention does not consider grace period

2021-02-09 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281834#comment-17281834 ] Marco Lotz edited comment on KAFKA-9524 at 2/9/21, 7:29 PM: I

[jira] [Created] (KAFKA-12317) Relax non-null key requirement for left KStream-KTable join

2021-02-09 Thread Matthias J. Sax (Jira)
Matthias J. Sax created KAFKA-12317: --- Summary: Relax non-null key requirement for left KStream-KTable join Key: KAFKA-12317 URL: https://issues.apache.org/jira/browse/KAFKA-12317 Project: Kafka

[GitHub] [kafka] rondagostino commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-09 Thread GitBox
rondagostino commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r573093783 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,491 @@ +/* + * Licensed to the Apache Sof

[GitHub] [kafka] bob-barrett commented on pull request #10064: KAFKA-12193: Re-resolve IPs after a client disconnects

2021-02-09 Thread GitBox
bob-barrett commented on pull request #10064: URL: https://github.com/apache/kafka/pull/10064#issuecomment-776128103 retest this please This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [kafka] mumrah merged pull request #10087: Backport Jenkinsfile fix and remove Travis build for 2.5

2021-02-09 Thread GitBox
mumrah merged pull request #10087: URL: https://github.com/apache/kafka/pull/10087 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] mumrah merged pull request #10088: Backport Jenkinsfile fix and remove Travis build for 2.4

2021-02-09 Thread GitBox
mumrah merged pull request #10088: URL: https://github.com/apache/kafka/pull/10088 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] junrao commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-09 Thread GitBox
junrao commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r573102028 ## File path: core/src/main/scala/kafka/server/FetchSession.scala ## @@ -353,50 +508,68 @@ class SessionlessFetchContext(val fetchData: util.Map[TopicPartit

[GitHub] [kafka] gitlw opened a new pull request #10089: KAFKA-12315: Clearing the ZkReplicaStateMachine request batch state upon ControllerMovedException

2021-02-09 Thread GitBox
gitlw opened a new pull request #10089: URL: https://github.com/apache/kafka/pull/10089 This PR resolves the issue by clearing the state of the request batch upon ControllerMovedException. Also as a safety measure, it also clears the state during shutting down of the ReplicaStateMachine

[GitHub] [kafka] abbccdda commented on a change in pull request #10072: KAFKA-9274: Throw TaskCorruptedException instead of TimeoutException when TX commit times out

2021-02-09 Thread GitBox
abbccdda commented on a change in pull request #10072: URL: https://github.com/apache/kafka/pull/10072#discussion_r573065903 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java ## @@ -213,27 +214,26 @@ private void recordSe

[GitHub] [kafka] jolshan edited a comment on pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-09 Thread GitBox
jolshan edited a comment on pull request #9944: URL: https://github.com/apache/kafka/pull/9944#issuecomment-776097476 > > Also, it seems that for more partitions, the performance with this PR is noticeably worse than trunk? > > ``` > FetchRequestBenchmark.testSerializeFetchReq

[GitHub] [kafka] jolshan commented on pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-09 Thread GitBox
jolshan commented on pull request #9944: URL: https://github.com/apache/kafka/pull/9944#issuecomment-776097476 > > Also, it seems that for more partitions, the performance with this PR is noticeably worse than trunk? > > ``` > FetchRequestBenchmark.testSerializeFetchRequestFor

[GitHub] [kafka] rhauch merged pull request #9780: KAFKA-10021: Changed Kafka backing stores to use shared admin client to get end offsets and create topics

2021-02-09 Thread GitBox
rhauch merged pull request #9780: URL: https://github.com/apache/kafka/pull/9780 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] mumrah merged pull request #9986: JUnit extensions for integration tests

2021-02-09 Thread GitBox
mumrah merged pull request #9986: URL: https://github.com/apache/kafka/pull/9986 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] jolshan commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-09 Thread GitBox
jolshan commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r573046669 ## File path: core/src/main/scala/kafka/server/ReplicaFetcherThread.scala ## @@ -216,10 +217,10 @@ class ReplicaFetcherThread(name: String, try {

[GitHub] [kafka] jolshan commented on a change in pull request #9944: KAFKA-10580: Add topic ID support to Fetch request

2021-02-09 Thread GitBox
jolshan commented on a change in pull request #9944: URL: https://github.com/apache/kafka/pull/9944#discussion_r573033865 ## File path: core/src/main/scala/kafka/server/FetchSession.scala ## @@ -353,50 +508,68 @@ class SessionlessFetchContext(val fetchData: util.Map[TopicParti

[jira] [Commented] (KAFKA-9524) Default window retention does not consider grace period

2021-02-09 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281834#comment-17281834 ] Marco Lotz commented on KAFKA-9524: --- I have reproduced the bug on the Unit Tests and ca

[jira] [Assigned] (KAFKA-9524) Default window retention does not consider grace period

2021-02-09 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Lotz reassigned KAFKA-9524: - Assignee: Marco Lotz > Default window retention does not consider grace period > ---

[GitHub] [kafka] mumrah commented on a change in pull request #10063: KAFKA-12258: Add support for splitting appending records

2021-02-09 Thread GitBox
mumrah commented on a change in pull request #10063: URL: https://github.com/apache/kafka/pull/10063#discussion_r572957037 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -79,24 +80,69 @@ public BatchAccumulator( } /** -

[GitHub] [kafka] ijuma commented on a change in pull request #10084: MINOR: Rename DecommissionBrokers to UnregisterBrokers

2021-02-09 Thread GitBox
ijuma commented on a change in pull request #10084: URL: https://github.com/apache/kafka/pull/10084#discussion_r572957476 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/Admin.java ## @@ -1486,41 +1486,40 @@ default DescribeFeaturesResult describeFeatures() {

[jira] [Commented] (KAFKA-12312) kafka_2.13:2.6.1 throws NoSuchMethodError when running against scala-sdk-2.13.4

2021-02-09 Thread Tamas Kornai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281820#comment-17281820 ] Tamas Kornai commented on KAFKA-12312: -- Thanks [~showuon] for looking into this. >

[GitHub] [kafka] mumrah opened a new pull request #10088: Backport Jenkinsfile fix to 2.4

2021-02-09 Thread GitBox
mumrah opened a new pull request #10088: URL: https://github.com/apache/kafka/pull/10088 This is a backport of #9453 which changes the Jenkinsfile over to the new new tool names used by Apache's Jenkins. This is an automated

[GitHub] [kafka] mumrah opened a new pull request #10087: Backport Jenkinsfile fix to 2.5

2021-02-09 Thread GitBox
mumrah opened a new pull request #10087: URL: https://github.com/apache/kafka/pull/10087 This is a backport of #9453 which changes the Jenkinsfile over to the new new tool names used by Apache's Jenkins. This is an automate

[jira] [Commented] (KAFKA-5146) Kafka Streams: remove compile dependency on connect-json

2021-02-09 Thread Marco Lotz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281789#comment-17281789 ] Marco Lotz commented on KAFKA-5146: --- [~mjsax] I recall we had a similar problem when up

[jira] [Created] (KAFKA-12316) Configuration is not defined: topic.creation.default.partitions

2021-02-09 Thread Goltseva Taisiia (Jira)
Goltseva Taisiia created KAFKA-12316: Summary: Configuration is not defined: topic.creation.default.partitions Key: KAFKA-12316 URL: https://issues.apache.org/jira/browse/KAFKA-12316 Project: Kafk

[GitHub] [kafka] kamalcph commented on pull request #10035: MINOR: Remove unused parameters in functions.

2021-02-09 Thread GitBox
kamalcph commented on pull request #10035: URL: https://github.com/apache/kafka/pull/10035#issuecomment-775838914 @chia7712 Addressed your comment. Please take another look. This is an automated message from the Apache Gi

[GitHub] [kafka] dengziming commented on pull request #10031: KAFKA-12269: Replace partition metada with auto-generated protocal

2021-02-09 Thread GitBox
dengziming commented on pull request #10031: URL: https://github.com/apache/kafka/pull/10031#issuecomment-775793384 ping @hachikuji @ijuma @chia7712 to have a look. This is an automated message from the Apache Git Service. To

[jira] [Assigned] (KAFKA-9295) KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2021-02-09 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-9295: Assignee: Luke Chen > KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPart

[jira] [Commented] (KAFKA-9295) KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2021-02-09 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281644#comment-17281644 ] Luke Chen commented on KAFKA-9295: -- Investigating > KTableKTableForeignKeyInnerJoinMult

  1   2   >