[GitHub] [kafka] chia7712 commented on pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
chia7712 commented on pull request #10113: URL: https://github.com/apache/kafka/pull/10113#issuecomment-781127384 There is a build error related to this PR. please take a look at #10146 This is an automated message from the

[GitHub] [kafka] chia7712 opened a new pull request #10146: HOTFIX: fix build error caused by ControllerApisTest.scala

2021-02-17 Thread GitBox
chia7712 opened a new pull request #10146: URL: https://github.com/apache/kafka/pull/10146 related to #10113 ### Error Message ` 14:01:06 > Task :core:compileTestScala 14:01:06 [Error]

[jira] [Commented] (KAFKA-12333) KafkaMetadataLog and MockLock should validate that appended epochs are monotonically

2021-02-17 Thread dengziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286305#comment-17286305 ] dengziming commented on KAFKA-12333: Should this validate in `Log.append` rather than `

[GitHub] [kafka] dengziming commented on a change in pull request #10021: KAFKA-12205: Delete snapshots less than the snapshot at the log start

2021-02-17 Thread GitBox
dengziming commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r578168685 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -267,30 +259,45 @@ final class KafkaMetadataLog private ( } } -

[GitHub] [kafka] cmccabe merged pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe merged pull request #10113: URL: https://github.com/apache/kafka/pull/10113 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] cmccabe commented on pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on pull request #10113: URL: https://github.com/apache/kafka/pull/10113#issuecomment-781065399 Thanks, @rondagostino ! The only test failures I can find are flaky streams tests that are not related. LGTM, will commit

[jira] [Commented] (KAFKA-10232) MirrorMaker2 internal topics Formatters

2021-02-17 Thread James Cheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286270#comment-17286270 ] James Cheng commented on KAFKA-10232: - This was implemented in 

[GitHub] [kafka] kamalcph commented on pull request #10126: MINOR: Updated the Kafka JUnit extension document.

2021-02-17 Thread GitBox
kamalcph commented on pull request #10126: URL: https://github.com/apache/kafka/pull/10126#issuecomment-781048342 @mumrah @chia7712 Ping for review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] kamalcph commented on pull request #10139: MINOR: Print the warning log before truncation.

2021-02-17 Thread GitBox
kamalcph commented on pull request #10139: URL: https://github.com/apache/kafka/pull/10139#issuecomment-781048153 @chia7712 ping for review This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] chia7712 opened a new pull request #10145: KAFKA-12335 Upgrade junit from 5.7.0 to 5.7.1

2021-02-17 Thread GitBox
chia7712 opened a new pull request #10145: URL: https://github.com/apache/kafka/pull/10145 issue: https://issues.apache.org/jira/browse/KAFKA-12335 junit 5.7.1 release notes: https://junit.org/junit5/docs/5.7.1/release-notes/ ### Committer Checklist (excluded from commit

[jira] [Created] (KAFKA-12335) Upgrade junit from 5.7.0 to 5.7.1

2021-02-17 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-12335: -- Summary: Upgrade junit from 5.7.0 to 5.7.1 Key: KAFKA-12335 URL: https://issues.apache.org/jira/browse/KAFKA-12335 Project: Kafka Issue Type:

[GitHub] [kafka] rondagostino commented on pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
rondagostino commented on pull request #10113: URL: https://github.com/apache/kafka/pull/10113#issuecomment-781024771 @cmccabe I reverted the dependency on #10070. This PR, if the build jobs are clean and you are satisfied with the change I made, could be merged. We will instantiate

[jira] [Commented] (KAFKA-12169) Consumer can not know paritions change when client leader restart with static membership protocol

2021-02-17 Thread zou shengfu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286251#comment-17286251 ] zou shengfu commented on KAFKA-12169: - "restart with unknown member id",  this mean restart the

[jira] [Updated] (KAFKA-10885) Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of (unnecessary) ignored test cases

2021-02-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-10885: --- Fix Version/s: 3.0.0 > Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot

[jira] [Resolved] (KAFKA-10885) Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of (unnecessary) ignored test cases

2021-02-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10885. Resolution: Fixed > Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of

[GitHub] [kafka] chia7712 merged pull request #9906: KAFKA-10885 Refactor MemoryRecordsBuilderTest/MemoryRecordsTest to avoid a lot of…

2021-02-17 Thread GitBox
chia7712 merged pull request #9906: URL: https://github.com/apache/kafka/pull/9906 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] chia7712 merged pull request #10136: MONOR: Import classes that is used in docs to fix warnings.

2021-02-17 Thread GitBox
chia7712 merged pull request #10136: URL: https://github.com/apache/kafka/pull/10136 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] chia7712 commented on pull request #10136: MONOR: Import classes that is used in docs to fix warnings.

2021-02-17 Thread GitBox
chia7712 commented on pull request #10136: URL: https://github.com/apache/kafka/pull/10136#issuecomment-781012118 the test failure is related to #10024 `org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testOneWayReplicationWithAutoOffsetSync()` is

[GitHub] [kafka] rondagostino commented on pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
rondagostino commented on pull request #10113: URL: https://github.com/apache/kafka/pull/10113#issuecomment-781011014 @cmccabe I made the change to `SocketServer`. I think we're all set here, except the PR won't build (and cannot be merged) until

[GitHub] [kafka] rondagostino commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
rondagostino commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578090610 ## File path: core/src/main/scala/kafka/server/AlterIsrManager.scala ## @@ -70,7 +70,8 @@ object AlterIsrManager { time: Time, metrics:

[GitHub] [kafka] jsancio commented on a change in pull request #10063: KAFKA-12258: Add support for splitting appending records

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10063: URL: https://github.com/apache/kafka/pull/10063#discussion_r578085923 ## File path: raft/src/main/java/org/apache/kafka/raft/internals/BatchAccumulator.java ## @@ -79,67 +80,110 @@ public BatchAccumulator( }

[GitHub] [kafka] jsancio commented on a change in pull request #10063: KAFKA-12258: Add support for splitting appending records

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10063: URL: https://github.com/apache/kafka/pull/10063#discussion_r578082861 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -77,6 +79,29 @@ default void handleResign() {} */ void

[GitHub] [kafka] iczellion opened a new pull request #10144: MINOR: Fix Typo in MirrorMaker README file

2021-02-17 Thread GitBox
iczellion opened a new pull request #10144: URL: https://github.com/apache/kafka/pull/10144 Fix Typo in metric name of MirrorMaker README file from 'replication-latecny-ms' to 'replication-latency-ms' ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[GitHub] [kafka] hachikuji commented on a change in pull request #10066: KAFKA-12278; Ensure exposed api versions are consistent within listener scopes

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10066: URL: https://github.com/apache/kafka/pull/10066#discussion_r578064446 ## File path: generator/src/main/java/org/apache/kafka/message/RequestApiScope.java ## @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on pull request #10113: URL: https://github.com/apache/kafka/pull/10113#issuecomment-780929991 I think this is mostly ready to go. I left a few small comments. Let's resolve the 32-bit / 64-bit raft leader epoch thing elsewhere so we can get this in for 2.8...

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578023525 ## File path: core/src/main/scala/kafka/server/AutoTopicCreationManager.scala ## @@ -91,11 +91,14 @@ class DefaultAutoTopicCreationManager( config:

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578021861 ## File path: core/src/main/scala/kafka/server/AlterIsrManager.scala ## @@ -70,7 +70,8 @@ object AlterIsrManager { time: Time, metrics:

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578020912 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -573,7 +584,7 @@ private[kafka] class Acceptor(val endPoint: EndPoint,

[GitHub] [kafka] jolshan commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
jolshan commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r578010946 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578010753 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftClientTestContext.java ## @@ -975,7 +975,7 @@ public void handleClaim(int epoch) { }

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578010628 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -57,15 +57,16 @@ default void handleClaim(int epoch) {} /**

[GitHub] [kafka] cmccabe commented on a change in pull request #10113: MINOR: Add KIP-500 BrokerServer and ControllerServer

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10113: URL: https://github.com/apache/kafka/pull/10113#discussion_r578010399 ## File path: core/src/main/scala/kafka/tools/TestRaftServer.scala ## @@ -160,7 +160,7 @@ class TestRaftServer(

[GitHub] [kafka] jolshan commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
jolshan commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r578007354 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] hachikuji commented on a change in pull request #10129: KAFKA-10817; Add clusterId validation to Fetch handling

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10129: URL: https://github.com/apache/kafka/pull/10129#discussion_r577993815 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/Errors.java ## @@ -354,7 +355,8 @@ "Requested position is not greater

[GitHub] [kafka] hachikuji commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r577978644 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] jolshan commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
jolshan commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r577986110 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] jolshan commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
jolshan commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r577985597 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[jira] [Commented] (KAFKA-12328) Find out partition of a store iterator

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286175#comment-17286175 ] Matthias J. Sax commented on KAFKA-12328: - Yes, the state store changelog topic has one

[GitHub] [kafka] hachikuji commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r577978644 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] hachikuji commented on a change in pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10143: URL: https://github.com/apache/kafka/pull/10143#discussion_r577972328 ## File path: core/src/main/scala/kafka/server/ReplicaManager.scala ## @@ -1364,34 +1364,59 @@ class ReplicaManager(val config: KafkaConfig,

[GitHub] [kafka] jsancio commented on a change in pull request #10138: KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10138: URL: https://github.com/apache/kafka/pull/10138#discussion_r577972986 ## File path: raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientSnapshotTest.java ## @@ -106,7 +106,8 @@ public void

[GitHub] [kafka] cmccabe commented on pull request #10094: KAFKA-12334: Add the KIP-631 metadata shell

2021-02-17 Thread GitBox
cmccabe commented on pull request #10094: URL: https://github.com/apache/kafka/pull/10094#issuecomment-780880646 I rebased this on trunk and created a JIRA for it This is an automated message from the Apache Git Service. To

[GitHub] [kafka] rondagostino commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
rondagostino commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577971959 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,916 @@ +/* + * Licensed to the Apache Software

[jira] [Created] (KAFKA-12334) Add the KIP-631 metadata shell

2021-02-17 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-12334: Summary: Add the KIP-631 metadata shell Key: KAFKA-12334 URL: https://issues.apache.org/jira/browse/KAFKA-12334 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-12328) Find out partition of a store iterator

2021-02-17 Thread fml2 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286159#comment-17286159 ] fml2 commented on KAFKA-12328: -- OK, I did not know that a task is created per partition, i.e. that there is

[GitHub] [kafka] rondagostino commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
rondagostino commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577961646 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,916 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan opened a new pull request #10143: KAFKA-12332: Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread GitBox
jolshan opened a new pull request #10143: URL: https://github.com/apache/kafka/pull/10143 Changes how invalid IDs are handled in LeaderAndIsr requests. The ID check now occurs before leader epoch. If the ID exists and is invalid, the partition is ignored and an UNKNOWN_TOPIC_ID error is

[GitHub] [kafka] abbccdda opened a new pull request #10142: KAFKA-12294: forward auto topic request within envelope on behalf of clients

2021-02-17 Thread GitBox
abbccdda opened a new pull request #10142: URL: https://github.com/apache/kafka/pull/10142 For the metadata auto topic creation case, it is favorable to use `Envelope` to wrap the CreateTopicsRequest alongside the original client principal for auditing purpose. ### Committer

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577929403 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577929118 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -0,0 +1,712 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577922831 ## File path: metadata/src/main/java/org/apache/kafka/controller/Controller.java ## @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577922334 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577921275 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577920610 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokersToIsrs.java ## @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577920156 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,519 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577919831 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java ## @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] hachikuji commented on a change in pull request #10138: KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch

2021-02-17 Thread GitBox
hachikuji commented on a change in pull request #10138: URL: https://github.com/apache/kafka/pull/10138#discussion_r577914843 ## File path: raft/src/test/java/org/apache/kafka/raft/KafkaRaftClientSnapshotTest.java ## @@ -106,7 +106,8 @@ public void

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577919102 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577918400 ## File path: metadata/src/main/java/org/apache/kafka/controller/BrokerHeartbeatManager.java ## @@ -0,0 +1,519 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577916988 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java ## @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577916633 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java ## @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577916179 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577915565 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577914392 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -0,0 +1,712 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] jsancio commented on a change in pull request #10138: KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10138: URL: https://github.com/apache/kafka/pull/10138#discussion_r577868426 ## File path: raft/src/test/java/org/apache/kafka/raft/MockLog.java ## @@ -250,30 +249,7 @@ private LogEntry buildEntry(Long offset, SimpleRecord

[GitHub] [kafka] jsancio commented on pull request #10138: KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch

2021-02-17 Thread GitBox
jsancio commented on pull request #10138: URL: https://github.com/apache/kafka/pull/10138#issuecomment-780816612 @hachikuji this PR is ready for review. Thanks! This is an automated message from the Apache Git Service. To

[jira] [Created] (KAFKA-12333) KafkaMetadataLog and MockLock should validate that appended epochs are monotonically

2021-02-17 Thread Jose Armando Garcia Sancio (Jira)
Jose Armando Garcia Sancio created KAFKA-12333: -- Summary: KafkaMetadataLog and MockLock should validate that appended epochs are monotonically Key: KAFKA-12333 URL:

[jira] [Commented] (KAFKA-12213) Kafka Streams aggregation Initializer to accept record key

2021-02-17 Thread Bill Bejeck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286114#comment-17286114 ] Bill Bejeck commented on KAFKA-12213: - Hi, [~MonCalamari]. I have an implementation for the

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577895640 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577894071 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java ## @@ -0,0 +1,456 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] junrao commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
junrao commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577879960 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,917 @@ +/* + * Licensed to the Apache Software

[jira] [Comment Edited] (KAFKA-10888) Sticky partition leads to uneven product msg, resulting in abnormal delays in some partations

2021-02-17 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286089#comment-17286089 ] Jason Gustafson edited comment on KAFKA-10888 at 2/17/21, 7:04 PM: ---

[jira] [Commented] (KAFKA-10888) Sticky partition leads to uneven product msg, resulting in abnormal delays in some partations

2021-02-17 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286089#comment-17286089 ] Jason Gustafson commented on KAFKA-10888: - [~junrao] That's a good observation. The partitioner

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577868843 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -0,0 +1,712 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] jsancio commented on a change in pull request #10138: KAFKA-12331: Use LEO for the base offset of LeaderChangeMessage batch

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10138: URL: https://github.com/apache/kafka/pull/10138#discussion_r577868426 ## File path: raft/src/test/java/org/apache/kafka/raft/MockLog.java ## @@ -250,30 +249,7 @@ private LogEntry buildEntry(Long offset, SimpleRecord

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577866850 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,920 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cmccabe commented on a change in pull request #10070: KAFKA-12276: Add the quorum controller code

2021-02-17 Thread GitBox
cmccabe commented on a change in pull request #10070: URL: https://github.com/apache/kafka/pull/10070#discussion_r577865782 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -0,0 +1,917 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jsancio commented on a change in pull request #10021: KAFKA-12205: Delete snapshot if leo less than the log start offset

2021-02-17 Thread GitBox
jsancio commented on a change in pull request #10021: URL: https://github.com/apache/kafka/pull/10021#discussion_r577852672 ## File path: core/src/main/scala/kafka/raft/KafkaMetadataLog.scala ## @@ -267,30 +259,45 @@ final class KafkaMetadataLog private ( } } -

[jira] [Updated] (KAFKA-12323) Record timestamps not populated in event

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12323: Priority: Blocker (was: Major) > Record timestamps not populated in event >

[jira] [Updated] (KAFKA-12323) Record timestamps not populated in event

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-12323: Fix Version/s: 2.7.1 2.8.0 > Record timestamps not populated in event

[jira] [Comment Edited] (KAFKA-12323) Record timestamps not populated in event

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286052#comment-17286052 ] Matthias J. Sax edited comment on KAFKA-12323 at 2/17/21, 6:20 PM: ---

[jira] [Commented] (KAFKA-12323) Record timestamps not populated in event

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286052#comment-17286052 ] Matthias J. Sax commented on KAFKA-12323: - Thanks [~guozhang]. SGTM. – Are we sure that we want

[jira] [Updated] (KAFKA-12332) Error partitions from topics with invalid IDs in LISR requests

2021-02-17 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-12332: --- Summary: Error partitions from topics with invalid IDs in LISR requests (was: Ignore LISR

[jira] [Created] (KAFKA-12332) Ignore LISR requests from topics with invalid IDs.

2021-02-17 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-12332: -- Summary: Ignore LISR requests from topics with invalid IDs. Key: KAFKA-12332 URL: https://issues.apache.org/jira/browse/KAFKA-12332 Project: Kafka Issue

[jira] [Commented] (KAFKA-12328) Find out partition of a store iterator

2021-02-17 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286047#comment-17286047 ] Matthias J. Sax commented on KAFKA-12328: - It's correct that a task may process multiple

[jira] [Commented] (KAFKA-10888) Sticky partition leads to uneven product msg, resulting in abnormal delays in some partations

2021-02-17 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286039#comment-17286039 ] Jun Rao commented on KAFKA-10888: - [~hachikuji]: Thanks for the analysis. One thing that led to this

[GitHub] [kafka] junrao commented on pull request #10125: MINOR: Add note about topic IDs to upgrade doc

2021-02-17 Thread GitBox
junrao commented on pull request #10125: URL: https://github.com/apache/kafka/pull/10125#issuecomment-780733439 cherry-picked to trunk too. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] junrao merged pull request #10125: MINOR: Add note about topic IDs to upgrade doc

2021-02-17 Thread GitBox
junrao merged pull request #10125: URL: https://github.com/apache/kafka/pull/10125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-12323) Record timestamps not populated in event

2021-02-17 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286016#comment-17286016 ] Guozhang Wang commented on KAFKA-12323: --- I checked the source code between 2.6 and trunk, and I

[GitHub] [kafka] C0urante commented on pull request #10112: KAFKA-12226: Prevent source task offset failure when producer is overwhelmed

2021-02-17 Thread GitBox
C0urante commented on pull request #10112: URL: https://github.com/apache/kafka/pull/10112#issuecomment-780718729 The only consistent test failure for the last run appears unrelated to the changes here. See https://github.com/apache/kafka/pull/10140/checks?check_run_id=1917700907 and

[GitHub] [kafka] jolshan commented on a change in pull request #10125: MINOR: Add note about topic IDs to upgrade doc

2021-02-17 Thread GitBox
jolshan commented on a change in pull request #10125: URL: https://github.com/apache/kafka/pull/10125#discussion_r577794185 ## File path: docs/upgrade.html ## @@ -23,13 +23,17 @@ Notable changes in 2 The 2.8.0 release added a new method to the Authorizer

[GitHub] [kafka] junrao commented on a change in pull request #10125: MINOR: Add note about topic IDs to upgrade doc

2021-02-17 Thread GitBox
junrao commented on a change in pull request #10125: URL: https://github.com/apache/kafka/pull/10125#discussion_r577781142 ## File path: docs/upgrade.html ## @@ -23,13 +23,17 @@ Notable changes in 2 The 2.8.0 release added a new method to the Authorizer

[GitHub] [kafka] tang7526 commented on pull request #10124: MINOR: apply Utils.isBlank to code base

2021-02-17 Thread GitBox
tang7526 commented on pull request #10124: URL: https://github.com/apache/kafka/pull/10124#issuecomment-780658142 > @tang7526 Thanks for your patch. Could you fix following code also? > > 1.

[jira] [Commented] (KAFKA-12328) Find out partition of a store iterator

2021-02-17 Thread fml2 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17285814#comment-17285814 ] fml2 commented on KAFKA-12328: -- Hrm... IIUC a task can process multiple partitions (e.g. a join task),

[GitHub] [kafka] chia7712 commented on a change in pull request #10141: KAFKA-12329; kafka-reassign-partitions command should give a better error message when a topic does not exist

2021-02-17 Thread GitBox
chia7712 commented on a change in pull request #10141: URL: https://github.com/apache/kafka/pull/10141#discussion_r577570241 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java ## @@ -1887,7 +1887,11 @@ void handleResponse(AbstractResponse

[GitHub] [kafka] dajac opened a new pull request #10141: KAFKA-12329; kafka-reassign-partitions command should give a better error message when a topic does not exist

2021-02-17 Thread GitBox
dajac opened a new pull request #10141: URL: https://github.com/apache/kafka/pull/10141 `kafka-reassign-partitions` command gives a generic error message when one tries to reassign a topic which does not exist: ``` $ ./bin/kafka-reassign-partitions.sh --bootstrap-server

[GitHub] [kafka] purplefox closed pull request #10132: Fix ssl close

2021-02-17 Thread GitBox
purplefox closed pull request #10132: URL: https://github.com/apache/kafka/pull/10132 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] chia7712 opened a new pull request #10140: MINOR: remove unused helper methods from utils classes

2021-02-17 Thread GitBox
chia7712 opened a new pull request #10140: URL: https://github.com/apache/kafka/pull/10140 ### removed methods 1. `CollectionUtils#subtractMap` 1. `CollectionUtils#groupPartitionDataByTopic` 1. `Utils#mkObjectProperties` 1. `Utils#safe` 1. `Utils#concatListsUnmodifiable`