[GitHub] [kafka] hachikuji commented on a change in pull request #10045: MINOR: Allow KafkaApis to be configured for Raft controller quorums
hachikuji commented on a change in pull request #10045:
URL: https://github.com/apache/kafka/pull/10045#discussion_r571174163
##
File path: core/src/main/scala/kafka/server/KafkaApis.scala
##
@@ -3511,4 +3501,14 @@ object KafkaApis {
FetchResponse.sizeOf(versionId, unconvertedResponse.responseData.entrySet
.iterator.asScala.filter(element =>
quota.isThrottled(element.getKey)).asJava)
}
+
+ // visible for testing
+ private[server] def shouldNeverReceive(request: RequestChannel.Request):
Exception = {
Review comment:
Why move these and not `unsupported`/`notYetSupported`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] hachikuji commented on a change in pull request #10045: MINOR: Allow KafkaApis to be configured for Raft controller quorums
hachikuji commented on a change in pull request #10045:
URL: https://github.com/apache/kafka/pull/10045#discussion_r570727037
##
File path: core/src/main/scala/kafka/server/MetadataSupport.scala
##
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.controller.KafkaController
+import kafka.network.RequestChannel
+import kafka.zk.{AdminZkClient, KafkaZkClient}
+import org.apache.kafka.common.requests.AbstractResponse
+
+sealed trait MetadataSupport {
+ /**
+ * Provide a uniform way of getting to the ForwardingManager, which is a
shared concept
+ * despite being optional when using ZooKeeper and required when using Raft
+ */
+ val forwardingManager: Option[ForwardingManager]
+
+ /**
+ * Return this instance downcast for use with ZooKeeper
+ *
+ * @param createException function to create an exception to throw
+ * @return this instance downcast for use with ZooKeeper
+ * @throws Exception if this instance is not for ZooKeeper
+ */
+ def requireZkOrThrow(createException: => Exception): ZkSupport
+
+ /**
+ * Return this instance downcast for use with Raft
+ *
+ * @param createException function to create an exception to throw
+ * @return this instance downcast for use with Raft
+ * @throws Exception if this instance is not for Raft
+ */
+ def requireRaftOrThrow(createException: => Exception): RaftSupport
+
+ /**
+ * Confirm that this instance is consistent with the given config
+ *
+ * @param config the config to check for consistency with this instance
+ * @throws IllegalStateException if there is an inconsistency (Raft for a
ZooKeeper config or vice-versa)
+ */
+ def confirmConsistentWith(config: KafkaConfig): Unit
+
+ def maybeForward(request: RequestChannel.Request,
+ handler: RequestChannel.Request => Unit,
+ responseCallback: Option[AbstractResponse] => Unit): Unit
+}
+
+case class ZkSupport(adminManager: ZkAdminManager,
+ controller: KafkaController,
+ zkClient: KafkaZkClient,
+ forwardingManager: Option[ForwardingManager]) extends
MetadataSupport {
+ val adminZkClient = new AdminZkClient(zkClient)
+
+ override def requireZkOrThrow(createException: => Exception): ZkSupport =
this
+ override def requireRaftOrThrow(createException: => Exception): RaftSupport
= throw createException
+
+ override def confirmConsistentWith(config: KafkaConfig): Unit = {
+if (!config.requiresZookeeper) {
+ throw new IllegalStateException("Config specifies Raft but metadata
support instance is for ZooKeeper")
+}
+ }
+
+ override def maybeForward(request: RequestChannel.Request,
+handler: RequestChannel.Request => Unit,
+responseCallback: Option[AbstractResponse] =>
Unit): Unit = {
+if (forwardingManager.isDefined && !request.isForwarded &&
!controller.isActive) {
Review comment:
nit: usually when you see `isDefined` followed by `get`, there is likely
an opportunity for a `match` or `foreach`.
```scala
forwardingManager match {
case Some(mgr) if !request.isForwarded && !controllers.isActive =>
forwardingManager.get.forwardRequest(request, responseCallback)
case _ =>
handler(request)
}
```
##
File path: core/src/main/scala/kafka/server/MetadataSupport.scala
##
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific lang
[GitHub] [kafka] hachikuji commented on a change in pull request #10045: MINOR: Allow KafkaApis to be configured for Raft controller quorums
hachikuji commented on a change in pull request #10045:
URL: https://github.com/apache/kafka/pull/10045#discussion_r570730096
##
File path: core/src/main/scala/kafka/server/MetadataSupport.scala
##
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.controller.KafkaController
+import kafka.network.RequestChannel
+import kafka.zk.{AdminZkClient, KafkaZkClient}
+import org.apache.kafka.common.requests.AbstractResponse
+
+sealed trait MetadataSupport {
+ /**
+ * Provide a uniform way of getting to the ForwardingManager, which is a
shared concept
+ * despite being optional when using ZooKeeper and required when using Raft
+ */
+ val forwardingManager: Option[ForwardingManager]
+
+ /**
+ * Return this instance downcast for use with ZooKeeper
+ *
+ * @param createException function to create an exception to throw
+ * @return this instance downcast for use with ZooKeeper
+ * @throws Exception if this instance is not for ZooKeeper
+ */
+ def requireZkOrThrow(createException: => Exception): ZkSupport
Review comment:
It's a little odd to see this in the trait. Would it be reasonable to
turn them into private defs in `KafkaApis`? I'm ok with it if you think it is
better. We'll only ever have the two implementations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] hachikuji commented on a change in pull request #10045: MINOR: Allow KafkaApis to be configured for Raft controller quorums
hachikuji commented on a change in pull request #10045:
URL: https://github.com/apache/kafka/pull/10045#discussion_r570727037
##
File path: core/src/main/scala/kafka/server/MetadataSupport.scala
##
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package kafka.server
+
+import kafka.controller.KafkaController
+import kafka.network.RequestChannel
+import kafka.zk.{AdminZkClient, KafkaZkClient}
+import org.apache.kafka.common.requests.AbstractResponse
+
+sealed trait MetadataSupport {
+ /**
+ * Provide a uniform way of getting to the ForwardingManager, which is a
shared concept
+ * despite being optional when using ZooKeeper and required when using Raft
+ */
+ val forwardingManager: Option[ForwardingManager]
+
+ /**
+ * Return this instance downcast for use with ZooKeeper
+ *
+ * @param createException function to create an exception to throw
+ * @return this instance downcast for use with ZooKeeper
+ * @throws Exception if this instance is not for ZooKeeper
+ */
+ def requireZkOrThrow(createException: => Exception): ZkSupport
+
+ /**
+ * Return this instance downcast for use with Raft
+ *
+ * @param createException function to create an exception to throw
+ * @return this instance downcast for use with Raft
+ * @throws Exception if this instance is not for Raft
+ */
+ def requireRaftOrThrow(createException: => Exception): RaftSupport
+
+ /**
+ * Confirm that this instance is consistent with the given config
+ *
+ * @param config the config to check for consistency with this instance
+ * @throws IllegalStateException if there is an inconsistency (Raft for a
ZooKeeper config or vice-versa)
+ */
+ def confirmConsistentWith(config: KafkaConfig): Unit
+
+ def maybeForward(request: RequestChannel.Request,
+ handler: RequestChannel.Request => Unit,
+ responseCallback: Option[AbstractResponse] => Unit): Unit
+}
+
+case class ZkSupport(adminManager: ZkAdminManager,
+ controller: KafkaController,
+ zkClient: KafkaZkClient,
+ forwardingManager: Option[ForwardingManager]) extends
MetadataSupport {
+ val adminZkClient = new AdminZkClient(zkClient)
+
+ override def requireZkOrThrow(createException: => Exception): ZkSupport =
this
+ override def requireRaftOrThrow(createException: => Exception): RaftSupport
= throw createException
+
+ override def confirmConsistentWith(config: KafkaConfig): Unit = {
+if (!config.requiresZookeeper) {
+ throw new IllegalStateException("Config specifies Raft but metadata
support instance is for ZooKeeper")
+}
+ }
+
+ override def maybeForward(request: RequestChannel.Request,
+handler: RequestChannel.Request => Unit,
+responseCallback: Option[AbstractResponse] =>
Unit): Unit = {
+if (forwardingManager.isDefined && !request.isForwarded &&
!controller.isActive) {
Review comment:
nit: usually when you see `isDefined` followed by `get`, there is likely
an opportunity for a `match` or `foreach`.
```scala
forwardingManager match {
case Some(mgr) if !request.isForwarded && !controllers.isActive =>
forwardingManager.get.forwardRequest(request, responseCallback)
case _ =>
handler(request)
}
```
##
File path: core/src/main/scala/kafka/server/MetadataSupport.scala
##
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific lang
