Magnus Edenhill created KAFKA-12712:
---------------------------------------
Summary: KRaft: Missing controller.quorom.voters config not
properly handled
Key: KAFKA-12712
URL: https://issues.apache.org/jira/browse/KAFKA-12712
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 2.8.0
Reporter: Magnus Edenhill
When trying out KRaft in 2.8 I mispelled controller.quorum.voters as
controller.quorum.voters, but the broker did not fail to start, nor did it
print any warning.
Instead it raised this error:
{code:java}
[2021-04-23 18:25:13,484] INFO Starting controller
(kafka.server.ControllerServer)[2021-04-23 18:25:13,484] INFO Starting
controller (kafka.server.ControllerServer)[2021-04-23 18:25:13,485] ERROR
[kafka-raft-io-thread]: Error due to
(kafka.raft.KafkaRaftManager$RaftIoThread)java.lang.IllegalArgumentException:
bound must be positive at java.util.Random.nextInt(Random.java:388) at
org.apache.kafka.raft.RequestManager.findReadyVoter(RequestManager.java:57) at
org.apache.kafka.raft.KafkaRaftClient.maybeSendAnyVoterFetch(KafkaRaftClient.java:1778)
at
org.apache.kafka.raft.KafkaRaftClient.pollUnattachedAsObserver(KafkaRaftClient.java:2080)
at
org.apache.kafka.raft.KafkaRaftClient.pollUnattached(KafkaRaftClient.java:2061)
at
org.apache.kafka.raft.KafkaRaftClient.pollCurrentState(KafkaRaftClient.java:2096)
at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2181) at
kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:53) at
kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
{code}
which I guess eventually (1 minute later) lead to this error which terminated
the broker:
{code:java}
[2021-04-23 18:26:14,435] ERROR [BrokerLifecycleManager id=2] Shutting down
because we were unable to register with the controller quorum.
(kafka.server.BrokerLifecycleManager)[2021-04-23 18:26:14,435] ERROR
[BrokerLifecycleManager id=2] Shutting down because we were unable to register
with the controller quorum. (kafka.server.BrokerLifecycleManager)[2021-04-23
18:26:14,436] INFO [BrokerLifecycleManager id=2] registrationTimeout: shutting
down event queue. (org.apache.kafka.queue.KafkaEventQueue)[2021-04-23
18:26:14,437] INFO [BrokerLifecycleManager id=2] Transitioning from STARTING to
SHUTTING_DOWN. (kafka.server.BrokerLifecycleManager)[2021-04-23 18:26:14,437]
INFO [broker-2-to-controller-send-thread]: Shutting down
(kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,438] INFO
[broker-2-to-controller-send-thread]: Stopped
(kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,438] INFO
[broker-2-to-controller-send-thread]: Shutdown completed
(kafka.server.BrokerToControllerRequestThread)[2021-04-23 18:26:14,441] ERROR
[BrokerServer id=2] Fatal error during broker startup. Prepare to shutdown
(kafka.server.BrokerServer)java.util.concurrent.CancellationException at
java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2276) at
kafka.server.BrokerLifecycleManager$ShutdownEvent.run(BrokerLifecycleManager.scala:474)
at
org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:174)
at java.lang.Thread.run(Thread.java:748)
{code}
But since the client listeners were made available prior to shutting down, the
broker was deemed up and operational by the (naiive) monitoring tool.
So..:
- Broker should fail on startup on invalid/unknown config properties. I
understand this is tehcnically tricky, so at least a warning log should be
printed.
- Perhaps not create client listeners before control plane is somewhat happy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
