Josep Prat created KAFKA-15071:
----------------------------------
Summary: Flaky test
kafka.admin.LeaderElectionCommandTest.testPreferredReplicaElection for Type=ZK,
MetadataVersion=3.5-IV2, Security=PLAINTEXT
Key: KAFKA-15071
URL: https://issues.apache.org/jira/browse/KAFKA-15071
Project: Kafka
Issue Type: Bug
Reporter: Josep Prat
Test became kafka.admin.LeaderElectionCommandTest.testPreferredReplicaElection
flaky again but failing because of different reason. In this case it might be a
missing cleanup
The values of the parameters are Type=ZK, MetadataVersion=3.5-IV2,
Security=PLAINTEXT
Related to https://issues.apache.org/jira/browse/KAFKA-13737
Ocurred:
https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-13824/1/testReport/junit/kafka.admin/LeaderElectionCommandTest/Build___JDK_8_and_Scala_2_12____3__Type_ZK__Name_testPreferredReplicaElection__MetadataVersion_3_5_IV2__Security_PLAINTEXT/
h3. Error Message
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists.{code}
h3. Stacktrace
{code:java}
org.apache.kafka.common.errors.TopicExistsException: Topic '__consumer_offsets'
already exists.{code}
{{ }}
h3. Standard Output
{code:java}
Successfully completed leader election (UNCLEAN) for partitions unclean-topic-0
[2023-06-07 14:42:33,845] ERROR [QuorumController id=3000] writeNoOpRecord:
unable to start processing because of RejectedExecutionException. Reason: The
event queue is shutting down (org.apache.kafka.controller.QuorumController:467)
[2023-06-07 14:42:42,699] WARN [AdminClient clientId=adminclient-65] Connection
to node -2 (localhost/127.0.0.1:35103) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) Successfully
completed leader election (UNCLEAN) for partitions unclean-topic-0 [2023-06-07
14:42:44,416] ERROR [QuorumController id=0] writeNoOpRecord: unable to start
processing because of RejectedExecutionException. Reason: The event queue is
shutting down (org.apache.kafka.controller.QuorumController:467) [2023-06-07
14:42:44,716] WARN maxCnxns is not configured, using default value 0.
(org.apache.zookeeper.server.ServerCnxnFactory:309) [2023-06-07 14:42:44,765]
WARN No meta.properties file under dir
/tmp/kafka-2117748934951771120/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:44,986] WARN No
meta.properties file under dir /tmp/kafka-5133306871105583937/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:45,214] WARN No
meta.properties file under dir /tmp/kafka-8449809620400833553/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 14:42:45,634] WARN
[ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-0. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
14:42:45,634] WARN [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0]
Received UNKNOWN_TOPIC_ID from the leader for partition __consumer_offsets-4.
This error may be returned transiently when the partition is being created or
deleted, but it is not expected to persist.
(kafka.server.ReplicaFetcherThread:70) [2023-06-07 14:42:45,872] WARN
[ReplicaFetcher replicaId=2, leaderId=0, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-1. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
14:42:46,010] WARN [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error
in response for fetch request (type=FetchRequest, replicaId=0, maxWait=500,
minBytes=1, maxBytes=10485760,
fetchData={__consumer_offsets-3=PartitionData(topicId=vAlEsYVbTFClcpnVRp3AOw,
fetchOffset=0, logStartOffset=0, maxBytes=1048576,
currentLeaderEpoch=Optional[0], lastFetchedEpoch=Optional.empty)},
isolationLevel=READ_UNCOMMITTED, removed=, replaced=,
metadata=(sessionId=INVALID, epoch=INITIAL), rackId=)
(kafka.server.ReplicaFetcherThread:72) java.io.IOException: Connection to 2 was
disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.server.BrokerBlockingSender.sendRequest(BrokerBlockingSender.scala:113)
at kafka.server.RemoteLeaderEndPoint.fetch(RemoteLeaderEndPoint.scala:79) at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:316)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:130)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:129)
at scala.Option.foreach(Option.scala:407) at
kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:129)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:112)
at kafka.server.ReplicaFetcherThread.doWork(ReplicaFetcherThread.scala:98) at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,010] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=62,
partitionStates=[LeaderAndIsrPartitionState(topicName='unclean-topic',
partitionIndex=0, controllerEpoch=1, leader=1, leaderEpoch=0, isr=[1, 2],
partitionEpoch=0, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=true, leaderRecoveryState=0)],
topicIds={unclean-topic=KnY8qAR8T4uk-GkDZS-l8w}, liveLeaders=(localhost:38923
(id: 1 rack: null))) to broker localhost:42679 (id: 2 rack: null). Reconnecting
to broker. (kafka.controller.RequestSendThread:72) java.io.IOException:
Connection to 2 was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,112] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,113] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,214] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,214] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,315] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,315] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,416] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,416] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,517] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,517] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,618] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,618] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,719] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,719] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,820] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,820] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:46,922] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:46,922] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,023] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,023] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,124] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,124] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,226] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,226] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,327] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,327] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,428] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,428] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,529] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,529] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,631] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,631] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,732] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,732] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,833] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:42679) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:47,833] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:42679 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:42679 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:47,883] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=62,
partitionStates=[LeaderAndIsrPartitionState(topicName='unclean-topic',
partitionIndex=0, controllerEpoch=1, leader=1, leaderEpoch=0, isr=[1, 2],
partitionEpoch=0, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=true, leaderRecoveryState=0)],
topicIds={unclean-topic=KnY8qAR8T4uk-GkDZS-l8w}, liveLeaders=(localhost:38923
(id: 1 rack: null))) to broker localhost:42679 (id: 2 rack: null). Reconnecting
to broker. (kafka.controller.RequestSendThread:72)
java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:48,018] WARN [KafkaServer id=1] Proceeding to do an unclean
shutdown as all the controlled shutdown attempts failed
(kafka.server.KafkaServer:70) Successfully completed leader election (UNCLEAN)
for partitions __consumer_offsets-0, unclean-topic-0 [2023-06-07 14:42:50,231]
WARN [KafkaServer id=0] Proceeding to do an unclean shutdown as all the
controlled shutdown attempts failed (kafka.server.KafkaServer:70) [2023-06-07
14:42:50,234] WARN [RequestSendThread controllerId=0] Controller 0 epoch 1
fails to send request (type: UpdateMetadataRequest=, controllerId=0,
controllerEpoch=1, brokerEpoch=26,
partitionStates=[UpdateMetadataPartitionState(topicName='__consumer_offsets',
partitionIndex=1, controllerEpoch=1, leader=2, leaderEpoch=2, isr=[2],
zkVersion=3, replicas=[0, 2], offlineReplicas=[0]),
UpdateMetadataPartitionState(topicName='__consumer_offsets', partitionIndex=3,
controllerEpoch=1, leader=2, leaderEpoch=2, isr=[2], zkVersion=3, replicas=[2,
0], offlineReplicas=[0])], liveBrokers=UpdateMetadataBroker(id=2, v0Host='',
v0Port=0, endpoints=[UpdateMetadataEndpoint(port=46571, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null),
UpdateMetadataBroker(id=0, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=39965, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null)) to broker
localhost:39965 (id: 0 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to 0
was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,235] WARN [KafkaServer id=2] Error during controlled
shutdown, possibly because leader movement took longer than the configured
controller.socket.timeout.ms and/or request.timeout.ms: Connection to 0 was
disconnected before the response was read (kafka.server.KafkaServer:70)
[2023-06-07 14:42:50,234] WARN [ReplicaFetcher replicaId=2, leaderId=0,
fetcherId=0] Partition __consumer_offsets-1 marked as failed
(kafka.server.ReplicaFetcherThread:70) [2023-06-07 14:42:50,237] WARN
[ReplicaFetcher replicaId=2, leaderId=0, fetcherId=0] Partition
__consumer_offsets-3 marked as failed (kafka.server.ReplicaFetcherThread:70)
[2023-06-07 14:42:50,238] WARN [ReplicaFetcher replicaId=2, leaderId=0,
fetcherId=0] Error in response for fetch request (type=FetchRequest,
replicaId=2, maxWait=500, minBytes=1, maxBytes=10485760, fetchData={},
isolationLevel=READ_UNCOMMITTED,
removed=vAlEsYVbTFClcpnVRp3AOw:__consumer_offsets-1,
vAlEsYVbTFClcpnVRp3AOw:__consumer_offsets-3, replaced=,
metadata=(sessionId=1379156303, epoch=1), rackId=)
(kafka.server.ReplicaFetcherThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.server.BrokerBlockingSender.sendRequest(BrokerBlockingSender.scala:108)
at kafka.server.RemoteLeaderEndPoint.fetch(RemoteLeaderEndPoint.scala:79) at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:316)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:130)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:129)
at scala.Option.foreach(Option.scala:407) at
kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:129)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:112)
at kafka.server.ReplicaFetcherThread.doWork(ReplicaFetcherThread.scala:98) at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,240] WARN [KafkaServer id=2] Proceeding to do an unclean
shutdown as all the controlled shutdown attempts failed
(kafka.server.KafkaServer:70) [2023-06-07 14:42:50,242] WARN [RequestSendThread
controllerId=0] Controller 0 epoch 1 fails to send request
(type=LeaderAndIsRequest, controllerId=0, controllerEpoch=1, brokerEpoch=109,
partitionStates=[LeaderAndIsrPartitionState(topicName='unclean-topic',
partitionIndex=0, controllerEpoch=1, leader=2, leaderEpoch=3, isr=[2],
partitionEpoch=3, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=1),
LeaderAndIsrPartitionState(topicName='__consumer_offsets', partitionIndex=0,
controllerEpoch=1, leader=2, leaderEpoch=3, isr=[2], partitionEpoch=3,
replicas=[2, 1], addingReplicas=[], removingReplicas=[], isNew=false,
leaderRecoveryState=1)], topicIds={unclean-topic=KnY8qAR8T4uk-GkDZS-l8w,
__consumer_offsets=vAlEsYVbTFClcpnVRp3AOw}, liveLeaders=(localhost:46571 (id: 2
rack: null))) to broker localhost:46571 (id: 2 rack: null). Reconnecting to
broker. (kafka.controller.RequestSendThread:72) java.io.IOException: Connection
to 2 was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,336] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,337] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,343] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,344] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,438] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,438] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,445] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,445] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,540] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,540] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,546] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,546] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,642] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,642] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,647] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,647] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,743] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,743] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,748] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,748] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,845] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,845] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,849] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,850] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,946] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,947] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:50,951] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:50,951] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,048] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,048] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,052] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,052] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,149] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,149] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,153] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,153] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,251] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,251] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,254] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,254] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,352] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,353] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,355] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,355] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,454] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,454] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,456] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,457] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,555] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,556] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,558] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,558] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,657] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,657] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:39965 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:39965 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,659] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46571) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,659] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46571 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46571 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:42:51,759] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:39965) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:42:51,759] WARN [RequestSendThread controllerId=0] Controller 0's
...[truncated 309404 chars]... lManager.scala:296) at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,255] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,255] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,270] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,270] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,355] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,356] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,370] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,370] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,456] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,457] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,471] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,471] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,557] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,558] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,572] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,572] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,659] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,659] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,673] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,673] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,760] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,760] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,774] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,774] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,861] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,861] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,875] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:37897) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,875] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37897 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37897 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,890] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=110,
partitionStates=[LeaderAndIsrPartitionState(topicName='unclean-topic',
partitionIndex=0, controllerEpoch=1, leader=2, leaderEpoch=3, isr=[2],
partitionEpoch=3, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=1)],
topicIds={unclean-topic=l44oy-3XQQG8VIambYEjbQ}, liveLeaders=(localhost:46749
(id: 2 rack: null))) to broker localhost:46749 (id: 2 rack: null). Reconnecting
to broker. (kafka.controller.RequestSendThread:72)
java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,892] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=26,
partitionStates=[LeaderAndIsrPartitionState(topicName='__consumer_offsets',
partitionIndex=1, controllerEpoch=1, leader=0, leaderEpoch=2, isr=[0],
partitionEpoch=3, replicas=[2, 0], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=0)],
topicIds={__consumer_offsets=Z7eXcfnaS06bBpvabr6NNw},
liveLeaders=(localhost:37897 (id: 0 rack: null))) to broker localhost:37897
(id: 0 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,913] WARN [Controller id=2, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:46749) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
14:44:53,914] WARN [RequestSendThread controllerId=2] Controller 2's connection
to broker localhost:46749 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:46749 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 14:44:53,947] WARN [RequestSendThread controllerId=2] Controller 2
epoch 2 fails to send request (type: UpdateMetadataRequest=, controllerId=2,
controllerEpoch=2, brokerEpoch=110, partitionStates=[],
liveBrokers=UpdateMetadataBroker(id=2, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=46749, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null)) to broker
localhost:46749 (id: 2 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
Successfully completed leader election (PREFERRED) for partitions
preferred-topic-0 [2023-06-07 16:34:52,184] ERROR [QuorumController id=3000]
writeNoOpRecord: unable to start processing because of
RejectedExecutionException. Reason: The event queue is shutting down
(org.apache.kafka.controller.QuorumController:467) Successfully completed
leader election (PREFERRED) for partitions preferred-topic-0 [2023-06-07
16:34:57,402] ERROR [QuorumController id=0] writeNoOpRecord: unable to start
processing because of RejectedExecutionException. Reason: The event queue is
shutting down (org.apache.kafka.controller.QuorumController:467) [2023-06-07
16:34:57,816] WARN maxCnxns is not configured, using default value 0.
(org.apache.zookeeper.server.ServerCnxnFactory:309) [2023-06-07 16:34:58,192]
WARN No meta.properties file under dir
/tmp/kafka-3825474106858693552/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 16:34:58,563] WARN No
meta.properties file under dir /tmp/kafka-3325525689897519891/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 16:34:58,736] WARN No
meta.properties file under dir /tmp/kafka-1389897181500766185/meta.properties
(kafka.server.BrokerMetadataCheckpoint:70) [2023-06-07 16:34:59,590] WARN
[ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-1. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
16:34:59,591] WARN [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0]
Received UNKNOWN_TOPIC_ID from the leader for partition __consumer_offsets-3.
This error may be returned transiently when the partition is being created or
deleted, but it is not expected to persist.
(kafka.server.ReplicaFetcherThread:70) [2023-06-07 16:34:59,694] WARN
[ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Received UNKNOWN_TOPIC_ID
from the leader for partition __consumer_offsets-0. This error may be returned
transiently when the partition is being created or deleted, but it is not
expected to persist. (kafka.server.ReplicaFetcherThread:70) [2023-06-07
16:34:59,919] WARN [RequestSendThread controllerId=0] Controller 0 epoch 1
fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=45,
partitionStates=[LeaderAndIsrPartitionState(topicName='preferred-topic',
partitionIndex=0, controllerEpoch=1, leader=1, leaderEpoch=0, isr=[1, 2],
partitionEpoch=0, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=true, leaderRecoveryState=0)],
topicIds={preferred-topic=uQFB7kaqSVa-ENEtCww_KA}, liveLeaders=(localhost:37895
(id: 1 rack: null))) to broker localhost:37895 (id: 1 rack: null). Reconnecting
to broker. (kafka.controller.RequestSendThread:72) java.io.IOException:
Connection to 1 was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:34:59,919] WARN [ReplicaFetcher replicaId=2, leaderId=1,
fetcherId=0] Error in response for fetch request (type=FetchRequest,
replicaId=2, maxWait=500, minBytes=1, maxBytes=10485760,
fetchData={__consumer_offsets-2=PartitionData(topicId=ULntyBEjQPGcw04bkvFNKw,
fetchOffset=0, logStartOffset=0, maxBytes=1048576,
currentLeaderEpoch=Optional[0], lastFetchedEpoch=Optional.empty)},
isolationLevel=READ_UNCOMMITTED, removed=, replaced=,
metadata=(sessionId=INVALID, epoch=INITIAL), rackId=)
(kafka.server.ReplicaFetcherThread:72) java.io.IOException: Connection to 1 was
disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.server.BrokerBlockingSender.sendRequest(BrokerBlockingSender.scala:113)
at kafka.server.RemoteLeaderEndPoint.fetch(RemoteLeaderEndPoint.scala:79) at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:316)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:130)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:129)
at scala.Option.foreach(Option.scala:407) at
kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:129)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:112)
at kafka.server.ReplicaFetcherThread.doWork(ReplicaFetcherThread.scala:98) at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,061] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:37895) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:00,064] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37895 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37895 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,166] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:37895) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:00,167] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37895 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37895 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,269] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:37895) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:00,270] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:37895 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:37895 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,290] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=45,
partitionStates=[LeaderAndIsrPartitionState(topicName='preferred-topic',
partitionIndex=0, controllerEpoch=1, leader=1, leaderEpoch=0, isr=[1, 2],
partitionEpoch=0, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=true, leaderRecoveryState=0)],
topicIds={preferred-topic=uQFB7kaqSVa-ENEtCww_KA}, liveLeaders=(localhost:37895
(id: 1 rack: null))) to broker localhost:37895 (id: 1 rack: null). Reconnecting
to broker. (kafka.controller.RequestSendThread:72)
java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
Successfully completed leader election (PREFERRED) for partitions
preferred-topic-0, __consumer_offsets-2 [2023-06-07 16:35:00,967] WARN
[ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Partition
__consumer_offsets-1 marked as failed (kafka.server.ReplicaFetcherThread:70)
[2023-06-07 16:35:00,975] WARN [KafkaServer id=2] Error during controlled
shutdown, possibly because leader movement took longer than the configured
controller.socket.timeout.ms and/or request.timeout.ms: Connection to 0 was
disconnected before the response was read (kafka.server.KafkaServer:70)
[2023-06-07 16:35:00,975] WARN [KafkaServer id=1] Error during controlled
shutdown, possibly because leader movement took longer than the configured
controller.socket.timeout.ms and/or request.timeout.ms: Connection to 0 was
disconnected before the response was read (kafka.server.KafkaServer:70)
[2023-06-07 16:35:00,977] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=StopReplicaRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=26, deletePartitions=false,
topicStates=StopReplicaTopicState(topicName='__consumer_offsets',
partitionStates=[StopReplicaPartitionState(partitionIndex=0, leaderEpoch=0,
deletePartition=false)])) to broker localhost:41409 (id: 0 rack: null).
Reconnecting to broker. (kafka.controller.RequestSendThread:72)
java.io.IOException: Connection to 0 was disconnected before the response was
read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,984] WARN [KafkaServer id=2] Proceeding to do an unclean
shutdown as all the controlled shutdown attempts failed
(kafka.server.KafkaServer:70) [2023-06-07 16:35:00,984] WARN [KafkaServer id=1]
Proceeding to do an unclean shutdown as all the controlled shutdown attempts
failed (kafka.server.KafkaServer:70) [2023-06-07 16:35:00,989] WARN
[RequestSendThread controllerId=0] Controller 0 epoch 1 fails to send request
(type=LeaderAndIsRequest, controllerId=0, controllerEpoch=1, brokerEpoch=62,
partitionStates=[LeaderAndIsrPartitionState(topicName='__consumer_offsets',
partitionIndex=4, controllerEpoch=1, leader=2, leaderEpoch=1, isr=[2],
partitionEpoch=1, replicas=[0, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=0)],
topicIds={__consumer_offsets=ULntyBEjQPGcw04bkvFNKw},
liveLeaders=(localhost:44029 (id: 2 rack: null))) to broker localhost:44029
(id: 2 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to 2
was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,989] WARN [ReplicaFetcher replicaId=1, leaderId=2,
fetcherId=0] Error in response for fetch request (type=FetchRequest,
replicaId=1, maxWait=500, minBytes=1, maxBytes=10485760, fetchData={},
isolationLevel=READ_UNCOMMITTED,
removed=ULntyBEjQPGcw04bkvFNKw:__consumer_offsets-2,
uQFB7kaqSVa-ENEtCww_KA:preferred-topic-0, replaced=,
metadata=(sessionId=1196487040, epoch=1), rackId=)
(kafka.server.ReplicaFetcherThread:72) java.io.IOException: Connection to 2 was
disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.server.BrokerBlockingSender.sendRequest(BrokerBlockingSender.scala:113)
at kafka.server.RemoteLeaderEndPoint.fetch(RemoteLeaderEndPoint.scala:79) at
kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:316)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3(AbstractFetcherThread.scala:130)
at
kafka.server.AbstractFetcherThread.$anonfun$maybeFetch$3$adapted(AbstractFetcherThread.scala:129)
at scala.Option.foreach(Option.scala:407) at
kafka.server.AbstractFetcherThread.maybeFetch(AbstractFetcherThread.scala:129)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:112)
at kafka.server.ReplicaFetcherThread.doWork(ReplicaFetcherThread.scala:98) at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:00,989] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=104,
partitionStates=[LeaderAndIsrPartitionState(topicName='__consumer_offsets',
partitionIndex=2, controllerEpoch=1, leader=1, leaderEpoch=2, isr=[2, 1],
partitionEpoch=3, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=0),
LeaderAndIsrPartitionState(topicName='preferred-topic', partitionIndex=0,
controllerEpoch=1, leader=1, leaderEpoch=2, isr=[2, 1], partitionEpoch=3,
replicas=[1, 2], addingReplicas=[], removingReplicas=[], isNew=false,
leaderRecoveryState=0)], topicIds={__consumer_offsets=ULntyBEjQPGcw04bkvFNKw,
preferred-topic=uQFB7kaqSVa-ENEtCww_KA}, liveLeaders=(localhost:45987 (id: 1
rack: null))) to broker localhost:45987 (id: 1 rack: null). Reconnecting to
broker. (kafka.controller.RequestSendThread:72) java.io.IOException: Connection
to 1 was disconnected before the response was read at
org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:99)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:256)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,081] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:41409) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,082] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:41409 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:41409 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,090] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,091] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,093] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:45987) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,093] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:45987 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:45987 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,184] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:41409) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,184] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:41409 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:41409 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,193] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,193] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,194] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:45987) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,195] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:45987 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:45987 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,286] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:41409) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,286] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:41409 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:41409 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,295] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,295] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,296] WARN [Controller id=0, targetBrokerId=1] Connection
to node 1 (localhost/127.0.0.1:45987) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,297] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:45987 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:45987 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,388] WARN [Controller id=0, targetBrokerId=0] Connection
to node 0 (localhost/127.0.0.1:41409) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,388] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:41409 (id: 0 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:41409 (id: 0 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,397] WARN [Controller id=0, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,398] WARN [Controller id=0, targetBrokerId=1] Connection to node 1
(localhost/127.0.0.1:45987) could not be established. Broker may not be
available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,398] WARN [RequestSendThread controllerId=0] Controller 0's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,398] WARN [RequestSendThread controllerId=0] Controller
0's connection to broker localhost:45987 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:45987 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,429] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=62,
partitionStates=[LeaderAndIsrPartitionState(topicName='__consumer_offsets',
partitionIndex=4, controllerEpoch=1, leader=2, leaderEpoch=1, isr=[2],
partitionEpoch=1, replicas=[0, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=0)],
topicIds={__consumer_offsets=ULntyBEjQPGcw04bkvFNKw},
liveLeaders=(localhost:44029 (id: 2 rack: null))) to broker localhost:44029
(id: 2 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,432] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=LeaderAndIsRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=104,
partitionStates=[LeaderAndIsrPartitionState(topicName='__consumer_offsets',
partitionIndex=2, controllerEpoch=1, leader=1, leaderEpoch=2, isr=[2, 1],
partitionEpoch=3, replicas=[1, 2], addingReplicas=[], removingReplicas=[],
isNew=false, leaderRecoveryState=0),
LeaderAndIsrPartitionState(topicName='preferred-topic', partitionIndex=0,
controllerEpoch=1, leader=1, leaderEpoch=2, isr=[2, 1], partitionEpoch=3,
replicas=[1, 2], addingReplicas=[], removingReplicas=[], isNew=false,
leaderRecoveryState=0)], topicIds={__consumer_offsets=ULntyBEjQPGcw04bkvFNKw,
preferred-topic=uQFB7kaqSVa-ENEtCww_KA}, liveLeaders=(localhost:45987 (id: 1
rack: null))) to broker localhost:45987 (id: 1 rack: null). Reconnecting to
broker. (kafka.controller.RequestSendThread:72) java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,433] WARN [RequestSendThread controllerId=0] Controller 0
epoch 1 fails to send request (type=StopReplicaRequest, controllerId=0,
controllerEpoch=1, brokerEpoch=26, deletePartitions=false,
topicStates=StopReplicaTopicState(topicName='__consumer_offsets',
partitionStates=[StopReplicaPartitionState(partitionIndex=0, leaderEpoch=0,
deletePartition=false)])) to broker localhost:41409 (id: 0 rack: null).
Reconnecting to broker. (kafka.controller.RequestSendThread:72)
java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,469] WARN [Controller id=2, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,470] WARN [Controller id=2, targetBrokerId=1] Connection to node 1
(localhost/127.0.0.1:45987) could not be established. Broker may not be
available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,471] WARN [RequestSendThread controllerId=2] Controller 2's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,472] WARN [RequestSendThread controllerId=2] Controller
2's connection to broker localhost:45987 (id: 1 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:45987 (id: 1 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,539] WARN [RequestSendThread controllerId=2] Controller 2
epoch 2 fails to send request (type: UpdateMetadataRequest=, controllerId=2,
controllerEpoch=2, brokerEpoch=104, partitionStates=[],
liveBrokers=UpdateMetadataBroker(id=2, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=44029, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null),
UpdateMetadataBroker(id=1, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=45987, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null)) to broker
localhost:45987 (id: 1 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,573] WARN [Controller id=2, targetBrokerId=2] Connection
to node 2 (localhost/127.0.0.1:44029) could not be established. Broker may not
be available. (org.apache.kafka.clients.NetworkClient:814) [2023-06-07
16:35:01,574] WARN [RequestSendThread controllerId=2] Controller 2's connection
to broker localhost:44029 (id: 2 rack: null) was unsuccessful
(kafka.controller.RequestSendThread:72) java.io.IOException: Connection to
localhost:44029 (id: 2 rack: null) failed. at
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:70)
at
kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:296)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:249)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127)
[2023-06-07 16:35:01,594] WARN [RequestSendThread controllerId=2] Controller 2
epoch 2 fails to send request (type: UpdateMetadataRequest=, controllerId=2,
controllerEpoch=2, brokerEpoch=62, partitionStates=[],
liveBrokers=UpdateMetadataBroker(id=2, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=44029, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null),
UpdateMetadataBroker(id=1, v0Host='', v0Port=0,
endpoints=[UpdateMetadataEndpoint(port=45987, host='localhost',
listener='PLAINTEXT', securityProtocol=0)], rack=null)) to broker
localhost:44029 (id: 2 rack: null). Reconnecting to broker.
(kafka.controller.RequestSendThread:72) java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at
org.apache.kafka.server.util.ShutdownableThread.pause(ShutdownableThread.java:113)
at
kafka.controller.RequestSendThread.backoff$1(ControllerChannelManager.scala:237)
at
kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:251)
at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:127){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
