Hi,
Al right, I am running kafka from an Debian Linux operating system – no docker images are involved. I just can exclude any problems with my producer application – the following command fails too: /home/kafka/kafka/bin/kafka-topics.sh --list --bootstrap-server 127.0.0.1:9092 Error while executing topic command : org.apache.kafka.common.errors.TimeoutException: Call(callName=listTopics, deadlineMs=1594206614222) timed out at 1594206614223 after 1 attempt(s) [2020-07-08 13:10:14,227] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Call(callName=listTopics, deadlineMs=1594206614222) timed out at 1594206614223 after 1 attempt(s) at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45) at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32) at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89) at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260) at kafka.admin.TopicCommand$AdminClientTopicService.getTopics(TopicCommand.scala:333) at kafka.admin.TopicCommand$AdminClientTopicService.listTopics(TopicCommand.scala:252) at kafka.admin.TopicCommand$.main(TopicCommand.scala:66) at kafka.admin.TopicCommand.main(TopicCommand.scala) Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=listTopics, deadlineMs=1594206614222) timed out at 1594206614223 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. (kafka.admin.TopicCommand$) This means, that there is a connection problem with the client, supporting what you had said. I just deactivated the firewall (ufw in this case), but it does not look better. Could you please suggest any options for further investigations? Is there any logging that could help? In the kafka log I found the following exception: 2020-07-08 13:09:33,066] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:09:39,072] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:09:45,075] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:09:51,079] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:09:57,085] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:03,088] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:09,092] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:14,239] DEBUG [SocketServer brokerId=1] Connection with /127.0.0.1 disconnected (org.apache.kafka.common.network.Selector) java.io.EOFException at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:97) at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448) at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398) at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580) at org.apache.kafka.common.network.Selector.poll(Selector.java:485) at kafka.network.Processor.poll(SocketServer.scala:861) at kafka.network.Processor.run(SocketServer.scala:760) at java.lang.Thread.run(Thread.java:748) [2020-07-08 13:10:15,096] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:21,099] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:27,104] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:33,108] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:39,112] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:45,115] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:51,120] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:10:57,124] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:03,131] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:09,136] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:15,140] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:21,144] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:27,148] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:33,152] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:39,155] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:45,162] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:51,168] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:11:57,172] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:03,175] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:09,181] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:15,187] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:21,193] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:27,199] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 0ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:33,204] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:39,207] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:45,212] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:51,216] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:12:57,220] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:03,226] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:09,231] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:15,236] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:21,240] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:27,246] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) [2020-07-08 13:13:33,251] DEBUG Got ping response for sessionid: 0x100003f85c80023 after 1ms (org.apache.zookeeper.ClientCnxn) Kind regards, Sebastian Von: Ricardo Ferreira <rifer...@riferrei.com> Gesendet: Dienstag, 7. Juli 2020 17:58 An: users@kafka.apache.org; s.schabb...@fluent-software.de Betreff: Re: Problem with replication?! Given the stack trace you've shared below I can tell that this is not a replication issue but rather -- your producer is not being able to write records into the partitions because the brokers that host them are unavailable. Now, I know that they are indeed running so "unavailable" here means that from the network perspective your client app is not being able to establish a TCP connection with them. Firewall is something you should look into, as well as SELinux, and Docker networking (if this is running on Docker). Thanks, -- Ricardo On 7/6/20 10:37 AM, s.schabb...@fluent-software.de <mailto:s.schabb...@fluent-software.de> wrote: Hi there, I just have a problem with my kafka brokers, maybe a firewall issue, but I don’t know. I have got 3 Brokers at three different Servers (each with another IP) and on the first server running zookeeper: Server1:9092 (zookeeper:2182) Server2:9092 Server3:9092 And I have got a topic with a replication factor of three. If I try to publish new messages to that topic, I got the following error: Confluent.Kafka.ProduceException`2[System.Byte[],System.String]: Local: Message timed out at Confluent.Kafka.Producer`2.ProduceAsync(TopicPartition topicPartition, Message`2 message, CancellationToken cancellationToken) at FluentSoftware.EventSourcing.KafkaProducerEventPublisher.PublishEvents[TEvent](String topic, IEnumerable`1 eventsToApply) at FluentSoftware.EventSourcing.CommandBus.SendAsync[TCommand](TCommand command) at Heliprinter.Mediator.Communication.OrderService.Controllers.OrderController.Post(PostOrderViewModel newOrder) in C:\agent\_work\15\s\Source\Services\Heliprinter.Mediator.Communication.OrderService\Controllers\OrderController.cs:line 55 and the REST API responses with an Gateway timeout. Could anyone tell me * If this error could be an replication issue? * How I can debug these issue? * And where I can find replication details in the logs? Kind regards, Sebastian.