[
https://issues.apache.org/jira/browse/KAFKA-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497570#comment-14497570
]
Jason Rosenberg edited comment on KAFKA-2127 at 4/16/15 5:19 AM:
-----------------------------------------------------------------
the ip's that show in the 'Closing socket' and 'Connection reset' log lines
appear to be from hosts which have kafka consumers running, so maybe that's the
explanation. Consumers consuming the topic had their connections closed by the
broker (except in some cases the consumer app gets a rebalance event first,
which causes the connection to be closed remotely first)?
I'm wondering if we can get rid of the ERROR logging for the connection resets
however. Since it is essentially expected behavior when adding/deleting
partitions, that connections will be closed, it seems we don't need to log the
stack trace for the connection reset and log it as ERROR.
was (Author: jbrosenb...@gmail.com):
the ip's that show in the 'Closing socket' and 'Connection reset' log lines
appear to be random hosts in our deployment, hosts which produce messages to
kafka, but are otherwise not part of the kafka cluster or zookeeper, etc.
> Running TopicCommand --alter causes connection close/reset errors in kafka
> logs
> -------------------------------------------------------------------------------
>
> Key: KAFKA-2127
> URL: https://issues.apache.org/jira/browse/KAFKA-2127
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Rosenberg
> Priority: Minor
>
> I am using 0.8.2.1. I've been noticing that any time I use TopicCommand to
> alter a topic (e.g. add partitions) or delete a topic, the broker logs show a
> bunch of closed connections, and usually 2 or 3 Connection reset exceptions.
> It logs these with ERROR status.
> I recently used the kafka.admin.TopicCommand tool to increase the partitions
> for a topic from 1 to 4. So I ran:
> {code}
> java -cp kafka.jar kafka.admin.TopicCommand --zookeeper myzkserver:12345
> --topic mytopic --alter --partitions 4
> {code}
> This resulted in the following sequence in the broker log (repeated pretty
> much in the logs of each broker):
> {code}
> 2015-04-16 03:51:26,156 INFO [kafka-network-thread-27330-1]
> network.Processor - Closing socket connection to /1.2.3.12.
> 2015-04-16 03:51:26,169 INFO [kafka-network-thread-27330-0]
> network.Processor - Closing socket connection to /1.2.3.89.
> 2015-04-16 03:51:26,169 INFO [kafka-network-thread-27330-0]
> network.Processor - Closing socket connection to /1.2.3.95.
> 2015-04-16 03:51:26,176 ERROR [kafka-network-thread-27330-2]
> network.Processor - Closing socket for /1.2.4.34 because of error
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at kafka.utils.Utils$.read(Utils.scala:380)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Processor.read(SocketServer.scala:444)
> at kafka.network.Processor.run(SocketServer.scala:340)
> at java.lang.Thread.run(Thread.java:745)
> 2015-04-16 03:51:26,178 ERROR [kafka-network-thread-27330-1]
> network.Processor - Closing socket for /1.2.4.59 because of error
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at kafka.utils.Utils$.read(Utils.scala:380)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Processor.read(SocketServer.scala:444)
> at kafka.network.Processor.run(SocketServer.scala:340)
> at java.lang.Thread.run(Thread.java:745)
> 2015-04-16 03:51:26,192 ERROR [kafka-network-thread-27330-1]
> network.Processor - Closing socket for /1.2.3.11 because of error
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> at sun.nio.ch.IOUtil.read(IOUtil.java:197)
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> at kafka.utils.Utils$.read(Utils.scala:380)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Processor.read(SocketServer.scala:444)
> at kafka.network.Processor.run(SocketServer.scala:340)
> at java.lang.Thread.run(Thread.java:745)
> 2015-04-16 03:51:26,451 INFO [kafka-request-handler-3]
> server.ReplicaFetcherManager - [ReplicaFetcherManager on broker 45] Removed
> fetcher for partitions [mytopic,2]
> 2015-04-16 03:51:26,453 INFO [kafka-request-handler-3] log.Log - Completed
> load of log mytopic-2 with log end offset 0
> 2015-04-16 03:51:26,454 INFO [kafka-request-handler-3] log.LogManager -
> Created log for partition [mytopic,2] in
> /data_sdd/app/samsa-kafkaserver-ng/kafka_logs_ng with properties
> {segment.index.bytes -> 10485760, file.delete.delay.ms -> 60000,
> segment.bytes -> 1073741824, flush.ms -> 9223372036854775807,
> delete.retention.ms -> 86400000, index.interval.bytes -> 4096,
> retention.bytes -> 50000000000, min.insync.replicas -> 1, cleanup.policy ->
> delete, unclean.leader.election.enable -> true, segment.ms -> 604800000,
> max.message.bytes -> 1000012, flush.messages -> 9223372036854775807,
> min.cleanable.dirty.ratio -> 0.5, retention.ms -> 86400000, segment.jitter.ms
> -> 0}.
> 2015-04-16 03:51:26,454 WARN [kafka-request-handler-3] cluster.Partition -
> Partition [mytopic,2] on broker 45: No checkpointed highwatermark is found
> for partition [mytopic,2]
> 2015-04-16 03:51:26,558 INFO [kafka-network-thread-27330-0]
> network.Processor - Closing socket connection to /1.2.4.34.
> 2015-04-16 03:51:26,658 INFO [kafka-network-thread-27330-0]
> network.Processor - Closing socket connection to /1.2.3.95.
> 2015-04-16 03:51:26,920 INFO [kafka-network-thread-27330-2]
> network.Processor - Closing socket connection to /1.2.4.246.
> 2015-04-16 03:51:26,921 INFO [kafka-network-thread-27330-2]
> network.Processor - Closing socket connection to /1.2.5.4.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
