[
https://issues.apache.org/jira/browse/STORM-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353645#comment-15353645
]
Noppanit Charassinvichai commented on STORM-973:
------------------------------------------------
I got the same problem but in my case one of the worker was just too busy doing
GC. I increased the memory and things are working fine.
> Netty-Client Connection Failed
> ------------------------------
>
> Key: STORM-973
> URL: https://issues.apache.org/jira/browse/STORM-973
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
> Affects Versions: 0.9.4
> Environment: apache-storm-0.9.4 JDK 1.7.0_75
> Reporter: darion yaphet
>
> When Storm Topology startup in a distribution cluster I found netty
> connection will failed and messages will be droped by client itself.
> worker log info as following :
> ```
> 2015-08-07T11:43:18.903+0800 b.s.m.n.StormClientErrorHandler [INFO]
> Connection failed Netty-Client-storm-01
> java.io.IOException: Connection reset by peer
> at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_75]
> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
> ~[na:1.7.0_75]
> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
> ~[na:1.7.0_75]
> at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[na:1.7.0_75]
> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
> ~[na:1.7.0_75]
> at
> org.apache.storm.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> [storm-core-0.9.4.jar:0.9.4]
> at
> org.apache.storm.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> [storm-core-0.9.4.jar:0.9.4]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_75]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_75]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
> 2015-08-07T11:43:19.426+0800 b.s.m.n.Client [INFO] connection attempt 1 to
> Netty-Client-syq-storm-01.meilishuo.com/172.16.7.25:8711 scheduled to run in
> 0 ms
> 2015-08-07T11:43:19.427+0800 b.s.m.n.Client [ERROR] connection to
> Netty-Client-storm-01 is unavailable
> 2015-08-07T11:43:19.427+0800 b.s.m.n.Client [ERROR] dropping 1 message(s)
> destined for Netty-Client-storm-01
> 2015-08-07T11:43:19.428+0800 b.s.m.n.Client [ERROR] connection to
> Netty-Client-storm-01 is unavailable
> 2015-08-07T11:43:19.428+0800 b.s.m.n.Client [ERROR] dropping 103 message(s)
> destined for Netty-Client-storm-01
> 2015-08-07T11:43:19.428+0800 b.s.m.n.Client [ERROR] connection to
> Netty-Client-storm-01 is unavailable
> 2015-08-07T11:43:19.428+0800 b.s.m.n.Client [ERROR] dropping 35 message(s)
> destined for Netty-Client-storm-01
> ```
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
