Hi Sam, Have you tried "netstat" and "lsof" commands recommended by Erik? What is the result? We need to find which process bound this port, then kill the process to release this port.
Regards Basti -----Original Message----- From: sam mohel [mailto:[email protected]] Sent: Friday, December 11, 2015 8:15 AM To: [email protected] Subject: Re: Problem with storm since 4 months > now i converted the change but problem still > i mean in the link you sent , he said "the ephermal port range is specified in /proc/sys/net/ipv4/ip_local_ port_range. You can probably extend it to run from 16k to 64k." how can i do that ? > how can i check if there is anothe dump worker working , or is there another way to kill any worker if it is dump or not "completely" and start from scratch thanks for your patience and your time On Thu, Dec 10, 2015 at 7:38 PM, Erik Weathers < [email protected]> wrote: > responses inline. > > >Ah, so, you shouldn't have touched the "local port range" setting > >since > you > don't fully understand ephemeral ports and TCP yet! :-) > > I'll give a brief synopsis: > > Say you're making a connection from a client to a server that listens > on port 80. The client needs to have a port of its own to receive the > response packets from the server. The port that is allocated for it by > the TCP stack is a "random" port, which is called an "ephemeral" port in this > context. > So with your original default config (32768 61000) the port would be > chosen from an available port on the machine that lies within that > range of > 32768->61000. That's good, it wouldn't conflict with the default TCP > 32768->ports > being listened to by the Storm Worker processes (67xx). So 1. isn't > your problem. But now you've made it possible for it to become a > problem, since now the range of ports that can be given as an > ephemeral port overlaps with the default Storm Worker ports > (1024->65535 includes 67xx). So you should revert that config change. > > Thanks for your replying , now i reverted the change > > >Sorry, I don't understand what you are asking. What thing is "16" > >that > you > are trying to extend? > > i mean in the link you sent , he adviced to extend from 16 to 64 > >You can kill topologies from the Nimbus UI (web page). Or with the > bin/storm command. > > > >The commands are *solely* intended to figure out what is conflicting. > Your > logs claim that there is something holding onto 67xx which prevents the > Storm Worker from launching. So if that is happening you should > *immediately* try to figure out what is actually holding onto the port > and preventing your Storm Worker from launching. > > > >Not sure what you mean here. > > Please note that Storm often suffers from "cascading" failures, where > there are a lot of exceptions and errors that aren't actually the root > cause of the problem. Often you need to spend time and effort looking > at lots of logs and tracing back to the real root cause. > > > > > > > > Really , Thanks for your time > > > > On Thu, Dec 10, 2015 at 6:22 AM, Erik Weathers < > > [email protected]> wrote: > > > > > Regarding Basti's suggestion (1.) that your host's configured > > > ephemeral ports might be conflicting with the storm worker ports, > > > here's how you > > can > > > check your "local port range" setting: > > > > > > > > > > > > https://serverfault.com/questions/261663/on-linux-how-can-i-tell-how-m > any-ephemeral-ports-are-left-available > > > > > > % cat /proc/sys/net/ipv4/ip_local_port_range > > > > > > > > > It's possible that there is a zombie worker process holding onto > > > port > > 6703. > > > I would try to identify the process like so: > > > > > > % sudo netstat -ap --numeric-ports --extend | grep -w LISTEN | > > > grep -w > > 6703 > > > > > > Alternatively you can try a global lsof search: > > > > > > % sudo lsof | grep TCP | grep -w LISTEN | grep -w 6703 > > > > > > - Erik > > > > > > > > > > > > On Wed, Dec 9, 2015 at 7:37 PM, 刘键(Basti Liu) < > [email protected]> > > > wrote: > > > > > > > Hi Sam, > > > > > > > > You can try to find which process has bound this port by > > > > "netstat > -anp" > > > > first. > > > > > > > > Generally, there are following cases for the binding error. > > > > 1. "local port range" is not set to exclude the port range used > > > > in > > Storm. > > > > 2. The previous worker was not killed correctly. > > > > 3. There is bug of assignment in some scenarios. Same port was > assigned > > > to > > > > two workers. > > > > > > > > Regards > > > > Basti > > > > > > > > -----Original Message----- > > > > From: sam mohel [mailto:[email protected]] > > > > Sent: Thursday, December 10, 2015 7:16 AM > > > > To: [email protected] > > > > Subject: Re: Problem with storm since 4 months > > > > > > > > i tried to use storm-0.9.5 but problem changed with > > > > > > > > cannot bind port 6703 i think it's same problem > > > > > > > > On Wed, Dec 9, 2015 at 8:42 PM, Harsha <[email protected]> wrote: > > > > > > > > > Sam, > > > > > you might be using very old version of storm since > > > > > its > > > showing > > > > > ZeroMQ. Can you try using newer version storm > > > > > without > zero > > > mq. > > > > > -Harsha > > > > > > > > > > On Wed, Dec 9, 2015, at 10:19 AM, sam mohel wrote: > > > > > > I have this problem since 4months when I submitted topology > > > > > > I got this in the worker log file [ERROR] Async loop died! > > > > org.zeromq.ZMQException: > > > > > > Address already in use(0x62) at > > > > > > org.zeromq.ZMQ$Socket.bind(Native Method) at > > > > > > zilch.mq$bind.invoke(mq.clj:69) at > > > > > > backtype.storm.messaging.zmq.ZMQContext.bind(zmq.clj:57)at > > > > > > > > > > > > > backtype.storm.messaging.loader$launch_receive_thread_BANG_$fn__1629 > > .i > > > > > nvoke(loader.clj:26) > > > > > > at > > > > > > backtype.storm.util$async_loop$fn__465.invoke(util.clj:375) > > > > > > at clojure.lang.AFn.run(AFn.java:24) at > > java.lang.Thread.run(Unknown > > > > > > Source) > > > > > > > > > > > > when i tried to connect port 6703 and 6702 > > > > > > > > > > > > And supervisor log file hadn't still start > > > > > > > > > > > > > > > > > > I searched everywhere but cannot find any solution I hope > > > > > > you can > > > > > > > > > > > > > > > > > > >
