Is this your own code or did you copy it from somewhere? Most sample code you find online will run in local cluster only for short duration.
See if your code does something like this cluster.submitTopology(); Thread.sleep(1000); cluster.shutdown(); ---> This will bring your process down Srikanth On Sun, May 3, 2015 at 12:05 PM, Joan Perez Esteban <[email protected]> wrote: > Thanks ageing Jungtaek, > Do you happen to know in where should I look at?, or if there are any > parameters I can tune to improve efficiency? maybe is shutting down the > process to save resources. > > rgds, > > Juan > > > On 3 May 2015, at 15:29, 임정택 <[email protected]> wrote: > > I don't find any suspicious things from logs. > Maybe nimbus was being shutdown gracefully (I mean normally), so you would > like to check it. > > Jungtaek Lim (HeartSaVioR) > > 2015-05-03 20:26 GMT+09:00 Joan Perez Esteban <[email protected]>: > >> Hi Jungtaek, >> >> Storm version is 0.9.2-incubating, I am running it locally in a virtual >> machine is just test environment. >> About the logs from Nimbus where are they stored? >> >> This is all I have in the console from other test: >> >> 36863 [main] INFO backtype.storm.daemon.nimbus - Shutting down master >> 36868 [main] INFO backtype.storm.daemon.nimbus - Shut down master >> 36872 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN >> org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception >> org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to >> read additional data from client sessionid 0x14d15efbfbf0005, likely client >> has closed socket >> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220) >> ~[zookeeper-3.4.5.jar:3.4.5-1392090] >> at >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) >> [zookeeper-3.4.5.jar:3.4.5-1392090] >> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55] >> 36873 [main] INFO backtype.storm.daemon.supervisor - Shutting down >> 9bd1c53b-9583-484d-9c0e-6b9208bbe130:a958afaf-04da-472f-a595-969ed155ccf2 >> 36874 [main] INFO backtype.storm.process-simulator - Killing process >> 8b0be90a-a805-4f46-9fe5-402b16eda600 >> 36874 [main] INFO backtype.storm.daemon.worker - Shutting down worker >> sentence-count-1-1430592148 9bd1c53b-9583-484d-9c0e-6b9208bbe130 1024 >> 36874 [main] INFO backtype.storm.daemon.worker - Shutting down receive >> thread >> 36875 [main] INFO backtype.storm.messaging.loader - Shutting down >> receiving-thread: [sentence-count-1-1430592148, 1024] >> 36875 [main] INFO backtype.storm.messaging.loader - Waiting for >> receiving-thread:[sentence-count-1-1430592148, 1024] to die >> 36875 [Thread-25-worker-receiver-thread-0] INFO >> backtype.storm.messaging.loader - >> Receiving-thread:[sentence-count-1-1430592148, 1024] received shutdown >> notice >> 36876 [main] INFO backtype.storm.messaging.loader - Shutdown >> receiving-thread: [sentence-count-1-1430592148, 1024] >> 36876 [main] INFO backtype.storm.daemon.worker - Shut down receive thread >> 36876 [main] INFO backtype.storm.daemon.worker - Terminating messaging >> context >> 36877 [main] INFO backtype.storm.daemon.worker - Shutting down executors >> 36877 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor count-bolt:[2 2] >> 36877 [Thread-7-disruptor-executor[2 2]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36877 [Thread-8-count-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36878 [main] INFO backtype.storm.daemon.executor - Shut down executor >> count-bolt:[2 2] >> 36878 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor count-bolt:[3 3] >> 36878 [Thread-10-count-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36878 [Thread-9-disruptor-executor[3 3]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36879 [main] INFO backtype.storm.daemon.executor - Shut down executor >> count-bolt:[3 3] >> 36879 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor count-bolt:[4 4] >> 36879 [Thread-11-disruptor-executor[4 4]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36880 [Thread-12-count-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36880 [main] INFO backtype.storm.daemon.executor - Shut down executor >> count-bolt:[4 4] >> 36880 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor report-bolt:[5 5] >> 36881 [Thread-13-disruptor-executor[5 5]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36881 [Thread-14-report-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36881 [main] INFO backtype.storm.daemon.executor - Shut down executor >> report-bolt:[5 5] >> 36882 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor report-bolt:[6 6] >> 36882 [Thread-15-disruptor-executor[6 6]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36882 [Thread-16-report-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36883 [main] INFO backtype.storm.daemon.executor - Shut down executor >> report-bolt:[6 6] >> 36883 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor report-bolt:[7 7] >> 36883 [Thread-18-report-bolt] INFO backtype.storm.util - Async loop >> interrupted! >> 36883 [Thread-17-disruptor-executor[7 7]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36884 [main] INFO backtype.storm.daemon.executor - Shut down executor >> report-bolt:[7 7] >> 36884 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor sentence-spout:[8 8] >> 36884 [Thread-20-sentence-spout] INFO backtype.storm.util - Async loop >> interrupted! >> 36884 [Thread-19-disruptor-executor[8 8]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36885 [main] INFO backtype.storm.daemon.executor - Shut down executor >> sentence-spout:[8 8] >> 36885 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor __system:[-1 -1] >> 36885 [Thread-22-__system] INFO backtype.storm.util - Async loop >> interrupted! >> 36885 [Thread-21-disruptor-executor[-1 -1]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36886 [main] INFO backtype.storm.daemon.executor - Shut down executor >> __system:[-1 -1] >> 36886 [main] INFO backtype.storm.daemon.executor - Shutting down >> executor __acker:[1 1] >> 36886 [Thread-24-__acker] INFO backtype.storm.util - Async loop >> interrupted! >> 36886 [Thread-23-disruptor-executor[1 1]-send-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36887 [main] INFO backtype.storm.daemon.executor - Shut down executor >> __acker:[1 1] >> 36887 [main] INFO backtype.storm.daemon.worker - Shut down executors >> 36887 [main] INFO backtype.storm.daemon.worker - Shutting down transfer >> thread >> 36887 [Thread-26-disruptor-worker-transfer-queue] INFO >> backtype.storm.util - Async loop interrupted! >> 36888 [main] INFO backtype.storm.daemon.worker - Shut down transfer >> thread >> 36889 [main] INFO backtype.storm.daemon.worker - Shutting down default >> resources >> 36889 [main] INFO backtype.storm.daemon.worker - Shut down default >> resources >> 36896 [main] INFO backtype.storm.daemon.worker - Disconnecting from >> storm cluster state context >> 36898 [main] INFO backtype.storm.daemon.worker - Shut down worker >> sentence-count-1-1430592148 9bd1c53b-9583-484d-9c0e-6b9208bbe130 1024 >> 36901 [main] INFO backtype.storm.daemon.supervisor - Shut down >> 9bd1c53b-9583-484d-9c0e-6b9208bbe130:a958afaf-04da-472f-a595-969ed155ccf2 >> 36902 [main] INFO backtype.storm.daemon.supervisor - Shutting down >> supervisor 9bd1c53b-9583-484d-9c0e-6b9208bbe130 >> 36902 [Thread-3] INFO backtype.storm.event - Event manager interrupted >> 36903 [Thread-4] INFO backtype.storm.event - Event manager interrupted >> 36905 [main] INFO backtype.storm.daemon.supervisor - Shutting down >> supervisor 10e953ec-1832-4f3d-b8f4-b2c1f46857ab >> 36906 [Thread-5] INFO backtype.storm.event - Event manager interrupted >> 36906 [Thread-6] INFO backtype.storm.event - Event manager interrupted >> 36908 [main] INFO backtype.storm.testing - Shutting down in process >> zookeeper >> 36909 [main] INFO backtype.storm.testing - Done shutting down in process >> zookeeper >> 36909 [main] INFO backtype.storm.testing - Deleting temporary path >> /tmp/d5babc7f-2c83-4e2f-9489-05fe73157fed >> 36910 [main] INFO backtype.storm.testing - Deleting temporary path >> /tmp/1ee9e61c-4496-4e05-ae5b-305d88e3c294 >> 36911 [main] INFO backtype.storm.testing - Deleting temporary path >> /tmp/35c83b74-5dce-4efd-9be5-73ee625d3098 >> 36912 [main] INFO backtype.storm.testing - Deleting temporary path >> /tmp/edd7365c-334e-40b8-ac29-4eb2812c945f >> >> >> Thanks for your help, >> >> Juan >> >> >> On 3 May 2015, at 00:17, 임정택 <[email protected]> wrote: >> >> Hi. >> >> Could you elavorate it? >> >> - What's your Storm version? >> - Did you run your topology with local mode or cluster mode? >> - Is there no Exception and Stack trace logs from Nimbus? >> - Could you share more log lines, or full Nimbus log file? >> >> Regards >> Jungtaek Lim (HeartSaVioR) >> >> >> 2015-05-03 2:23 GMT+09:00 Joan Perez Esteban <[email protected]>: >> >>> Hi and thanks in advance, >>> >>> I am trying to perform a log analyser with storm, starting from the >>> beginning I am just reading a file and sending lines (spout) receiving bolt >>> just save the lines to a new file. >>> As simple as that by now, I want to compare origin and destination files >>> and check they are the same. >>> Problem is that when about 52K are written storm stops with following >>> error: >>> >>> >>> 34870 [main] INFO backtype.storm.daemon.nimbus - Shutting down master >>> 34876 [main] INFO backtype.storm.daemon.nimbus - Shut down master >>> 34880 [main] INFO backtype.storm.daemon.supervisor - Shutting down >>> 74037900-6997-4732-a987-a5f996db5928:b5e20ba0-33eb-4236-bc38-c323a840cac1 >>> 34880 [main] INFO backtype.storm.process-simulator - Killing process >>> db74c56d-6e27-4cc6-91d8-b4ac66a552ae >>> 34881 [main] INFO backtype.storm.daemon.worker - Shutting down worker >>> sentence-count-1-1430586398 74037900-6997-4732-a987-a5f996db5928 1024 >>> 34881 [main] INFO backtype.storm.daemon.worker - Shutting down receive >>> thread >>> 34881 [main] INFO backtype.storm.messaging.loader - Shutting down >>> receiving-thread: [sentence-count-1-1430586398, 1024] >>> 34882 [Thread-19-worker-receiver-thread-0] INFO >>> backtype.storm.messaging.loader - >>> Receiving-thread:[sentence-count-1-1430586398, 1024] received shutdown >>> notice >>> 34882 [main] INFO backtype.storm.messaging.loader - Waiting for >>> receiving-thread:[sentence-count-1-1430586398, 1024] to die >>> 34883 [main] INFO backtype.storm.messaging.loader - Shutdown >>> receiving-thread: [sentence-count-1-1430586398, 1024] >>> 34883 [main] INFO backtype.storm.daemon.worker - Shut down receive >>> thread >>> 34883 [main] INFO backtype.storm.daemon.worker - Terminating messaging >>> context >>> 34884 [main] INFO backtype.storm.daemon.worker - Shutting down executors >>> 34884 [main] INFO backtype.storm.daemon.executor - Shutting down >>> executor report-bolt:[2 2] >>> 34884 [Thread-8-report-bolt] INFO backtype.storm.util - Async loop >>> interrupted! >>> 34884 [Thread-7-disruptor-executor[2 2]-send-queue] INFO >>> backtype.storm.util - Async loop interrupted! >>> 34885 [main] INFO backtype.storm.daemon.executor - Shut down executor >>> report-bolt:[2 2] >>> 34885 [main] INFO backtype.storm.daemon.executor - Shutting down >>> executor report-bolt:[3 3] >>> 34885 [Thread-9-disruptor-executor[3 3]-send-queue] INFO >>> backtype.storm.util - Async loop interrupted! >>> 34886 [Thread-10-report-bolt] INFO backtype.storm.util - Async loop >>> interrupted! >>> 34886 [main] INFO backtype.storm.daemon.executor - Shut down executor >>> report-bolt:[3 3] >>> 34886 [main] INFO backtype.storm.daemon.executor - Shutting down >>> executor report-bolt:[4 4] >>> 34886 [Thread-11-disruptor-executor[4 4]-send-queue] INFO >>> backtype.storm.util - Async loop interrupted! >>> 34887 [Thread-12-report-bolt] INFO backtype.storm.util - Async loop >>> interrupted! >>> 34887 [main] INFO backtype.storm.daemon.executor - Shut down executor >>> report-bolt:[4 4] >>> 34887 [main] INFO backtype.storm.daemon.executor - Shutting down >>> executor sentence-spout:[5 5] >>> 34887 [Thread-14-sentence-spout] INFO backtype.storm.util - Async loop >>> interrupted! >>> >>> >>> >>> Any idea why this may be happening? >>> >>> Rgds, >>> >>> Juan >> >> >> >> >> -- >> Name : 임 정택 >> Blog : http://www.heartsavior.net / http://dev.heartsavior.net >> Twitter : http://twitter.com/heartsavior >> LinkedIn : http://www.linkedin.com/in/heartsavior >> >> >> > > > -- > Name : 임 정택 > Blog : http://www.heartsavior.net / http://dev.heartsavior.net > Twitter : http://twitter.com/heartsavior > LinkedIn : http://www.linkedin.com/in/heartsavior > > >
