Hi Jungtaek,
Storm version is 0.9.2-incubating, I am running it locally in a virtual machine
is just test environment.
About the logs from Nimbus where are they stored?
This is all I have in the console from other test:
36863 [main] INFO backtype.storm.daemon.nimbus - Shutting down master
36868 [main] INFO backtype.storm.daemon.nimbus - Shut down master
36872 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN
org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception
org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read
additional data from client sessionid 0x14d15efbfbf0005, likely client has
closed socket
at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
~[zookeeper-3.4.5.jar:3.4.5-1392090]
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[zookeeper-3.4.5.jar:3.4.5-1392090]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_55]
36873 [main] INFO backtype.storm.daemon.supervisor - Shutting down
9bd1c53b-9583-484d-9c0e-6b9208bbe130:a958afaf-04da-472f-a595-969ed155ccf2
36874 [main] INFO backtype.storm.process-simulator - Killing process
8b0be90a-a805-4f46-9fe5-402b16eda600
36874 [main] INFO backtype.storm.daemon.worker - Shutting down worker
sentence-count-1-1430592148 9bd1c53b-9583-484d-9c0e-6b9208bbe130 1024
36874 [main] INFO backtype.storm.daemon.worker - Shutting down receive thread
36875 [main] INFO backtype.storm.messaging.loader - Shutting down
receiving-thread: [sentence-count-1-1430592148, 1024]
36875 [main] INFO backtype.storm.messaging.loader - Waiting for
receiving-thread:[sentence-count-1-1430592148, 1024] to die
36875 [Thread-25-worker-receiver-thread-0] INFO
backtype.storm.messaging.loader -
Receiving-thread:[sentence-count-1-1430592148, 1024] received shutdown notice
36876 [main] INFO backtype.storm.messaging.loader - Shutdown receiving-thread:
[sentence-count-1-1430592148, 1024]
36876 [main] INFO backtype.storm.daemon.worker - Shut down receive thread
36876 [main] INFO backtype.storm.daemon.worker - Terminating messaging context
36877 [main] INFO backtype.storm.daemon.worker - Shutting down executors
36877 [main] INFO backtype.storm.daemon.executor - Shutting down executor
count-bolt:[2 2]
36877 [Thread-7-disruptor-executor[2 2]-send-queue] INFO backtype.storm.util -
Async loop interrupted!
36877 [Thread-8-count-bolt] INFO backtype.storm.util - Async loop interrupted!
36878 [main] INFO backtype.storm.daemon.executor - Shut down executor
count-bolt:[2 2]
36878 [main] INFO backtype.storm.daemon.executor - Shutting down executor
count-bolt:[3 3]
36878 [Thread-10-count-bolt] INFO backtype.storm.util - Async loop interrupted!
36878 [Thread-9-disruptor-executor[3 3]-send-queue] INFO backtype.storm.util -
Async loop interrupted!
36879 [main] INFO backtype.storm.daemon.executor - Shut down executor
count-bolt:[3 3]
36879 [main] INFO backtype.storm.daemon.executor - Shutting down executor
count-bolt:[4 4]
36879 [Thread-11-disruptor-executor[4 4]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36880 [Thread-12-count-bolt] INFO backtype.storm.util - Async loop interrupted!
36880 [main] INFO backtype.storm.daemon.executor - Shut down executor
count-bolt:[4 4]
36880 [main] INFO backtype.storm.daemon.executor - Shutting down executor
report-bolt:[5 5]
36881 [Thread-13-disruptor-executor[5 5]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36881 [Thread-14-report-bolt] INFO backtype.storm.util - Async loop
interrupted!
36881 [main] INFO backtype.storm.daemon.executor - Shut down executor
report-bolt:[5 5]
36882 [main] INFO backtype.storm.daemon.executor - Shutting down executor
report-bolt:[6 6]
36882 [Thread-15-disruptor-executor[6 6]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36882 [Thread-16-report-bolt] INFO backtype.storm.util - Async loop
interrupted!
36883 [main] INFO backtype.storm.daemon.executor - Shut down executor
report-bolt:[6 6]
36883 [main] INFO backtype.storm.daemon.executor - Shutting down executor
report-bolt:[7 7]
36883 [Thread-18-report-bolt] INFO backtype.storm.util - Async loop
interrupted!
36883 [Thread-17-disruptor-executor[7 7]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36884 [main] INFO backtype.storm.daemon.executor - Shut down executor
report-bolt:[7 7]
36884 [main] INFO backtype.storm.daemon.executor - Shutting down executor
sentence-spout:[8 8]
36884 [Thread-20-sentence-spout] INFO backtype.storm.util - Async loop
interrupted!
36884 [Thread-19-disruptor-executor[8 8]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36885 [main] INFO backtype.storm.daemon.executor - Shut down executor
sentence-spout:[8 8]
36885 [main] INFO backtype.storm.daemon.executor - Shutting down executor
__system:[-1 -1]
36885 [Thread-22-__system] INFO backtype.storm.util - Async loop interrupted!
36885 [Thread-21-disruptor-executor[-1 -1]-send-queue] INFO
backtype.storm.util - Async loop interrupted!
36886 [main] INFO backtype.storm.daemon.executor - Shut down executor
__system:[-1 -1]
36886 [main] INFO backtype.storm.daemon.executor - Shutting down executor
__acker:[1 1]
36886 [Thread-24-__acker] INFO backtype.storm.util - Async loop interrupted!
36886 [Thread-23-disruptor-executor[1 1]-send-queue] INFO backtype.storm.util
- Async loop interrupted!
36887 [main] INFO backtype.storm.daemon.executor - Shut down executor
__acker:[1 1]
36887 [main] INFO backtype.storm.daemon.worker - Shut down executors
36887 [main] INFO backtype.storm.daemon.worker - Shutting down transfer thread
36887 [Thread-26-disruptor-worker-transfer-queue] INFO backtype.storm.util -
Async loop interrupted!
36888 [main] INFO backtype.storm.daemon.worker - Shut down transfer thread
36889 [main] INFO backtype.storm.daemon.worker - Shutting down default
resources
36889 [main] INFO backtype.storm.daemon.worker - Shut down default resources
36896 [main] INFO backtype.storm.daemon.worker - Disconnecting from storm
cluster state context
36898 [main] INFO backtype.storm.daemon.worker - Shut down worker
sentence-count-1-1430592148 9bd1c53b-9583-484d-9c0e-6b9208bbe130 1024
36901 [main] INFO backtype.storm.daemon.supervisor - Shut down
9bd1c53b-9583-484d-9c0e-6b9208bbe130:a958afaf-04da-472f-a595-969ed155ccf2
36902 [main] INFO backtype.storm.daemon.supervisor - Shutting down supervisor
9bd1c53b-9583-484d-9c0e-6b9208bbe130
36902 [Thread-3] INFO backtype.storm.event - Event manager interrupted
36903 [Thread-4] INFO backtype.storm.event - Event manager interrupted
36905 [main] INFO backtype.storm.daemon.supervisor - Shutting down supervisor
10e953ec-1832-4f3d-b8f4-b2c1f46857ab
36906 [Thread-5] INFO backtype.storm.event - Event manager interrupted
36906 [Thread-6] INFO backtype.storm.event - Event manager interrupted
36908 [main] INFO backtype.storm.testing - Shutting down in process zookeeper
36909 [main] INFO backtype.storm.testing - Done shutting down in process
zookeeper
36909 [main] INFO backtype.storm.testing - Deleting temporary path
/tmp/d5babc7f-2c83-4e2f-9489-05fe73157fed
36910 [main] INFO backtype.storm.testing - Deleting temporary path
/tmp/1ee9e61c-4496-4e05-ae5b-305d88e3c294
36911 [main] INFO backtype.storm.testing - Deleting temporary path
/tmp/35c83b74-5dce-4efd-9be5-73ee625d3098
36912 [main] INFO backtype.storm.testing - Deleting temporary path
/tmp/edd7365c-334e-40b8-ac29-4eb2812c945f
Thanks for your help,
Juan
On 3 May 2015, at 00:17, 임정택 <[email protected]> wrote:
> Hi.
>
> Could you elavorate it?
>
> - What's your Storm version?
> - Did you run your topology with local mode or cluster mode?
> - Is there no Exception and Stack trace logs from Nimbus?
> - Could you share more log lines, or full Nimbus log file?
>
> Regards
> Jungtaek Lim (HeartSaVioR)
>
>
> 2015-05-03 2:23 GMT+09:00 Joan Perez Esteban <[email protected]>:
> Hi and thanks in advance,
>
> I am trying to perform a log analyser with storm, starting from the beginning
> I am just reading a file and sending lines (spout) receiving bolt just save
> the lines to a new file.
> As simple as that by now, I want to compare origin and destination files and
> check they are the same.
> Problem is that when about 52K are written storm stops with following error:
>
>
> 34870 [main] INFO backtype.storm.daemon.nimbus - Shutting down master
> 34876 [main] INFO backtype.storm.daemon.nimbus - Shut down master
> 34880 [main] INFO backtype.storm.daemon.supervisor - Shutting down
> 74037900-6997-4732-a987-a5f996db5928:b5e20ba0-33eb-4236-bc38-c323a840cac1
> 34880 [main] INFO backtype.storm.process-simulator - Killing process
> db74c56d-6e27-4cc6-91d8-b4ac66a552ae
> 34881 [main] INFO backtype.storm.daemon.worker - Shutting down worker
> sentence-count-1-1430586398 74037900-6997-4732-a987-a5f996db5928 1024
> 34881 [main] INFO backtype.storm.daemon.worker - Shutting down receive thread
> 34881 [main] INFO backtype.storm.messaging.loader - Shutting down
> receiving-thread: [sentence-count-1-1430586398, 1024]
> 34882 [Thread-19-worker-receiver-thread-0] INFO
> backtype.storm.messaging.loader -
> Receiving-thread:[sentence-count-1-1430586398, 1024] received shutdown notice
> 34882 [main] INFO backtype.storm.messaging.loader - Waiting for
> receiving-thread:[sentence-count-1-1430586398, 1024] to die
> 34883 [main] INFO backtype.storm.messaging.loader - Shutdown
> receiving-thread: [sentence-count-1-1430586398, 1024]
> 34883 [main] INFO backtype.storm.daemon.worker - Shut down receive thread
> 34883 [main] INFO backtype.storm.daemon.worker - Terminating messaging
> context
> 34884 [main] INFO backtype.storm.daemon.worker - Shutting down executors
> 34884 [main] INFO backtype.storm.daemon.executor - Shutting down executor
> report-bolt:[2 2]
> 34884 [Thread-8-report-bolt] INFO backtype.storm.util - Async loop
> interrupted!
> 34884 [Thread-7-disruptor-executor[2 2]-send-queue] INFO backtype.storm.util
> - Async loop interrupted!
> 34885 [main] INFO backtype.storm.daemon.executor - Shut down executor
> report-bolt:[2 2]
> 34885 [main] INFO backtype.storm.daemon.executor - Shutting down executor
> report-bolt:[3 3]
> 34885 [Thread-9-disruptor-executor[3 3]-send-queue] INFO backtype.storm.util
> - Async loop interrupted!
> 34886 [Thread-10-report-bolt] INFO backtype.storm.util - Async loop
> interrupted!
> 34886 [main] INFO backtype.storm.daemon.executor - Shut down executor
> report-bolt:[3 3]
> 34886 [main] INFO backtype.storm.daemon.executor - Shutting down executor
> report-bolt:[4 4]
> 34886 [Thread-11-disruptor-executor[4 4]-send-queue] INFO
> backtype.storm.util - Async loop interrupted!
> 34887 [Thread-12-report-bolt] INFO backtype.storm.util - Async loop
> interrupted!
> 34887 [main] INFO backtype.storm.daemon.executor - Shut down executor
> report-bolt:[4 4]
> 34887 [main] INFO backtype.storm.daemon.executor - Shutting down executor
> sentence-spout:[5 5]
> 34887 [Thread-14-sentence-spout] INFO backtype.storm.util - Async loop
> interrupted!
>
>
>
> Any idea why this may be happening?
>
> Rgds,
>
> Juan
>
>
>
> --
> Name : 임 정택
> Blog : http://www.heartsavior.net / http://dev.heartsavior.net
> Twitter : http://twitter.com/heartsavior
> LinkedIn : http://www.linkedin.com/in/heartsavior
