[
https://issues.apache.org/jira/browse/STORM-582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiahong Li updated STORM-582:
-----------------------------
Description:
To notice, it is different from STORM-130. It is nimbus to halt. We ran into
this problem several times, every time it happens is after several days of
stable running. Here is the stacktrace
======================================================
2014-11-03 14:30:56 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
stormjar-c1c856f0-cf8b-4299-9c20-712f169802b7.jar
2014-11-12 19:32:33 b.s.d.nimbus [ERROR] Error when processing event
java.io.FileNotFoundException: File
'/tmp/storm-0.9.3/nimbus/stormdist/DNSAnalyse-7-1414992073/stormconf.ser' does
not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
~[commons-io-2.4.jar:2.4]
at
org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763)
~[commons-io-2.4.jar:2.4]
at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:324)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$mk_assignments$iter__3100__3104$fn__3105.invoke(nimbus.clj:649)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30)
~[clojure-1.5.1.jar:na]
at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
~[clojure-1.5.1.jar:na]
at
clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
~[clojure-1.5.1.jar:na]
at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na]
at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na]
at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:648)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287$fn__3288.invoke(nimbus.clj:907)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287.invoke(nimbus.clj:906)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.timer$schedule_recurring$this__2169.invoke(timer.clj:99)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at backtype.storm.timer$mk_timer$fn__2152$fn__2153.invoke(timer.clj:50)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at backtype.storm.timer$mk_timer$fn__2152.invoke(timer.clj:42)
[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:679) [na:1.6.0_22]
2014-11-12 19:32:33 b.s.util [INFO] Halting process: ("Error when processing an
event")
2014-11-12 19:32:33 b.s.d.nimbus [INFO] Shutting down master
=====================================================
To notice, after stable running for 9 days, without printing "Clean up
{storm-id}" logs, nimbus halt.
was:
To notice, it is different from STORM-130. It is nimbus to halt. We ran into
this problem several times, every time it happens is after several days of
stable running. Here is the stacktrace
==========================================================
2014-11-03 14:30:56 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
stormjar-c1c856f0-cf8b-4299-9c20-712f169802b7.jar
2014-11-12 19:32:33 b.s.d.nimbus [ERROR] Error when processing event
java.io.FileNotFoundException: File
'/tmp/storm-0.9.3/nimbus/stormdist/DNSAnalyse-7-1414992073/stormconf.ser' does
not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
~[commons-io-2.4.jar:2.4]
at
org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763)
~[commons-io-2.4.jar:2.4]
at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:324)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$mk_assignments$iter__3100__3104$fn__3105.invoke(nimbus.clj:649)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30)
~[clojure-1.5.1.jar:na]
at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
~[clojure-1.5.1.jar:na]
at
clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
~[clojure-1.5.1.jar:na]
at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na]
at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na]
at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:648)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na]
at
backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287$fn__3288.invoke(nimbus.clj:907)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287.invoke(nimbus.clj:906)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at
backtype.storm.timer$schedule_recurring$this__2169.invoke(timer.clj:99)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at backtype.storm.timer$mk_timer$fn__2152$fn__2153.invoke(timer.clj:50)
~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at backtype.storm.timer$mk_timer$fn__2152.invoke(timer.clj:42)
[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:679) [na:1.6.0_22]
2014-11-12 19:32:33 b.s.util [INFO] Halting process: ("Error when processing an
event")
2014-11-12 19:32:33 b.s.d.nimbus [INFO] Shutting down master
=========================================================
To notice, after stable running for 9 days, without printing "Clean up
{storm-id}" logs, nimbus halt.
> Nimbus Halt with FileNotFoundException: '../nimbus/../stormconf.ser' dose not
> exists
> ------------------------------------------------------------------------------------
>
> Key: STORM-582
> URL: https://issues.apache.org/jira/browse/STORM-582
> Project: Apache Storm
> Issue Type: Bug
> Affects Versions: 0.9.2-incubating
> Reporter: Jiahong Li
>
> To notice, it is different from STORM-130. It is nimbus to halt. We ran into
> this problem several times, every time it happens is after several days of
> stable running. Here is the stacktrace
> ======================================================
> 2014-11-03 14:30:56 b.s.d.nimbus [INFO] Cleaning inbox ... deleted:
> stormjar-c1c856f0-cf8b-4299-9c20-712f169802b7.jar
> 2014-11-12 19:32:33 b.s.d.nimbus [ERROR] Error when processing event
> java.io.FileNotFoundException: File
> '/tmp/storm-0.9.3/nimbus/stormdist/DNSAnalyse-7-1414992073/stormconf.ser'
> does not exist
> at
> org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)
> ~[commons-io-2.4.jar:2.4]
> at
> org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763)
> ~[commons-io-2.4.jar:2.4]
> at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at
> backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:324)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at
> backtype.storm.daemon.nimbus$mk_assignments$iter__3100__3104$fn__3105.invoke(nimbus.clj:649)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na]
> at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na]
> at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na]
> at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na]
> at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30)
> ~[clojure-1.5.1.jar:na]
> at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
> ~[clojure-1.5.1.jar:na]
> at
> clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
> ~[clojure-1.5.1.jar:na]
> at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na]
> at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na]
> at
> backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:648)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na]
> at
> backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287$fn__3288.invoke(nimbus.clj:907)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at
> backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287.invoke(nimbus.clj:906)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at
> backtype.storm.timer$schedule_recurring$this__2169.invoke(timer.clj:99)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at
> backtype.storm.timer$mk_timer$fn__2152$fn__2153.invoke(timer.clj:50)
> ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at backtype.storm.timer$mk_timer$fn__2152.invoke(timer.clj:42)
> [storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
> at java.lang.Thread.run(Thread.java:679) [na:1.6.0_22]
> 2014-11-12 19:32:33 b.s.util [INFO] Halting process: ("Error when processing
> an event")
> 2014-11-12 19:32:33 b.s.d.nimbus [INFO] Shutting down master
> =====================================================
> To notice, after stable running for 9 days, without printing "Clean up
> {storm-id}" logs, nimbus halt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
