Jiahong Li created STORM-582: -------------------------------- Summary: Nimbus Halt with FileNotFoundException: '../nimbus/../stormconf.ser' dose not exists Key: STORM-582 URL: https://issues.apache.org/jira/browse/STORM-582 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Jiahong Li
To notice, it is different from STORM-130. It is nimbus to halt. We ran into this problem several times, every time it happens is after several days of stable running. Here is the stacktrace ========================================================== 2014-11-03 14:30:56 b.s.d.nimbus [INFO] Cleaning inbox ... deleted: stormjar-c1c856f0-cf8b-4299-9c20-712f169802b7.jar 2014-11-12 19:32:33 b.s.d.nimbus [ERROR] Error when processing event java.io.FileNotFoundException: File '/tmp/storm-0.9.3/nimbus/stormdist/DNSAnalyse-7-1414992073/stormconf.ser' does not exist at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1763) ~[commons-io-2.4.jar:2.4] at backtype.storm.daemon.nimbus$read_storm_conf.invoke(nimbus.clj:89) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:324) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.daemon.nimbus$mk_assignments$iter__3100__3104$fn__3105.invoke(nimbus.clj:649) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na] at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na] at clojure.lang.RT.seq(RT.java:484) ~[clojure-1.5.1.jar:na] at clojure.core$seq.invoke(core.clj:133) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__6026.invoke(protocols.clj:54) ~[clojure-1.5.1.jar:na] at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13) ~[clojure-1.5.1.jar:na] at clojure.core$reduce.invoke(core.clj:6177) ~[clojure-1.5.1.jar:na] at clojure.core$into.invoke(core.clj:6229) ~[clojure-1.5.1.jar:na] at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:648) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at clojure.lang.RestFn.invoke(RestFn.java:410) ~[clojure-1.5.1.jar:na] at backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287$fn__3288.invoke(nimbus.clj:907) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.daemon.nimbus$fn__3281$exec_fn__1205__auto____3282$fn__3287.invoke(nimbus.clj:906) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.timer$schedule_recurring$this__2169.invoke(timer.clj:99) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.timer$mk_timer$fn__2152$fn__2153.invoke(timer.clj:50) ~[storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at backtype.storm.timer$mk_timer$fn__2152.invoke(timer.clj:42) [storm-core-0.9.3-incubating-SNAPSHOT.jar:0.9.3-incubating-SNAPSHOT] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:679) [na:1.6.0_22] 2014-11-12 19:32:33 b.s.util [INFO] Halting process: ("Error when processing an event") 2014-11-12 19:32:33 b.s.d.nimbus [INFO] Shutting down master ========================================================= To notice, after stable running for 9 days, without printing "Clean up {storm-id}" logs, nimbus halt. -- This message was sent by Atlassian JIRA (v6.3.4#6332)