Hi Benjamin, Correct me if I missed it , in your config I don't see storm.local.dir defined. If its not defined in config storm will create one in the storm_installation dir which seems to be
/home/bsoulas/incubator-storm-master/storm-dist/binary/target/a pache-storm-0.9.3-ben/apache-storm-0.9.3-ben/ and are you running the supervisor and nimbus as user "bsoulas". When you are running "storm nimbus or storm supervisor" command which storm command its pointing. Did you export STORM_HOME=/home/bsoulas/incubator-storm-master/storm-dist/bina ry/target/apache-storm-0.9.3-ben" and also added it to PATH. I am checking to see if you had any previous installation of storm and invoking the storm command from previous installation. Can you also check zookeeper logs . -Harsha On Tue, Sep 2, 2014, at 03:39 AM, Benjamin SOULAS wrote: Hi everyone, I followed your instructions for installing a zookeeper server, i downloaded it on the website, extract the tar file somewhere in a machine on my cluster, i made those modifications in my zoo.cfg : # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/home/bsoulas/zookeeper/zookeeper-3.4.6/data/ # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # [1]http://zookeeper.apache.org/doc/current/zookeeperAdmin.html# sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 In the log4j.properties, i uncommented the line for the log file : # Example with rolling log file log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE Then i went to my storm.yaml (located here in my case, because i took the source version) : /home/bsoulas/incubator-storm-master/storm-dist/binary/target/a pache-storm-0.9.3-ben/apache-storm-0.9.3-ben/conf This file contain this configuration : ########### These MUST be filled in for a storm configuration storm.zookeeper.servers: - "paradent-4" # - "paradent-47" # - "paradent-48" # nimbus.host: "paradent-4" # # # ##### These may optionally be filled in: # ## List of custom serializations # topology.kryo.register: # - org.mycompany.MyType # - org.mycompany.MyType2: org.mycompany.MyType2Serializer # ## List of custom kryo decorators # topology.kryo.decorators: # - org.mycompany.MyDecorator # ## Locations of the drpc servers # drpc.servers: # - "server1" # - "server2" ## Metrics Consumers # topology.metrics.consumer.register: # - class: "backtype.storm.metric.LoggingMetricsConsumer" # parallelism.hint: 1 # - class: "org.mycompany.MyMetricsConsumer" # parallelism.hint: 1 # argument: # - endpoint: "[2]metrics-collector.mycompany.org" dev.zookeeper.path: "paradent-4.rennes.grid5000.fr:~/home/bsoulas/zookeeper/zookeep er-3.4.6/" storm.zookeeper.port: 2181 To launch storm on the cluster, i launch it thanks to storm nimbus (on a machine named paradent-4), then my zookeeper Server sh zkServer.sh start (on paradent-4 again)(which create a zookeeper_server.pid where the pid of the zookeeper is written, i know it's obvious ...>_< ). After i launch my storm ui for having a visual of my storm app (on paradent-4). Until now, everything work fine. Now, the logical way implies i launch my supervisor, on a different machine (here paradent-39) thanks to storm supervisor, it is launched but once again, 3 or 4 seconds after it's down. So i watched the supervisor.log located : /home/bsoulas/incubator-storm-master/storm-dist/binary/target/a pache-storm-0.9.3-ben/apache-storm-0.9.3-ben/logs And here appear a tricky error : 2014-09-02 09:31:37 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting 2014-09-02 09:31:37 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=paradent-4:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@220df4c8 2014-09-02 09:31:37 o.a.z.ClientCnxn [INFO] Opening socket connection to server [3]paradent-4.rennes.grid5000.fr/172.16.97.4:2181. Will not attempt to authenticate using SASL (unknown error) 2014-09-02 09:31:37 o.a.z.ClientCnxn [INFO] Socket connection established to [4]paradent-4.rennes.grid5000.fr/172.16.97.4:2181, initiating session 2014-09-02 09:31:37 o.a.z.ClientCnxn [INFO] Session establishment complete on server [5]paradent-4.rennes.grid5000.fr/172.16.97.4:2181, sessionid = 0x14835a48ca90004, negotiated timeout = 20000 2014-09-02 09:31:37 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED 2014-09-02 09:31:37 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered. 2014-09-02 09:31:37 b.s.zookeeper [INFO] Zookeeper state update: :connected:none 2014-09-02 09:31:38 o.a.z.ZooKeeper [INFO] Session: 0x14835a48ca90004 closed 2014-09-02 09:31:38 o.a.z.ClientCnxn [INFO] EventThread shut down 2014-09-02 09:31:38 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting 2014-09-02 09:31:38 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=paradent-4:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@c6d625b 2014-09-02 09:31:38 o.a.z.ClientCnxn [INFO] Opening socket connection to server [6]paradent-4.rennes.grid5000.fr/172.16.97.4:2181. Will not attempt to authenticate using SASL (unknown error) 2014-09-02 09:31:38 o.a.z.ClientCnxn [INFO] Socket connection established to [7]paradent-4.rennes.grid5000.fr/172.16.97.4:2181, initiating session 2014-09-02 09:31:38 o.a.z.ClientCnxn [INFO] Session establishment complete on server [8]paradent-4.rennes.grid5000.fr/172.16.97.4:2181, sessionid = 0x14835a48ca90005, negotiated timeout = 20000 2014-09-02 09:31:38 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED 2014-09-02 09:31:38 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered. 2014-09-02 09:31:38 b.s.d.supervisor [INFO] Starting supervisor with id 280caffa-d6c5-4fd4-8282-7d8c1dec7e66 at host [9]paradent-39.rennes.grid5000.fr 2014-09-02 09:31:39 b.s.event [ERROR] Error when processing event java.io.FileNotFoundException: File '/home/bsoulas/storm-local/workers/fc350518-ded6-48f4-abf9-da73 cbaf7c5c/heartbeats/1409146760275' does not exist at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java: 299) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.j ava:1763) ~[commons-io-2.4.jar:2.4] at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at backtype.storm.utils.LocalState.get(LocalState.java:56) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at backtype.storm.daemon.supervisor$read_worker_heartbeat.invoke(s upervisor.clj:77) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at backtype.storm.daemon.supervisor$read_worker_heartbeats$iter__6 381__6385$fn__6386.invoke(supervisor.clj:90) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at clojure.lang.LazySeq.sval(LazySeq.java:42) ~[clojure-1.5.1.jar:na] at clojure.lang.LazySeq.seq(LazySeq.java:60) ~[clojure-1.5.1.jar:na] at clojure.lang.Cons.next(Cons.java:39) ~[clojure-1.5.1.jar:na] at clojure.lang.LazySeq.next(LazySeq.java:92) ~[clojure-1.5.1.jar:na] at clojure.lang.RT.next(RT.java:598) ~[clojure-1.5.1.jar:na] at clojure.core$next.invoke(core.clj:64) ~[clojure-1.5.1.jar:na] at clojure.core$dorun.invoke(core.clj:2781) ~[clojure-1.5.1.jar:na] at clojure.core$doall.invoke(core.clj:2796) ~[clojure-1.5.1.jar:na] at backtype.storm.daemon.supervisor$read_worker_heartbeats.invoke( supervisor.clj:89) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at backtype.storm.daemon.supervisor$read_allocated_workers.invoke( supervisor.clj:106) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at backtype.storm.daemon.supervisor$sync_processes.invoke(supervis or.clj:209) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at clojure.lang.AFn.applyToHelper(AFn.java:161) [clojure-1.5.1.jar:na] at clojure.lang.AFn.applyTo(AFn.java:151) [clojure-1.5.1.jar:na] at clojure.core$apply.invoke(core.clj:619) ~[clojure-1.5.1.jar:na] at clojure.core$partial$fn__4190.doInvoke(core.clj:2396) ~[clojure-1.5.1.jar:na] at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.5.1.jar:na] at backtype.storm.event$event_manager$fn__4687.invoke(event.clj:39 ) ~[storm-core-0.9.3-ben.jar:0.9.3-ben] at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] 2014-09-02 09:31:39 b.s.util [INFO] Halting process: ("Error when processing an event") I understood that there was a missing file, my question is "why?????". If i watch the rights with ls -l at this path : /home/bsoulas/storm-local/workers/fc350518-ded6-48f4-abf9-da73c baf7c5c/ I have this : drwxr-xr-x 2 bsoulas users 4096 Aug 27 15:39 heartbeats So for me this is not the problem, can someone help me? I am really stuck here :S I sincerely hope to be clear and precise enough ... Kind regards. 2014-08-29 16:47 GMT+02:00 Harsha <[10]st...@harsha.io>: Hi Benjamin, Storm cluster needs a zookeeper quorum to function. ExclamationTopology accepts command line params to deploy on a storm cluster. If you don't pass any arguments it will use LocalCluster(a simulated local cluster) to deploy. I recommend you to go through [11]http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html for setting up zookeeper. Here is an excellent write up on storm cluster setup along with zookeeper [12]http://www.michael-noll.com/tutorials/running-mul ti-node-storm-cluster/. Hope that helps. -Harsha On Fri, Aug 29, 2014, at 05:34 AM, Benjamin SOULAS wrote: Hello everyone, i have a problem during implementing storm on a cluster (Grid 5000 if anyone knows). I took the inubator-storm-master from the github branch with the sources, i succeeded to create my own release (no code modification, just for maven errors that were disturbing...) It's working fine on my own laptop in local, i modified the ExclamationTopology in adding 40 more bolts. I also modified this Topology to allow 50 workers in the configuration. Now on a cluster, when I try to do the same thing, supervisors are down just 3s after their execution. Nimbus is ok, dev-zookeeeper too, storm ui too. I read somewhere on the apache website you need to implement a real zookeeper (not the one in storm). Please, does someone knows a good tutorial explaining how running a zookeeper server on a cluster for storm? I hope I am clear ... Kind regards. Benjamin SOULAS References Visible links 1. http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance 2. http://metrics-collector.mycompany.org/ 3. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 4. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 5. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 6. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 7. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 8. http://paradent-4.rennes.grid5000.fr/172.16.97.4:2181 9. http://paradent-39.rennes.grid5000.fr/ 10. mailto:st...@harsha.io 11. http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html 12. http://www.michael-noll.com/tutorials/running-multi-node-storm-cluster/ Hidden links: 14. http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html