I think the problem is in Storm. Because previous version works fine. Maybe I need to add some security configuration in Storm conf?
thanks, Dmitry. 2015-11-18 10:17 GMT+02:00 Dmitry Sergeev <[email protected]>: > So where you think I need to search? In Zoo configurations? > I've come to so many processes connected to Zoo because of many > services(Kafka, Spark etc.) running on this machine. > > thanks! > > 2015-11-17 15:02 GMT+02:00 Nick R. Katsipoulakis <[email protected]>: > >> Hello again, >> >> The connection is not the problem then. It is clear that Nimbus connects >> properly to ZooKeeper. As I mentioned earlier, the problem is with the >> acess control list you are providing for a given session. By the way, how >> come so many processes are connected to your ZooKeeper and your nimbus is >> not running? >> >> Nick >> >> On Tue, Nov 17, 2015 at 7:55 AM, Dmitry Sergeev <[email protected]> >> wrote: >> >>> Hi Nick, >>> thanks for response. >>> 1. My ZooKeeper instance in running on localhost:5181, here is lsof >>> -i:5181 output: >>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME >>> >>>> java 1267 user 32u IPv6 11008 0t0 TCP *:5181 (LISTEN) >>>> java 1267 user 33u IPv6 12398 0t0 TCP >>>> localhost:5181->localhost:41394 (ESTABLISHED) >>>> java 1267 user 36u IPv6 12402 0t0 TCP >>>> localhost:5181->localhost:41395 (ESTABLISHED) >>>> java 1267 user 37u IPv6 12405 0t0 TCP >>>> localhost:5181->localhost:41396 (ESTABLISHED) >>>> java 1267 user 38u IPv6 12234 0t0 TCP >>>> localhost:5181->localhost:41397 (ESTABLISHED) >>>> java 1267 user 39u IPv6 12236 0t0 TCP >>>> localhost:5181->localhost:41398 (ESTABLISHED) >>>> java 1267 user 40u IPv6 12238 0t0 TCP >>>> localhost:5181->localhost:41399 (ESTABLISHED) >>>> java 1267 user 41u IPv6 12428 0t0 TCP >>>> localhost:5181->localhost:41400 (ESTABLISHED) >>>> java 1267 user 42u IPv6 12665 0t0 TCP >>>> localhost:5181->localhost:41401 (ESTABLISHED) >>>> java 1267 user 43u IPv6 82902 0t0 TCP >>>> localhost:5181->localhost:41467 (ESTABLISHED) >>>> java 1267 user 44u IPv6 13313 0t0 TCP >>>> localhost:5181->localhost:41403 (ESTABLISHED) >>>> java 1267 user 45u IPv6 98702 0t0 TCP >>>> localhost:5181->localhost:41469 (ESTABLISHED) >>>> java 1267 user 46u IPv6 32367 0t0 TCP >>>> localhost:5181->localhost:41456 (ESTABLISHED) >>>> java 1648 user 118u IPv6 12279 0t0 TCP >>>> localhost:41401->localhost:5181 (ESTABLISHED) >>>> java 1648 user 127u IPv6 82901 0t0 TCP >>>> localhost:41467->localhost:5181 (ESTABLISHED) >>>> java 1648 user 132u IPv6 12397 0t0 TCP >>>> localhost:41394->localhost:5181 (ESTABLISHED) >>>> java 1648 user 137u IPv6 12232 0t0 TCP >>>> localhost:41395->localhost:5181 (ESTABLISHED) >>>> java 1648 user 141u IPv6 12404 0t0 TCP >>>> localhost:41396->localhost:5181 (ESTABLISHED) >>>> java 1648 user 145u IPv6 12233 0t0 TCP >>>> localhost:41397->localhost:5181 (ESTABLISHED) >>>> java 1648 user 149u IPv6 12235 0t0 TCP >>>> localhost:41398->localhost:5181 (ESTABLISHED) >>>> java 1648 user 154u IPv6 12237 0t0 TCP >>>> localhost:41399->localhost:5181 (ESTABLISHED) >>>> java 1648 user 158u IPv6 12272 0t0 TCP >>>> localhost:41400->localhost:5181 (ESTABLISHED) >>>> java 1648 user 164u IPv6 12288 0t0 TCP >>>> localhost:41403->localhost:5181 (ESTABLISHED) >>>> java 3893 user 38u IPv6 31428 0t0 TCP >>>> localhost:41456->localhost:5181 (ESTABLISHED) >>>> java 15889 user 138u IPv6 98701 0t0 TCP >>>> localhost:41469->localhost:5181 (ESTABLISHED) >>> >>> 2. nimbus logs: >>> >>>> 2015-11-17 11:28:49.676 o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl [INFO] >>>> Starting >>>> 2015-11-17 11:28:49.676 o.a.s.s.o.a.z.ZooKeeper [INFO] Initiating >>>> client connection, connectString=localhost:5181/storm sessionTimeout=20000 >>>> watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5057ae47 >>>> 2015-11-17 11:28:49.677 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket >>>> connection to server localhost/127.0.0.1:5181. Will not attempt to >>>> authenticate using SASL (unknown error) >>>> 2015-11-17 11:28:49.677 o.a.s.s.o.a.z.ClientCnxn [INFO] Socket >>>> connection established to localhost/127.0.0.1:5181, initiating session >>>> 2015-11-17 11:28:49.679 o.a.s.s.o.a.z.ClientCnxn [INFO] Session >>>> establishment complete on server localhost/127.0.0.1:5181, sessionid = >>>> 0x151153299cd000e, negotiated timeout = 20000 >>>> 2015-11-17 11:28:49.680 o.a.s.s.o.a.c.f.s.ConnectionStateManager [INFO] >>>> State change: CONNECTED >>> >>> Cleaning storm-local directory doesn't help, the problem persists >>> >>> >>> 2015-11-17 14:27 GMT+02:00 Nick R. Katsipoulakis <[email protected]> >>> : >>> >>>> Hello Dmitry, >>>> >>>> KeeperException has to do with ZooKeeper. Why don't you check the >>>> following first: >>>> >>>> >>>> 1. Check if your ZooKeeper instance is running properly and on the >>>> machine:port you are expecting it to run. >>>> 2. Check that your Nimbus attempts to connect to the right ZooKeeper >>>> >>>> The ACL Exception you are getting might be because there is a mismatch >>>> between your Nimbus's previous state and the current one stored on >>>> ZooKeeper. If you verify that Steps 1 and 2 are correct and the problem >>>> persists, I would consider re-initializing Nimbus (erasing the nimbus >>>> directory in your storm-local directory) and restarting ZooKeeper. Keep in >>>> mind that re-initialization might cause the loss of data. Hence, do the >>>> previous only if you are ok with losing any data. >>>> >>>> Cheers, >>>> Nick >>>> >>>> On Tue, Nov 17, 2015 at 6:36 AM, Dmitry Sergeev <[email protected]> >>>> wrote: >>>> >>>>> First, I started Nimbus with >>>>> ${storm.home}/bin/storm nimbus >>>>> It starts successfully >>>>> When I'm trying to start supervisor with this command: >>>>> ${storm.home}/bin/storm supervisor >>>>> It prints JVM options and exit. >>>>> In supervisor.log I see next message: >>>>> >>>>>> 2015-11-17 11:28:56.111 b.s.d.supervisor [ERROR] Error on >>>>>> initialization of server mk-supervisor >>>>>> java.lang.RuntimeException: >>>>>> org.apache.storm.shade.org.apache.zookeeper.KeeperException$InvalidACLException: >>>>>> KeeperErrorCode = InvalidACL for >>>>>> /supervisors/62a290be-610d-4177-9702-58479ebc80d2 >>>>> >>>>> I guess it is problem with security configuration. >>>>> Storm 0.9.4 works fine. >>>>> Can you help me? >>>>> >>>>> Thanks >>>>> >>>> >>>> >>>> >>>> -- >>>> Nick R. Katsipoulakis, >>>> Department of Computer Science >>>> University of Pittsburgh >>>> >>> >>> >> >> >> -- >> Nick R. Katsipoulakis, >> Department of Computer Science >> University of Pittsburgh >> > >
