Re: storm crashes when one of the zookeeper server dies

2014-07-04 Thread 이승진
Actually, it was not just a zookeeper process down, but whole machine dies due to kernel panic, and ping to that server failed accordingly.All servers were connected to each other and in replication mode, of course. -Original Message- From: Irek Khasyanovlt;qua...@gmail.comgt; To:

RE: storm crashes when one of the zookeeper server dies

2014-07-04 Thread Simon Cooper
Was that machine also running storm nimbus? Nimbus is currently a sort-of SPOF, but it’s not really; if nimbus goes down the supervisors continue running the topologies as before. There is currently an issue to implement a high-availability nimbus at

Choosing where your tasks run in Storm

2014-07-04 Thread jeff saremi
I'm wondering if this concept applies to Storm and if there's a way to do this. I'd like to limit the machines that certain spouts or bolts run on. There are many reasons for this. But for one let's assume that I have a bolt that is just a proxy for some legacy service. I want to monitor that

Re: Choosing where your tasks run in Storm

2014-07-04 Thread Michael Rose
You can make it happen with a custom scheduler, see this article (sorry for mangling, getting this link through SpamAssassin on the group was a nightmare): http xumingming dot sinaapp dotcom slash 885/twitter-storm-how-to-develop-a-pluggable-scheduler/ But it's nothing I've seriously attempted

RE: Choosing where your tasks run in Storm

2014-07-04 Thread jeff saremi
Michael and Andrew. thanks so much Date: Fri, 4 Jul 2014 14:41:17 -0600 Subject: Re: Choosing where your tasks run in Storm From: mich...@fullcontact.com To: user@storm.incubator.apache.org You can make it happen with a custom scheduler, see this article (sorry for mangling, getting this link

Re: ShellSpout Cannot Execute Arbitrary Command

2014-07-04 Thread Alex Merritt
In ShellProcess.java, method launch, this line seems to cause the IOException: builder.directory(new File(context.getCodeDir())); The path it is attempting to use is /tmp/c86c165a-762b-42aa-8238-ea291f14c577/supervisor/stormdist/stitcher-1-1404517781/resources (my topology name is 'stitcher')

Re: Unit testing topologies

2014-07-04 Thread Eugene
I did session at storm meetup in NYC about testing storm topologies. I personally use Groovy and Spock for all my testing. Groovy is easy to pick up for any java programmer and provides big productivity boost in writing tests.. Spock practically replacing junit, mockito and have BDD style of