Supervisor is down after running topology for a while

The error info in supervior.log: 2016-07-19 11:58:41.508 o.a.s.d.supervisor [INFO] 80c00622-5d5d-4221-9307-f418036db4a0 still hasn't started 2016-07-19 11:58:42.008 o.a.s.d.supervisor [INFO] 80c00622-5d5d-4221-9307-f418036db4a0 still hasn't started 2016-07-19 11:58:42.509 o.a.s.d.supervisor [INFO

Re: Doesn't every supervisor have a default 4 slots?

Any help please? On Mon, Jul 18, 2016 at 3:49 PM, Navin Ipe wrote: > Hi, > > On my local system I was able to run 5 topologies only when I increased > the number of slots in storm.yaml, to 5. > > But when submitting my jar to storm on an Amazon node, even though I had 5 > supervisor nodes runnin

Re: Question on Storm 1.0 State Management

Thanks Arun for the clarification. A follow up question on the same when Increasing/Decreasing the Parallelism of a Bolt. If I use field grouping on some ID field and I track state in Bolt for that ID, after a parallelism increase, it is possible to have the state maintained in a different bolt in

Re: Question on Storm 1.0 State Management

Each bolt instance (task) has its own state, so in your case each of the 5 instances would have its own state. All these state instances could be in a same underlying storage instance (e.g. same Redis cluster). Thanks, Arun From: Jins George Reply-To: "user@storm.apache.org" Date

Re: Storm Benchmarks

Thank you Anshu. I tried that but it doesn't work. Same error: Exception in thread "main" java.lang.NoClassDefFoundError: backtype/storm/generated/AlreadyAliveException. Regards WA From: anshu shukla To: user@storm.apache.org Cc: Walid Aljoby ; "d...@storm.apache.org" Sent: Tuesday, J

What is storm_env.ini?

Using Storm 0.10.0. I did not see docs on this in http://storm.apache.org/releases/0.10.0/Setting-up-a-Storm-cluster.html, but I see it in the python bootstrap code. I was wondering what this is? - Joaquin -- 是故勝兵先勝而後求戰，敗兵先戰而後求勝。

Re: Need help with Unknown Error, not sure

Your supervisor's local host name is not getting resolved. You can override this by configuring storm.local.hostname with a valid hostname. Thanks, Satish. On Tue, Jul 19, 2016 at 12:22 AM, Joaquin Menchaca wrote: > Hi. > > Anyone have any suggestions how to debug this and find out what is > h

ApacheCon: Getting the word out internally

ApacheCon: Getting the word out internally Dear Apache Enthusiast, As you are no doubt already aware, we will be holding ApacheCon in Seville, Spain, the week of November 14th, 2016. The call for papers (CFP) for this event is now open, and will remain open until September 9th. The event is divid

Storm 0.10.0 "storm nimbus" ignores storm.yaml configuration?

This may be obvious, but just cannot see it now. I launch my nimbus process using */usr/local/bin/storm nimbus* This is a symlink to */usr/lib/apache/storm/0.10.0/bin/storm* My *storm.yaml* is in */usr/lib/apache/storm/0.10.0/conf* I see that it list it has one parameter as *-Dstorm.conf.file *wi

Question on Storm 1.0 State Management

Hello Group, I have a question on the implementation of State management in Storm(Storm 1.0 - IStatefulBolt) . Is the sate is shared across instances of same Bolt ? Say I have a bolt with parallelism as 5, Does each bolt instance have its own copy of State or there is only one State which is s

Storm 0.10.0 - Zookeeper Connection ignoring configuration?

I'm trying to understand the Zk interaction, is it trying to really run on 127.0.0.1? 3959 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x155fffd30df0004 closed 3959 [main] INFO b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [3] the maxRetries [5]

Need help with Unknown Error, not sure

Hi. Anyone have any suggestions how to debug this and find out what is happening? Any troubleshooting tools I can use to test the functionality of the storm cluster? 3899 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED 3908 [main] INFO b.s.d.superviso

Re: Storm Benchmarks

This was suggested from someone in mailing list only . I have tested that already it works . You can run the old code with the newer version by putting up this config in storm.yaml of client (machine where you submit the topologies from) - client.jartransformer.class: "org.apache.storm.hack.StormS

Re: Storm Benchmarks

I don't think storm-benchmark can be run against storm 1.0.1 without code changes. Yes, both computation and communication parts are included. The metrics are collected by storm nimbus including end-to-end throughput and latency. I know nothing more than you on that micro-benchmark. Maybe + ptgo...

Re: How to clean up Storm logs

Yes, we can do that .. actually, I used to do the same .You can even delete the whole logs directory . On Mon, Jul 18, 2016 at 9:32 PM, I PVP wrote: > Can I just setup a script to archive and delete the directories/files on > $STORM_HOME/logs/workers-artifacts that have the last modified date i

Re: File processing using trident

Hi Sherwin,For 1 yes you can increase the parallelism to be same as number of kafka partitions.  2. depends on TridentKafkaUpdater and Kafka configurations. I am not sure about that. 3. You can write your own updater function and do a streaming update. You will need to take care of duplicate rec

Re: How to clean up Storm logs

Can I just setup a script to archive and delete the directories/files on $STORM_HOME/logs/workers-artifacts that have the last modified date in the past ( x days ago)? Or would that break something ? Thanks -- IPVP From: IPVP Date: July 12, 2016 at 7:45:00 PM To: u

Unable to run Storm 1.0.1 on Windows

Hello, We are using Storm as a core component of a large-scale document processing/analysis system. We have been developing this for a while now, starting with Storm 0.8 (I think). We are currently using Storm 0.10.0 and were hoping to upgrade to 1.0.1 (as the release notes indicate that some o

Re: Storm Benchmarks

Hi Manu Zhang, Thank you for your prompt replies. I am using Apache storm 1.0.1. So I don't whether it is feasible to change the version of storm inside pom.xml file of storm-benchmark to 1.0.1. ?Do the metrics you use include both computation and communication parts?Do you have an idea about th

Re: Storm Benchmarks

Hey Walid, storm-benchmark only supports storm 0.9.x for now. Which storm version do you want to run against ? Regards, Manu Zhang On Mon, Jul 18, 2016 at 7:37 PM Walid Aljoby wrote: > Thank you. > You mean that it could not be run with new version of Apache storm? > Regards > Walid > >

Re: Storm Benchmarks

Thank you. You mean that it could not be run with new version of Apache storm? Regards Walid From: darion.yaphet To: d...@storm.apache.org; Walid Aljoby Cc: User Sent: Monday, July 18, 2016 6:35 PM Subject: Re:Storm Benchmarks It seems not found backtype.storm.generated.AlreadyAli

Re:Available Metrices

Hi : I'm not very sure what is available network and application metrics ? Is all kind of metics ? 在 2016-07-18 12:01:45，"Walid Aljoby" 写道： Hi Everyone, Anyone can please list the available network and application metrics in storm? It is good if there is specific topology uses these matri

Re:Storm Benchmarks

It seems not found backtype.storm.generated.AlreadyAliveException .Because AlreadyAliveException have move to org.apache.storm.generated At 2016-07-18 18:06:36, "Walid Aljoby" wrote: >Hi Everyone, >Anyone used to run Storm Benchmarks in this site:  >https://github.com/roshannaik/storm-benchmark-s

Doesn't every supervisor have a default 4 slots?

Hi, On my local system I was able to run 5 topologies only when I increased the number of slots in storm.yaml, to 5. But when submitting my jar to storm on an Amazon node, even though I had 5 supervisor nodes running, only 4 of my topologies were assigned a worker. The 5th topology got 0 workers.

Re: Is there a reason storm has a default of 4 ports?

Ok, if there's an answer to the first question, then anyone who knows details about Storm's design, please help in this thread. For the second question, I'll be starting a separate thread, since there would be people who'd have experience with running multiple topologies. On Mon, Jul 18, 2016 at 1

Storm Benchmarks

Hi Everyone, Anyone used to run Storm Benchmarks in this site:  https://github.com/roshannaik/storm-benchmark-sol ?The compilation was successful, but I got into error through run time? .0.1/bin/storm -jar ./target/storm-benchmark-0.1.0-jar-with-dependencies.jar -conf ./conf/sol.yaml -c topology