Re: Supervisor believes worker has not started.

disallowed = the request to shut down came from nimbus. It takes a village. Also, it takes all of the logs. Get them together to see a better picture: Nimbus, Supervisor, Worker Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: No logs directory in apache storm on Mavericks

: @skgadalay by ./logs I meant logs directory in apache storm. @Jeffery Maass, I tried that, the log files don't appear inside the folder even if I manually create it. Any other suggestions? Thanks. Abhishek Date: Fri, 29 May 2015 22:44:22 +0530 Subject: Re: No logs directory in apache storm

Re: Supervisor repeatedly killing worker

Set logging to info level. The reason is explained every time. Sorry, I don't have any examples You have to look at 3 logs: * nimbus - will say that it is killing a task/executor. As I recall, you have to figure out that the task/executor links up to the supervisor. * supervisor - will say

Re: No logs directory in apache storm on Mavericks

Maybe if you create the logs directory, then restart the processes? Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On Fri, May 29, 2015 at 12:42 AM, Abhishek Raj

Re: Supervisor believes worker has not started.

When you look at the worker logs, do some of the workers sometimes kill themselves because there is a missing stormconf.ser file? If so, grab that error message and have fun googling. Some will say those problems went away with the latest release. Apparently, it is complicated. My best advice

Re: tuple size limitation?

I would take the kafka spout, JSON, your code out of the equation and replicate the problem with a spout that generates strings of various lengths around 75KB. Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: Throughput : local mode is faster than cluster mode

/maassql + On Thu, May 28, 2015 at 10:51 AM, Denis DEBARBIEUX ddebarbi...@norsys.fr wrote: Thanks for your help. Denis Le 28/05/2015 15:15, Jeffery Maass a écrit : Did you know that a horse can race a human and the human will win -- in the first several yards? Spread

Re: Throughput : local mode is faster than cluster mode

Did you know that a horse can race a human and the human will win -- in the first several yards? Spread out the race longer and the horse will always win. The tests being run aren't covering enough situations. Storm was designed to take business code and scale it out in order to handle massive

Re: No logs directory in apache storm on Mavericks

Couple of items to get you going: When you run a topology, you will see that your supervisor and workers both have System Variables set at the command line: -Dstorm.log.dir=/opt/apache-storm-0.9.4/logs -Dlogfile.name=supervisor.log

Re: No logs directory in apache storm on Mavericks

When the workers and supervisor are run, what are the values for: -Dstorm.log.dir= -Dlogfile.name= -Dlogback.configurationFile= Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On

Re: Status of running storm on yarn (the yahoo project)

I have heard Nathan Marz mention Mesos. How is yarn / storm-yarn / slider-yarn different from Mesos? These are the links I found to Mesos: https://github.com/mesos/storm https://github.com/nathanmarz/storm-mesos http://mesos.apache.org/ Thank you for your time! + Jeff

Re: Issues with storm 0.9.4

So you are saying that your organization has been doing security scans for a good long while now. Those security scans are called qualys scan. After upgrading your storm cluster from 0.8.3 to 0.9.4, you've seen instances of workers dying and nimbus issuing rebalances. You believe that the

Re: Whats the alternative to creating a fat jar for topology deployment

Matthias : Good suggestion! On Thu, May 21, 2015 at 9:32 AM, Matthias J. Sax mj...@informatik.hu-berlin.de wrote: You can put your dependency jars into Storm's jar folder (eg /opt/storm-0.9.4/lib/). -Matthias On 05/21/2015 04:16 PM, rajesh_kall...@dellteam.com wrote: *Dell - Internal

Re: STORM UI -- trident topology continuously emitting ..

There are tons of metrics. You just need to get them out of storm and into something you can use. Get the metrics out: https://github.com/verisign/storm-graphite Capture / record / view the metrics: http://graphite.wikidot.com/ Thank you for your time! + Jeff Maass

Re: java.lang.RuntimeException: Returned channel was actually not established

What procedure did you follow for the upgrade? Are both 0.9.3 and 0.9.4 still installed? What is the full path of your storm installations? Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql

Re: Storm 0.9.4 cluster running on Amazon EC2 crashes with an unexpected Netty error

I'm just a little curious about the 2 different IP sets: 172.31.10.201 http://172.31.10.201:6703 and 52.7.165.232 http://52.7.165.232/52.7.165.232:2181 are you doing that on purpose? On Fri, May 22, 2015 at 10:04 AM, Nick R. Katsipoulakis nick.kat...@gmail.com wrote:

Re: snapshots deployment?

You want to ask the dev mailing list: http://mail-archives.apache.org/mod_mbox/storm-dev/ Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On Thu, May 21, 2015 at 1:56 AM, Sergio

Re: Question concerning process latency and number of acked

I highly recommend these videos : Learning Storm: - https://www.youtube.com/watch?v=iLZrYPbNypg - InfoChimps - uses storm quite a bit - covers storm networking / buffering - https://www.youtube.com/watch?v=bdps8tE0gYo - This is DEFN

Re: StackOverFlow

Nathan, could you expand on what you mean by your data structure. yang...@bupt.edu.cn, could you try turning off *Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION ?**Config conf = new backtype.storm.Config();* conf.setFallBackOnJavaSerialization(false);StormSubmitter.submitTopology(topo_name,

Re: STORM UI -- trident topology continuously emitting ..

In your UI, on the topology's page, scroll to the bottom, click, show system stats. Now, every bolt is shown. List out everything you have. Whose numbers are changing? Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: Java memory exception using multilang

This is not a storm issue. https://plumbr.eu/outofmemoryerror/unable-to-create-new-native-thread Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On Tue, May 19, 2015 at 1:51 PM,

Re: storm release versions

, Jeffery Maass maas...@gmail.com wrote: Storm releases are presented on this page: https://storm.apache.org/downloads.html If you are looking at GitHub, you can see different versions of Storm in the tags. v0.9.4 Thank you for your time! + Jeff Maass maas

Re: Decreasing Complete latency with growing number of executors

Maybe the difference has to do with where the executors were running. If your entire topology is running within the same worker, it would mean that a serialization for the worker to worker networking layer is left out of the picture. I suppose that would mean the complete latency could decrease.

Re: storm release versions

Storm releases are presented on this page: https://storm.apache.org/downloads.html If you are looking at GitHub, you can see different versions of Storm in the tags. v0.9.4 Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: Need help

See the 0.9.4 release codebase @ https://github.com/apache/storm/tree/v0.9.4 There is a project called Storm Starter @ https://github.com/apache/storm/tree/v0.9.4/examples/storm-starter Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: No of Records Processed By Topology Per second ???

Ashish: Take a look at these videos: Learning Storm: - https://www.youtube.com/watch?v=iLZrYPbNypg - InfoChimps - uses storm quite a bit - covers storm networking / buffering - https://www.youtube.com/watch?v=bdps8tE0gYo - This is

Re: error in the cluster

Like I said, it's only a feeling I have. Is the disk full? df -h How many open connections are there / grouped by state: netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n run top - see what the processor utilization is are you monitoring the storm worker box? What does the monitoring

Re: handling Kafka log roll

In this specific case, what do you mean by kafka log roll? How did you correlate the storm problems with the kafka log roll? Do you have any logs to show? Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass

Re: logback file in jar

I think that question is probably better asked of the developer's group : http://mail-archives.apache.org/mod_mbox/storm-dev/ Incidentally, how is the inclusion of the logback.xml file causing you a problem? Thank you for your time! + Jeff Maass maas...@gmail.com

Re: Cleanup method not called for the BaseBasicBolt when the topology is killed

Bolts which implement IBolt have a method called cleanup() which is called by the Storm framework. https://storm.apache.org/apidocs/backtype/storm/task/IBolt.html It is not guaranteed to be called, but then again, neither is any custom code you write to perform the same functionality. Thank

Re: How much is the overhead of time to deploy a system on Storm ?

Nathan: Where can I find this? See for example published single machine benchmarks Thank you for your time! + Jeff Maass maas...@gmail.com linkedin.com/in/jeffmaass stackoverflow.com/users/373418/maassql + On Tue, May 12, 2015 at 7:57 AM, Nathan Leung

Re: Multiple Workers in local mode gives error

Asif: Would love to help you, however, we will need way more details. Why don't you start with telling us what the error is? How are you detecting the error? You will want to look over both the supervisor and worker logs. Thank you for your time! + Jeff Maass

Re: Storm thrift connection

The supervisor log you posted covers multiple different workers... This looks expected to me. If an unhandled exception occurs in a worker, it will die. Then either nimbus or the supervisor will cease to see its heartbeats, the supervisor will attempt to kill it, then nimbus will ask a