Re: UI problem

Hi, Could you open developer tools from your browser and check any API calls are failing with UI page load? Jungtaek Lim (HeartSaVioR) 2016년 4월 29일 (금) 오후 2:47, Sai Dilip Reddy Kiralam < dkira...@aadhya-analytics.com>님이 작성: > > Hello, > > I Installed storm 0.10.0 in aws (ubuntu)

Re: Distribute load across multiple storm clusters.

Darsh, I am in a bit of a crush for a deployment and perf testing but I will ask my experts to take a look tomorrow. I will set up a single nimbus and 3 zookeepers in one data center, then I will distribute the supervisors and see what happens. I think we will always be limited in how we back

Re: Distribute load across multiple storm clusters.

Patrick, Thank you for replying. I did try but load isn't distributed. Both clusters are processing all the events individually on the topic. Darsh On Thu, Apr 28, 2016 at 2:56 PM, Patrick.Brinton wrote: > Darsh, > I have never tried but I have a setup where I

Re: Distribute load across multiple storm clusters.

Darsh, I have never tried but I have a setup where I could try. As long as you point to the same zooKeeper I think it would work. Give it a try and let me know if you hit issues. If you do I will mimic your setup and we should be able to figure it out. Patrick Patrick Brinton Sr. Engineer

Re: thread safe output collector

I was partly right. In earlier versions (pre-1.0) was not thread safe (due to shuffle grouping). 1.0 introduced a new shuffle grouping implementation that is thread-safe, in addition to a load-aware shuffle grouping. I had missed the fact that the new shuffle grouping is thread safe. -Taylor

Re: thread safe output collector

We tried implementing our own thread safe output collector but it seems ridiculous for such a concurrent system. Why don’t they implement it in core Storm? They can have the current one, and then build a separate one called Concurrent_OutputCollector or some such. I was hoping they fixed that

Distribute load across multiple storm clusters.

Hi, Is it possible to to distribute load across 2 clusters if I deploy same topology to 2 storm clusters with same spout id? We have 2 separate storm clusters with 0.10.0 version of storm running in 2 different availability zones. We are using storm-kafka spout (with simple consumer) to

Log4j2 Issue: worker.xml Configuration is not Applied

Hello, We've recently upgraded to Storm 1.0 and noticed the behavior of the topology logs changed from a flattened structure to a directory structure. We updated the worker.xml filefrom this: fileName="${sys:storm.id}/${sys:worker.port}/${sys:logfile.name}.metrics"

Re: thread safe output collector

Thanks for the clarification On 28 April 2016 at 15:12, P. Taylor Goetz wrote: > The documentation is wrong. See: > > https://issues.apache.org/jira/browse/STORM-841 > > At some point it looks like the change made there got reverted. I will > reopen it to make sure the

Re: Slots vs. Topology

I recommend against aligning the number of slots on a supervisor to the number of cores in its CPU/s. In general I think this is too high a granularity, but then I was also used to supervisor machines that are pretty big (12+ cores). On Thu, Apr 28, 2016 at 10:48 AM, Matthias J. Sax

Re: Slots vs. Topology

@Nathan: I am not sure what you recommend against? I did not recommend anything so far... From my point of view, I doubt there is a good general recommendation to configure the slots per supervisor. As Nathan mentioned correctly, if you have less workers, a single worker needs to do more work.

Re: thread safe output collector

The documentation is wrong. See: https://issues.apache.org/jira/browse/STORM-841 At some point it looks like the change made there got reverted. I will reopen it to make sure the documentation is corrected. OutputCollector is NOT thread-safe.

Re: Slots vs. Topology

I would recommend against this. Storm will automatically run multiple threads for you, especially if you have more than 1 executor / worker. Every time data transfers between workers, it must be serialized and deserialized. On the other hand, if you have larger workers and one goes down, your

Re: Slots vs. Topology

Matthias , Thanks for the clear explanation. Is there any initial guidance to align the number of slots a supervisor could handle based on the machine # of cpus/cores ? -- IPVP From: Matthias J. Sax Reply: user@storm.apache.org

Re: thread safe output collector

"Its perfectly fine to launch new threads in bolts that do processing asynchronously. OutputCollector is thread-safe and can be called at any time." >From the docs for 0.9.6:

Re: thread safe output collector

IIRC there was discussion about making it thread safe, but I don't believe it was implemented. -Taylor > On Apr 28, 2016, at 3:52 AM, Julien Nioche > wrote: > > Hi Stephen > > I asked the same question in February but did not get a reply > >

Re: thread safe output collector

Hi Stephen I asked the same question in February but did not get a reply https://mail-archives.apache.org/mod_mbox/storm-user/201602.mbox/%3cca+-fm0urpf3fuerozywpzmxu-kdbgf-zj3wbyr8evsaqjc6...@mail.gmail.com%3E Anyone who could confirm this? Thanks On 27 April 2016 at 14:05, Steven Lewis

Re: docs on storm starter frustrating, but success

Documentation for storm 0.10 is available here - http://storm.apache.org/releases/0.10.0/index.html On Wed, Apr 27, 2016 at 4:11 AM, Joaquin Menchaca wrote: > Ignore the docs. I looked at the final packaged jar from mvn clean > install, noticed the path was different. I

Re: Slots vs. Topology

The number of slots defines the number of worker JVM a supervisor can start. And a single worker JVM only executes code of a single topology (to isolate topologies for fault-tolerance reasons). Thus, you need to have a least a single worker for each topology in your cluster (ie, sum of all slots

Re: Storm topology using all the Max connections of db

Hi Spico, I use 9 bolts with parallelism of 1 and with 1 task for each bolt(default) and I'm not using any ConnectionPool for connecting to postgres.Just using jdbc classe examples here is source example http://storm.apache.org/releases/2.0.0-SNAPSHOT/storm-jdbc.html. I will check by using the

Re: Storm topology using all the Max connections of db

Hello! How many tasks do you have for inserting the data to your database? Are you using ConnectionPool for connecting to Postgres? If your number of task superseeds the number of max connections provided in connection pool then your have a problem. Please also check the number of max