Re: compiling storm code with maven and using storm client

2014-04-14 Thread David Crossland
I'm not sure about 1 But 2, you can just copy any topology dependant jars to storms lib directory D From: Xing Yongmailto:xyong...@gmail.com Sent: ?Monday?, ?14? ?April? ?2014 ?12?:?25 To: user@storm.incubator.apache.orgmailto:user@storm.incubator.apache.org 1. when I compile the storm-0.9.1

Re: storm.yml conflict, which one to use?

2014-04-08 Thread David Crossland
To my (rather limited) knowledge storm.yaml should be consistent across all locations. It can be overridden in your topology jar. D From: howard chenmailto:howac...@gmail.com Sent: ?Tuesday?, ?8? ?April? ?2014 ?14?:?28 To: user@storm.incubator.apache.orgmailto:user@storm.incubator.apache.org

Re: storm.yml conflict, which one to use?

2014-04-08 Thread David Crossland
@storm.incubator.apache.org Hi Because it does not make sense to duplicate, for example, ui.childopts in supervisor'storm.yaml right? So I want to know how the storm.yaml is being used, so I can remove these duplicates if they are really dummy and not used. On Tue, Apr 8, 2014 at 9:38 PM, David Crossland da

Re: java.lang.IllegalArgumentException: timeout value is negative - seen in Worker logs

2014-04-02 Thread David Crossland
This is my netty config, perhaps this will help? Swap yours for this, does the problem persist? Introduce your preferred values in a binary search pattern to track down the problem configuration item. Hopefully that specified below will get your topology working again..

Re: Storm UI

2014-04-02 Thread David Crossland
a new Issue to take a look. -- Derek On 4/2/14, 8:04, David Crossland wrote: I have a curious issue with the UI, it displays different stats depending on whether or not I show/hide system stats. How much the stats change seems to differ between Chrome and IE. Is this a known issue? Thanks

Curiosity

2014-04-01 Thread David Crossland
I'm just wondering about the storm.yaml settings, should these be replicated to all nodes in the topology or need it only be configured on the “master” node where I deploy the topology? thanks David

Re: Basic storm question

2014-04-01 Thread David Crossland
On said subject, how does memory allocation work I these cases? Assuming 1 worker per node would you just dump all the memory available into worker.childopts? I guess the memory pool would be shared between the spawned threads as appropriate to their needs? I'm assuming the equivalent options

Re: Server load - Topology optimization

2014-03-20 Thread David Crossland
Values(message), msgId); _collector.ack(tuple); } On Thu, Mar 20, 2014 at 12:18 AM, David Crossland da...@elastacloud.commailto:da...@elastacloud.com wrote: _collector.emit(new Values(message));

Re: Server load - Topology optimization

2014-03-20 Thread David Crossland
); _collector.ack(tuple); } On Thu, Mar 20, 2014 at 12:18 AM, David Crossland da...@elastacloud.commailto:da...@elastacloud.com wrote: _collector.emit(new Values(message));

Re: Server load - Topology optimization

2014-03-19 Thread David Crossland
will sleep 1ms to call nextTuple if this calling of nextTuple sent nothing. 2014年3月19日 上午7:14于 David Crossland da...@elastacloud.com写道: Perhaps these screenshots might shed some light? I don't think there is much of a latency issue. I'm really starting to suspect there is some consumption rate

Re: Server load - Topology optimization

2014-03-19 Thread David Crossland
On Tue, Mar 18, 2014 at 5:14 PM, David Crossland da...@elastacloud.commailto:da...@elastacloud.com wrote: Perhaps these screenshots might shed some light? I don't think there is much of a latency issue. I'm really starting to suspect there is some consumption rate issue from the topic. I set

Server load - Topology optimization

2014-03-18 Thread David Crossland
Being very new to storm I'm not sure what to expect in some regards. Ive been playing about with the number of workers/executors/tasks trying to improve throughput on my cluster. I have a 3 nodes, two 4 core and a 2 core node (I can't increase the 3rd node to a medium until the customer gets

Re: Server load - Topology optimization

2014-03-18 Thread David Crossland
Could my issue relate to memory allocated to the JVM? Most of the setting are pretty much the defaults. Are there any other settings that could be throttling the topology? I'd like to be able to identify the issue without all this constant stabbing in the dark... ? D From: David

Re: Server load - Topology optimization

2014-03-18 Thread David Crossland
achieve a level of performance you are satisfied from the spout, add one bolt. What bottlenecks does the bolt introduce? etc etc. On Tue, Mar 18, 2014 at 2:31 PM, David Crossland da...@elastacloud.commailto:da...@elastacloud.com wrote: Could my issue relate to memory allocated to the JVM? Most

Re: Advice

2014-03-17 Thread David Crossland
as possible for the guys doing more of the work. I will post something on my site on this topic in the next day or so; I'll reply back on this thread when I do. Cheers, Lajos theconsultantcto.com Enterprise Lucene/Solr On 16/03/2014 12:59, David Crossland wrote: Hi, I have a 3 node cluster

Advice

2014-03-16 Thread David Crossland
Hi, I have a 3 node cluster 2 medium, 1 small instances (I'm probably going to up this to a medium). 10 Cores total. My main bottleneck is a service bus which has approx. 3.5mil json string messages published to it in a day. I don't seem to be consuming messages at a fast enough rate. Ive

Error when submitting toplogy

2014-03-12 Thread David Crossland
I was investigating an issue with my storm deployment and came across this posting https://groups.google.com/forum/#!msg/storm-user/RXozK0SHlp8/WDIFtp9V1lAJhttps://groups.google.com/forum/#%21msg/storm-user/RXozK0SHlp8/WDIFtp9V1lAJ More specifically this error java.io.IOException: . doesn't