Understanding metrics.log

2014-04-22 Thread P Ghosh
I added metrics to my storm implementation by implementing IMetric. It is working and can see the metrics log populated with all stats. I've a 3 node (3 worker) and 1 nimbus/zookeeper in Development. On WORKER1's Metrics Log I can see , some metrics with reference to WORKER2 and WORKER3. For exam

Re: Understanding metrics.log

2014-04-23 Thread P Ghosh
om > > > On Wed, Apr 23, 2014 at 12:19 AM, P Ghosh wrote: > >> I added metrics to my storm implementation by implementing IMetric. It is >> working and can see the metrics log populated with all stats. I've a 3 node >> (3 worker) and 1 nimbus/zookeeper in Developmen

Best practice for shutting down storm

2014-05-01 Thread P Ghosh
I have few topologies running. The spout puts the ID of the object it is emitting into an WIP list in REDIS. When the spout gets the ack or fail method called, it takes it out of the WIP list. The environment and application are undergoing lot of changes.. and as a result I'm required to occasiona

Understanding ACKing mechanism

2014-05-19 Thread P Ghosh
I have a topology, that looks like *All Bolts emits Fields "id", "json"* *Spout emits only id* *All bolts/spout uses the stream _stream name while emitting* *Topology Definition* *==* builder.setSpout("citySpout", citySpout,10); builder.setBolt("impactBol

Re: Externalize storm.yaml file

2014-05-19 Thread P Ghosh
Use soft links. Prasun Sent from Galaxy Nexus On May 14, 2014 6:50 AM, "Neha Jain" wrote: > Hello, > > > > I have created a storm cluster on Amazon EC2 machines. The requirement we > have is to save Storm configuration file to Amazon S3. > > There would be a bucket in S3 where storm config

Re: Understanding ACKing mechanism

2014-05-20 Thread P Ghosh
s was resulting in leak. Thanks, Prasun On Mon, May 19, 2014 at 1:12 AM, P Ghosh wrote: > I have a topology, that looks like > *All Bolts emits Fields "id", "json"* > *Spout emits only id* > *All bolts/spout uses the stream _stream name while

Re: Is it possible join another spou/bolt later to the topology?

2014-05-22 Thread P Ghosh
Can you elaborate on what exactly you mean by "join". You can have a bolt defined as part of topology which will load the other jar in the prepare() method call actual functional methods in the "execute" method. This way , you are dynamically loading the other jar into your storm topo...(which in m

Conceptual question on Streams definition...

2014-05-23 Thread P Ghosh
My definition of stream is continuous feed of data of certain type or with certain purpose (depends on how you want to define your process) I have a situation, where the Domain Object is same across the whole topology, however, each component working on bits and pieces to construct the final docum

Conceptual question on Streams definition...

2014-05-23 Thread P Ghosh
My definition of stream is continuous feed of data of certain type or with certain purpose (depends on how you want to define your process) I have a situation, where the Domain Object is same across the whole topology, however, each component working on bits and pieces to construct the final docum

java.lang.IllegalArgumentException: timeout value is negative

2014-06-03 Thread P Ghosh
I'm getting this exception 2014-06-03 19:59:13 STDIO [ERROR][id:] Jun 03, 2014 7:59:13 PM org.jboss.netty.channel.DefaultChannelPipeline WARNING: An exception was thrown by a user handler while handling an exception event ([id: 0xdcf3be42] EXCEPTION: java.net.ConnectException: Connection refused) j

Re: java.lang.IllegalArgumentException: timeout value is negative

2014-06-04 Thread P Ghosh
Thanks Srinath for that ref. Prasun On Tue, Jun 3, 2014 at 5:41 PM, Srinath C wrote: > There is a known bug - https://issues.apache.org/jira/browse/STORM-187 > > > > On Wed, Jun 4, 2014 at 1:38 AM, P Ghosh wrote: > >> I'm getting this exception >> 2014-06