Re: LoggingMetricsConsumer

2014-09-25 Thread John Reilly
It is called by the MetricsConsumerBolt which is created by storm when a worker is starting up. When you define a metrics consumer, you should see metrics output every 60 seconds. Also, I think the metrics code was only introduced in 0.9.0 so you would need to be running at least that version. O

RE: Cassandra bolt

2014-09-25 Thread Strulovitch, Zack
Thank you very much Dan From: Dan DeCapria, CivicScience [dan.decap...@civicscience.com] Sent: Thursday, September 25, 2014 3:08 PM To: user@storm.incubator.apache.org Subject: Re: Cassandra bolt This might be useful to you as well: Netty Issue (I'm using Storm-Kafka and Kafka as well): rm /opt

Re: Cassandra bolt

2014-09-25 Thread Dan DeCapria, CivicScience
This might be useful to you as well: Netty Issue (I'm using Storm-Kafka and Kafka as well): rm /opt/programs/storm/lib/netty-3.2.2.Final.jar CassandraWriterBolt Topology implementation: CassandraWriterBolt cassandraWriterImpressionsBolt = new CassandraWriterBolt( properti

Re: metrics consumer logging stormUI data

2014-09-25 Thread John Reilly
It is not necessary to have debug enabled to get metrics. Also, it is independent of storm ui. The ui does not need to be running - the metrics are reported by the workers. On Wed, Sep 24, 2014 at 12:12 PM, Raphael Hsieh wrote: > In order to get this / have the metrics consumer work, do I nee

RE: Cassandra bolt

2014-09-25 Thread Strulovitch, Zack
Thank you Robert and Harsha. I like Robert's suggestion since it was updated recently. From: Robert Lee [lee.robert...@gmail.com] Sent: Thursday, September 25, 2014 2:24 PM To: user@storm.incubator.apache.org Subject: Re: Cassandra bolt If you are looking for CQL, I'd suggest: https://github.c

Re: Cassandra bolt

2014-09-25 Thread Robert Lee
If you are looking for CQL, I'd suggest: https://github.com/hmsonline/storm-cassandra-cql On Thu, Sep 25, 2014 at 2:22 PM, Harsha wrote: > did you tried https://github.com/ptgoetz/storm-cassandra. > > > On Thu, Sep 25, 2014, at 11:20 AM, Strulovitch, Zack wrote: > > I've updated to 0.9.2 from

Re: Cassandra bolt

2014-09-25 Thread Harsha
did you tried [1]https://github.com/ptgoetz/storm-cassandra. On Thu, Sep 25, 2014, at 11:20 AM, Strulovitch, Zack wrote: I've updated to 0.9.2 from pre-apache version 0.9.0.1 (which broke my Cassandra bolt implemented using this code : [2]https://github.com/tjake/stormscraper[3]https://github

Cassandra bolt

2014-09-25 Thread Strulovitch, Zack
I've updated to 0.9.2 from pre-apache version 0.9.0.1 (which broke my Cassandra bolt implemented using this code : https://github.com/tjake/stormscraperhttps://github.com/tjake/stormscraper ) According to some posts, this is due to netty conflict. Could anyone please suggest me an alternative re

LoggingMetricsConsumer

2014-09-25 Thread Raphael Hsieh
Hi, I've been trying to figure out why registerinfg a LoggingMetricsConsumer isn't working for me. I've been able to figure out that it is indeed running, however the "handleDataPoints()" function is never called. Can someone explain to me how this class is used by Storm in order to log metrics? W

Re: Configuration changes and storm cluster

2014-09-25 Thread Nathan Leung
https://github.com/apache/storm/blob/master/conf/defaults.yaml On Thu, Sep 25, 2014 at 12:48 PM, Gunderson, Richard-CW < richard.gunder...@bestbuy.com> wrote: > Thank you for the responses guys. > > > > I’ve done a little experimenting and discovered restarting the daemons > (nimbus, supervisor)

RE: Configuration changes and storm cluster

2014-09-25 Thread Gunderson, Richard-CW
Thank you for the responses guys. I’ve done a little experimenting and discovered restarting the daemons (nimbus, supervisor) is not such a big deal for our topologies. We already have a dedicated zookeeper server in our environment, so we had no need to use the included zk library. It was easy

Re: Rolling Upgrade from 0.91

2014-09-25 Thread Bobby Evans
0.9.1 to 0.9.2 is not possible to do a rolling upgrade. The data structures stored in Zk and on disk have changed in a non-backwards compatible way. Most EBFs we do rolling upgrades. For them we install the new yinst package on all of the nodes and then restart the supervisor, logviewer, nibs

Re: What happens on a batch timeout?

2014-09-25 Thread Mayur Rustagi
Seems to me, it depends on which spout you are using. If you are using Kafka & Transactional Spout then replay is consistent each time. In any other queue, batch may be different. This contains the type of spouts & their limitations. http://storm.incubator.apache.org/documentation/Trident-spouts.ht

RE: What happens on a batch timeout?

2014-09-25 Thread Simon Cooper
Does anyone have any information that could help with this? I'm baffled and don't understand the behaviour we're seeing - events are being received out of order on a batch replay, the only reason I can think is that tuples are left over from the previous batch in the input queues, but trying to

RE: High CPU usage by storm workers

2014-09-25 Thread Simon Cooper
We’ve found the same thing. A significant amount of cpu time is being taken up by unlocking (?) within the disruptor queue. From: Rahul Mittal [mailto:rahulmitta...@gmail.com] Sent: 24 September 2014 16:16 To: user@storm.incubator.apache.org Subject: High CPU usage by storm workers Hi We are e

Rolling Upgrade from 0.91

2014-09-25 Thread Noam Cohen
Hey guys - We want to upgrade our Storm cluster from 0.91 to 0.92. I couldn't find any procedure out there on how to perform a rolling upgrade for Storm. Do you know if such a procedure is supported? Could a cluster temporarily operate when some of the worker nodes are in lower versions than the