storm logback freezing

Hi all, Has anyone experienced a case where storm logback freezes? The topology seems to be functioning without an issue (I can see the results, in the destination consumers), but the storm log shows no progress. This usually happens a couple of hours after the topology starts, and not right away.

Re: Use Load Balancer instead of zookeeper IPs

We used Puppet to generate the Kafka config list, so if there was a change to the Zookeeper server inventory, we could regenerate the Kafka config automatically, and then do a rolling restart to pick up the new Zookeeper servers. On Tue, May 5, 2015 at 8:58 AM Dillian Murphey wrote: > It seems t

Missing Complete latency/Process latency in Storm UI.

Hi, I was experiencing a weird issue while I am using storm UI, sometimes I don't see Complete latency/Process latency values being shown and instead they are all zeroes. If I resubmit the same topology, it works sometimes but not always. Has anyone faced this issue? Is there any solution for the

Re: Use Load Balancer instead of zookeeper IPs

We’re using Exhibitor but Storm still requires the zookeeper hosts to be hard-coded into storm.yaml. I found a patch that addresses this (https://issues.apache.org/jira/browse/STORM-702 ) but I don’t know if/when this will land. > On May 4, 2015

Re: Use Load Balancer instead of zookeeper IPs

Netflix's Exhibitor has the functionality you are describing. https://github.com/Netflix/exhibitor https://github.com/Netflix/exhibitor/wiki https://github.com/Netflix/exhibitor/wiki/Automatic-Instance-Management https://github.com/Netflix/exhibitor/wiki/Rolling-Ensemble-Change From: Dillian Mu

Re: Use Load Balancer instead of zookeeper IPs

Also, FYI, Netflix Exhibitor works pretty well to manage the zookeeper cluster. It is not related to the question in this thread, but just FYI if you're not using it but are using AWS, look into it for maintaining your zookeeper cluster in general. On Mon, May 4, 2015 at 6:55 AM, Andrew Medeiros

Re: Use Load Balancer instead of zookeeper IPs

It seems to work, but we're just starting to use the system. As I get more developers experimenting we'll see how it goes. There are still a lot of people out there not benefiting from AWS services like autoscaling groups and load balancers so it seems there isn't a lot of documention on using AWS

Replacing the Nimbus Node in a running topology

Hi Folks, The Nimbus node in our currently running topology is up for maintenance (The cluster is in AWS). I was wondering if there was any way we could spin up a new Nimbus node and have it take over the administration of the topology so that we can avoid any potential data loss due to the downti

Re: Use Load Balancer instead of zookeeper IPs

Brian, I am under the same situation of not wanting to hardcore the zookeeper IPs otherwise it makes autoscaling pretty impossible. Have you tried putting your zookeeper cluster behind an ELB yet? If so what where your results? Thank you! Cheers, Andrew Medeiros — Sent from Mailbox On

Re: Unable to process topology fast?

I would read / listen to these: http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/ https://storm.apache.org/javadoc/apidocs/backtype/storm/Config.html https://www.youtube.com/watch?v=bdps8tE0gYo https://www.youtube.com/watch?v=iLZrYPbNypg http://www.slideshar

Re: Netty connection issue

that message is about kafka it is not saying that a kafka broker is down, but, a particular partition of a kafka topic is "down" two ways for you to see this: * /opt/kafka_2.9.2-0.8.1.1/bin/kafka-topics.sh --describe --zookeeper {a_node_from_kafkas_zookeeper_cluster} or * install and make use of

Unable to process topology fast?

Hi, I executed all storm examples.i designed the application with one spout and two bolts. here my spout is reading list of files content and emitting to bolt.here i want to process everything fast can you help me with what are the configurations need to set. That means what are minimum basic

Re: Unable to use remote slave node in Apache Storm cluster

"0 free slots” explains why there are no workers. I would start by removing references to “localhost” to use the external ip. It sounds to me like you have guessed what I did, that your supervisor node is not connecting to the cluster. What do the logs of your supervisor instance on 10.0.0.79

Relation between buffer size and max spout pending

Hello, >From http://stackoverflow.com/questions/24413088/storm-max-spout-pending looks like max spout pending is the maximum number messages that are within the topology at any point in time. From http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/ the variou

Re: Netty connection issue

Sorry for delayed response to your mail. I dont think IP address is changing as i use private IP's. On Fri, May 1, 2015 at 7:59 AM, Supun Kamburugamuva wrote: > Does your machines ips change after sometime? > > Thanks, > Supun.. > On Apr 30, 2015 10:15 PM, "Tousif" wrote: > >> I get this error