Re: Storm not processing topology without logs

However, now my topology is failing to start worker process again. :( This time is not showing me any good clue to resolve it. Running the command manually on console causes "Address already in use" error for supervisor ports (6700,6701). So, it is not letting me move forward to see what actually

sometimes netty connection fails

Dear all, When running a topology in cluster mode, worker cannot connect to Netty in other node like below. Is there any possible reason for this? Thanks in advance 2014-08-26 18:11:42 b.s.m.n.Client [INFO] Reconnect started for Netty-Client-somehostname/10.someip:6700... [30] 2014-08-26 18:11:43

MultiCountMetric is empty

Dear all, Imagine there a 3 bolts A,B and C linked to one another and one metric consumer(IMetricConsumer implementation) Inside the execute method in each bolt, there is a small piece of code sending metric to consumer AMetric.scope("cnt").incr(); BMetric.scope("cnt").incr(); CMetric.scope

Re: Question on failing ack

Thanks Srinath for your response. I have not looked into configuring the buffers. If you can point me to the direction where I can get more information on it would be helpful. AFAIK the records from bulk load is mostly at constant rate. When I am just doing a message at a time it works fine. Also

Re: Question on failing ack

Thanks, Michael, How do you verify the reliability of the KafkaSpout? I am using the KafkaSpout that came with storm 0.9.2. AFAIK kafkaSpout is quite reliable. I am guessing it the processing time for each record in the bolt. Yes form the log I do see few Cassandra exceptions while inserting the r

Re: issue in storm submit on multiple machine

Hi Ankit, I have 2 workers on each topology. I used your suggestion, but when i submitted jar from worker machine then it only registered that topology on nimbus machine(server 1) but doesn’t start. And when i ran supervisor on nimbus machine(server 1), then it started those topology on nimbus

Re: Question on failing ack

I started looking into setting up internal message buffer as mentioned in this link. http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/#how-to-configure-storms-internal-message-buffers I found out that my message size could be as big as 10K. So does that mean

Re: Question on failing ack

Also FYI, I do not have any failure in any of my bolts. So I am guessing it has to do with the amount of message that the spout is trying to read form kafka. as per the document by Michael Noll I am trying to see if setting up the config as per suggestion as config.put(Config.TOPOLOGY_RECEIVER_BU

Re: supervisor not listening on port 6700?

I fail to understand why that should happen, as testing with LocalCluster goes through fine. I did a clean fresh start to figure out what could be happening, and here are my observations - - fresh clean start: cleanup in zk (rmr /storm), and /bin/rm -fr {storm's tmp dir} - used local pseudo clust

Re: Question on failing ack

These changes did help a lot. Now I dont see any failed acks for alsmot 70k data load. Thanks a lot for your help. -- Kushan Maskey 817.403.7500 On Tue, Aug 26, 2014 at 9:45 AM, Kushan Maskey < kushan.mas...@mmillerassociates.com> wrote: > Also FYI, I do not have any failure in any of my bolts.

Re: Question on failing ack

Yes, the defaults mentioned in the blog worked for me too. On Wed, Aug 27, 2014 at 2:49 AM, Kushan Maskey < kushan.mas...@mmillerassociates.com> wrote: > These changes did help a lot. Now I dont see any failed acks for alsmot > 70k data load. Thanks a lot for your help. > > -- > Kushan Maskey >

Re: supervisor not listening on port 6700?

I am even having the almost same versions of storm (0.9.1) and kafka. And my topologies were also facing the same issue. When I ran the worker command directly, I came to know that somehow hostname was wrong in the configuration passed to the workers. So, I fixed that in storm config and my topolog

Re: transient errors of "Tuple created with wrong number of fields"

Thanks Dmitry. We are using Trident with transactional spout/state, and this happens in the first 10 minutes when the topolgoy starts, and recover itself after then. Since the topology is transactional, same messages are replayed, so this doesn't seem a data issue, and does seem some storm bug?

Re: supervisor not listening on port 6700?

I left supervisor running with the `still hasn't started` state on one window, and tried starting the worker on another window. That triggered an attempt to start another worker (with another distinct id) in the first window (the supervisor window) which in turn went into the `still hasn't started

Re: supervisor not listening on port 6700?

When it gets into `still hasn't started` state, I have noticed this in UI - java.lang.RuntimeException: java.net.ConnectException: Connection refused at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) at backtype.storm.utils.DisruptorQueue. and am wondering how t

Re: supervisor not listening on port 6700?

Does anyone know what the git branch name is for 0.9.2 ? On Tue, Aug 26, 2014 at 10:24 PM, Naga Vij wrote: > When it gets into `still hasn't started` state, I have noticed this in UI - > > java.lang.RuntimeException: java.net.ConnectException: Connection refused > at > backtype.storm.utils.Disr