However, now my topology is failing to start worker process again. :(
This time is not showing me any good clue to resolve it. Running the
command manually on console causes "Address already in use" error for
supervisor ports (6700,6701). So, it is not letting me move forward to see
what actually
Dear all,
When running a topology in cluster mode, worker cannot connect to Netty in
other node like below.
Is there any possible reason for this?
Thanks in advance
2014-08-26 18:11:42 b.s.m.n.Client [INFO] Reconnect started for
Netty-Client-somehostname/10.someip:6700... [30]
2014-08-26 18:11:43
Dear all,
Imagine there a 3 bolts A,B and C linked to one another and one metric
consumer(IMetricConsumer implementation)
Inside the execute method in each bolt, there is a small piece of code sending
metric to consumer
AMetric.scope("cnt").incr();
BMetric.scope("cnt").incr();
CMetric.scope
Thanks Srinath for your response.
I have not looked into configuring the buffers. If you can point me to the
direction where I can get more information on it would be helpful. AFAIK
the records from bulk load is mostly at constant rate. When I am just doing
a message at a time it works fine. Also
Thanks, Michael,
How do you verify the reliability of the KafkaSpout? I am using the
KafkaSpout that came with storm 0.9.2. AFAIK kafkaSpout is quite reliable.
I am guessing it the processing time for each record in the bolt. Yes form
the log I do see few Cassandra exceptions while inserting the r
Hi Ankit,
I have 2 workers on each topology. I used your suggestion, but when i
submitted jar from worker machine then it only registered that topology
on nimbus machine(server 1) but doesn’t start. And when i ran supervisor
on nimbus machine(server 1), then it started those topology on nimbus
I started looking into setting up internal message buffer as mentioned in
this link.
http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/#how-to-configure-storms-internal-message-buffers
I found out that my message size could be as big as 10K. So does that mean
Also FYI, I do not have any failure in any of my bolts. So I am guessing it
has to do with the amount of message that the spout is trying to read form
kafka.
as per the document by Michael Noll I am trying to see if setting up the
config as per suggestion as
config.put(Config.TOPOLOGY_RECEIVER_BU
I fail to understand why that should happen, as testing with LocalCluster
goes through fine.
I did a clean fresh start to figure out what could be happening, and here
are my observations -
- fresh clean start: cleanup in zk (rmr /storm), and /bin/rm -fr {storm's
tmp dir}
- used local pseudo clust
These changes did help a lot. Now I dont see any failed acks for alsmot 70k
data load. Thanks a lot for your help.
--
Kushan Maskey
817.403.7500
On Tue, Aug 26, 2014 at 9:45 AM, Kushan Maskey <
kushan.mas...@mmillerassociates.com> wrote:
> Also FYI, I do not have any failure in any of my bolts.
Yes, the defaults mentioned in the blog worked for me too.
On Wed, Aug 27, 2014 at 2:49 AM, Kushan Maskey <
kushan.mas...@mmillerassociates.com> wrote:
> These changes did help a lot. Now I dont see any failed acks for alsmot
> 70k data load. Thanks a lot for your help.
>
> --
> Kushan Maskey
>
I am even having the almost same versions of storm (0.9.1) and kafka. And
my topologies were also facing the same issue. When I ran the worker
command directly, I came to know that somehow hostname was wrong in the
configuration passed to the workers. So, I fixed that in storm config and
my topolog
Thanks Dmitry. We are using Trident with transactional spout/state, and
this happens in the first 10 minutes when the topolgoy starts, and recover
itself after then. Since the topology is transactional, same messages are
replayed, so this doesn't seem a data issue, and does seem some storm bug?
I left supervisor running with the `still hasn't started` state on one
window, and tried starting the worker on another window. That triggered an
attempt to start another worker (with another distinct id) in the first
window (the supervisor window) which in turn went into the `still hasn't
started
When it gets into `still hasn't started` state, I have noticed this in UI -
java.lang.RuntimeException: java.net.ConnectException: Connection refused
at
backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)
at backtype.storm.utils.DisruptorQueue.
and am wondering how t
Does anyone know what the git branch name is for 0.9.2 ?
On Tue, Aug 26, 2014 at 10:24 PM, Naga Vij wrote:
> When it gets into `still hasn't started` state, I have noticed this in UI -
>
> java.lang.RuntimeException: java.net.ConnectException: Connection refused
> at
> backtype.storm.utils.Disr
16 matches
Mail list logo