Hi all,
I want to read new assignment by nimbus on zookeeper.
Can i do it ?
I have a requirement to perform some action on any change in assignment.
As our spout has the standard virtual ip. external world push the data to
this spout.
I want that when the spout accidentally killed and spawned
I would rather say, if the operation to validate/sync the clock is required
(which I am not able to understand because clock sync is mostly done once
machine starts) to validate the state of the bolts, you can catch the
exceptions, however, it would be better to have separate scheduled service
whic
The solution is to use a Singleton to hold the pool. Guard the
initialization of Singleton such that when the 'N' tasks on a worker try to
initialize it, only one of them succeeds and the rest of them see that the
pool is already initialized. This you would do once in the prepare()
method. There is
I think its a common problem in distributed system:
I only need a redis pool in one VM(or less complicated, one redis pool for
a bolt class), and I want to be able to change the host/port in the config
file without recompiling code.
so when I submit the topology, i read from file, and put into sto
>From reading the docs what I concluded is that all tuples know their parent
spout tuple so all tuples from the same tree go to the same acker task.
However, for one instance I saw all tuples going to only one acker task no
matter how many acker threads I gave the topology.
I was working with a t
Saurabh,
Storm 0.9.0 didin't ship kafka connector but 0.9.2
comes with kafka connector. It used be external
project [1]https://github.com/wurstmeister/storm-kafka-0.8-plus
and it works with kafka 0.8. You can modify the connector to
work with kafka 0.7 . Storm core doesn't have any depen
I think handling the exceptions thrown by your calls will make it work.
On Fri, Sep 5, 2014 at 9:20 AM, Matteo - frman wrote:
> Your hints have been very useful!
>
> I was synchronizing local clock using the NTP protocol within the
> "prepare()" method of the bolt. In order to check the functio
Your hints have been very useful!
I was synchronizing local clock using the NTP protocol within the
"prepare()" method of the bolt. In order to check the functionalities of
the system, I've disabled this step and it seems to work properly.
Which is, in your opinion, the best way of doing this? Us
If the data is emitted in terms of streams continuously from a sensor then
Storm would be the ideal framework to process the data.
Storm infact lets you process streams of data in real time.
To get brief overview go through documentation over storm.
https://storm.incubator.apache.org/
On Thu, Se
That is exactly what the paper, I shared, says. There are two ways for
doing that, offline and online. Offline decides the hot edges only once and
assign bolts on hot edges to same machine while online one takes it to the
next step with adaptive changes for hot edges detection.
On Fri, Sep 5, 201
Hello!
Thank you for your responses. My concerns are related with the fact that
the bolts will be placed in a cluster in a round robin fashion. Thus,
you'll encounter the latency due to the network transfer between nodes. In
my opinion, in my case the more I collocate the bolts on the same node (
It seems there is some error in initializing the bolts or spouts and it may
be occasional if the error is something like error in connecting to an
external service or database during bolt prepare method. So, you need to
look into the supervisor.log for exact error detail when topology does not
pick
There are schedulers which can adapt themselves based on topology workload
and historical performances.
I found a good paper on this:
http://www.dis.uniroma1.it/~midlab/articoli/ABQ13storm.pdf
On Fri, Sep 5, 2014 at 3:20 PM, Michael Vogiatzis <
michaelvogiat...@gmail.com> wrote:
> Hi Spico,
>
>
Hi,
I increased the netty wait time in storm.yaml and it seemed like the issue
was solved.
storm.messaging.netty.max_retries: 300
storm.messaging.netty.max_wait_ms: 1
storm.messaging.netty.min_wait_ms: 4000
But I got the same error again after a few hours. So the above fix just
delayed the e
Hi Spico,
The Scheduler mechanism assigns tasks to available worker slots, but does
not distribute the workload in real time continuously. Therefore the
scheduler approach is unrelated to the network latency your topology may
encounter.
An example case for implementing a custom scheduler for netw
I want to execute a sequential topology, composed by a source (spout) and 7
bolts.
I submit it to the system without errors, but if I go on the UI the bolts
are not emitting tuples; it seems like they haven't been assigned to worker
processes (I'm not sure about that).
Honestly I don't believe it
Hello!
I'm running (in LocalCluster) a topology containing one spout that
generates data and 600 dummy bolts that are passing the data from one to
the other like a chain, meaning:
Spout->B1->B2->...->B600. After a while the abolve exception occurs. Can
you please advice me how to avoid this error
Hello!
I have a test case that involves a dataflow containing a spout and N
serial bolts. For better understanding the dataflow looks like this:
Spout->B_1->B_2->...->B_N
where the "->" repersents the transition of the data from one bolt to
another.
Given the fact that by default Storm is using
18 matches
Mail list logo
