Hello, Sasi!
Thank your very much for your response.Due to the fact that I would like
to deploy also a Storm application on Amazon, it really helps to spare time
. If it possible, just as an information, do you have some measurement on
how much time did take the cluster to be up and running?
Bes
I've got the same error. In a running cluster, I kill the supervisor running on
one of the machines, wait until storm reassigns the topology that was on that
machine (called Sync), and then bring the supervisor up again. It immediately
dies, with the following in the log:
2014-03-27 10:50:12 b.
Hi Florin,
I just wanted to suggest that you also look at Wirbelsturm by Michael
Noll as an alternative to storm-deploy
https://github.com/miguno/wirbelsturm. I think that you will find it
more complete, better documented, and more mainstream because it uses
vagrant and puppet instead of pallet
Personally I would not use storm for such long running computations. I
would lean towards a batch processing system such as hadoop.
On Mar 27, 2014 12:34 AM, "Swara Desai" wrote:
> Hi,
> Can someone please shed some light on this? Has anyone used storm to
> handle long processing, like video tran
Hi,
Which version of Storm and Zookeper are you using? Storm 0.9.1 seems to be
compatible with zookeeper 3.3.3.Try downgrading the zookeeper version to
3.3.3,I think that should resolve the problem.
Thanks
Bijoy
On Thu, Mar 27, 2014 at 4:33 PM, Simon Cooper <
simon.coo...@featurespace.co.uk> wr
Yes that is exactly right, the submission to Nimbus is in the form of a big
thrift message describing the topology...this message includes java
serialized blobs of your topology components (spouts/bolts). They get
instantiated within the VM calling StormSubmitter. Typically you would
pass configur
Hello everyone,
I'm building a topology in which I'm counting connected devices. Each
device can send a start and a stop message, both with a same unique ID (to
match the device).
Due to the high throughput of messages, I'm storing a state (in my case, a
list of connected devices) in RAM, shared
How do you put a datetime, let's say a jodatime datetime value, in a tuple?
How do you get a datetime out of a tuple, what sort of method corresponds to
tuple.getLongByField for a datetime?
Michael Campbell
--
Try this:
(DateTime)tuple.getValueByField("myDateTimeFieldName");
On Thu, Mar 27, 2014 at 8:50 AM, michael campbell <
michael.campb...@dsl.pipex.com> wrote:
>
> How do you put a datetime, let's say a jodatime datetime value, in a tuple?
>
> How do you get a datetime out of a tuple, what sort of
Also it might be worth reading:
https://github.com/nathanmarz/storm/wiki/Serialization
On Thu, Mar 27, 2014 at 9:01 AM, Dan Guja wrote:
> Try this:
> (DateTime)tuple.getValueByField("myDateTimeFieldName");
>
>
> On Thu, Mar 27, 2014 at 8:50 AM, michael campbell <
> michael.campb...@dsl.pipex.co
Thanks. I think that cleared up most of my misunderstanding.
On Thu, Mar 27, 2014 at 6:16 AM, Adam Lewis wrote:
> Yes that is exactly right, the submission to Nimbus is in the form of a big
> thrift message describing the topology...this message includes java
> serialized blobs of your topology c
How are many of you typically using storm? Use cases usually work best ;)
Say you have one source for a stream of user activity (page views,
product searches, purchases, etc). Now lets say you wanted to
calculate some running metrics for each of these activities. Is it
best to have one major topol
It always depends. :) But here's one factor to consider:
From a R.A.S. perspective (descibed here: http://bit.ly/1fS01MU) a
one-topology approach
*may* negatively impact operational agility when those metrics can be
calculated independently
of one another. In such cases, merging them into one
current storm-deply script currently points to
https://github.com/nathanmarz/storm
0.9.0.1 is the latest tag that works
-
Yi
-
Yi
On Wed, Mar 26, 2014 at 6:01 AM, Spico Florin wrote:
> Hello!
> I would like to know what changes should be applied to the storm-deploy
> script (https://github
Ok, I havent come across that class.
What creates that class?
On Wed, Mar 26, 2014 at 10:21 PM, Angelo Genovese wrote:
> BasicBolts are wrapped automatically with a basic bolt executor. It does the
> back/fail
>
> On Mar 26, 2014 9:40 PM, "Software Dev" wrote:
>>
>> Sorry for the basic question
>
> Also it might be worth reading:
> https://github.com/nathanmarz/storm/wiki/Serialization
After which you'll seek out this library:
https://github.com/magro/kryo-serializers
On Thu, Mar 27, 2014 at 11:09 AM, Dan Guja wrote:
> Also it might be worth reading:
> https://github.com/nathanmarz/
Hi Florin,
The cluster consisted of 1 master, 1 zookeeper, and 5 workers (test env)
and it took about 10 minutes for everything to be up and view the UI
console. The storm-deploy script was run from another ec2 machine running
in the same zone.
I was then able to run my topology in less than 2 min
Actually.. some more questions. We are running 0.9.1-incubating so we
do in fact have the prepare method.
Confused on..
Should I add all configuration values (from a properties file) onto
the StormConf and then in the prepare method of my workers just
instantiate an Injector?
Or...
Should I ins
Similar to hadoop, is it possible to add a local file as a resource
that would be available to storm workers?
Padma, the exception were in our application code. We observed the storm
logs on the supervisors to identify the problems.
~Naresh
On Thu, Mar 27, 2014 at 4:50 AM, Nathan Leung wrote:
> Personally I would not use storm for such long running computations. I
> would lean towards a batch processi
What I've done is just embed them in the topology jar and extract them on
prepare().
On Mar 27, 2014 5:16 PM, "Software Dev" wrote:
> Similar to hadoop, is it possible to add a local file as a resource
> that would be available to storm workers?
>
I am curios has anyone integrated Storm with Spring Web MVC Framework?
I am new to Storm and working on designing new app the integrates Storm
into existing Spring MVC App.
While I haven’t used storm in many months, but that’s not exactly how storm is
integrated.
You would typically have a separate storm cluster that operates independently
of any web app. That is, the Storm cluster starts off with a set of Spouts.
These are the input bits. What you most likely wan
Hi Daniel,
Your answer is very helpful. I am actually looking into Kafka as well and now
have better idea of how to make these technologies.
Thanks.
On Mar 27, 2014 10:21 PM, Daniel Fagnan wrote:
>
> While I haven’t used storm in many months, but that’s not exactly how storm
> is integrated.
Hi all,
Am using storm-0.9.0.1.
The following error is seen in the worker logs:
2014-03-25 16:18:24 STDIO [ERROR] Mar 25, 2014 4:18:24 PM
org.jboss.netty.channel.DefaultChannelPipeline
WARNING: An exception was thrown by a user handler while handling an
exception event ([id: 0x8068e4b0] EXCEPTIO
Hi,
Try modifying "storm.messaging.netty.max_retries: 100" to a lower
value,e.g. "storm.messaging.netty.max_retries: 10".
Thanks
Bijoy
On Fri, Mar 28, 2014 at 8:48 AM, Binita Bharati wrote:
> Hi all,
>
> Am using storm-0.9.0.1.
>
> The following error is seen in the worker logs:
>
> 2014-03-2
Sorry for the ignorance but how is that accomplished?
On Thu, Mar 27, 2014 at 6:08 PM, Mikhail Davidov wrote:
> What I've done is just embed them in the topology jar and extract them on
> prepare().
>
> On Mar 27, 2014 5:16 PM, "Software Dev" wrote:
>>
>> Similar to hadoop, is it possible to add
Hi all,
Can you please suggest any useful link for zookeeper. I need some examples in
java.
I want to write meta data on zookeeper using curator framework.
Can I implement some kind of signal on write on zookeeper? I mean to say that
if one process write on zookeeper, it should signal the anoth
Hi all,
I'm wondering if there is anyway to retrieve a tuple from its messageId, I
need to get the tuple from spout's fail() method to rollback some
operations after the tuple failed.
Thank you & Regards
Nhan Nguyen
29 matches
Mail list logo
