Hello!
I'm interesting in running the storm topologies on yarn.
I was looking at the yahoo project https://github.com/yahoo/storm-yarn, and
I could observed that there is no activity since 7 months ago. Also, the
issues and requests lists are not updated.
Therefore I have some questions:
1. Is
I have heard Nathan Marz mention Mesos.
How is yarn / storm-yarn / slider-yarn different from Mesos?
These are the links I found to Mesos:
https://github.com/mesos/storm
https://github.com/nathanmarz/storm-mesos
http://mesos.apache.org/
Thank you for your time!
+
Jeff
Mesos is very similar to YARN. It is a resource scheduler. Storm in the past
had support for mesos, through a separate repo
https://github.com/nathanmarz/storm-mesos
it might still work with the latest versions of storm. I don't know. The
concept here is that there was a special layer
storm-yarn was originally done as a proof of concept. We had plans to take it
further, but the amount of work required to make it production ready on a very
heavily used cluster was more then we were willing to invest at the time. Most
of that work was around network scheduling, isolation and
I also developed a prototype/proof-of-concept (read: duck tape and bailer
twine) for running Storm on YARN.
I took a slightly different approach than Yahoo’s storm-yarn and Slider which
from a high level allow you to spin up a Storm cluster on top of YARN. In my
PoC a topology is treated as a
Dell - Internal Use - Confidential
Thanks Bobby, for the detailed answer.
So it sounds like , it is better not to combine Storm with batch workloads at
this point (yarn, mesos or ec2), due to the network saturation and timeout
threats.
Is this behavior also seen in other streaming frameworks
Hi Nathan,
I want to do real time computation using storm, which one is best storm or
trident. i need to handle huge amount of data , exactly once please help me
Thanks!
Date: Wed, 27 May 2015 12:40:43 -0400
Subject: Re: Status of running storm on yarn (the yahoo project)
From:
Mesosphere has official support for Storm on Mesos:
https://github.com/mesos/storm
On Wed, May 27, 2015 at 11:14 AM, rajesh_kall...@dellteam.com wrote:
*Dell - Internal Use - Confidential *
Thanks Bobby, for the detailed answer.
So it sounds like , it is better not to combine Storm with
