Sure,
The main thing that storm is missing right now is an metrics feedback system to
the scheduler (or possibly a separate piece of code that decides when to
grow/shrink a topology). We have some basic metrics, but they are not exposed
to the scheduler. The other question is how do we deal with
creating/destroying new bolts, especially around dealing with different
groupings. Some groupings make it difficult. There a number of ways to get
around this, but I think the simplest way is to not create/destroy instances,
but to treat it as a resources problem, and give them more or less resources as
needed. But that still needs to be discussed. - Bobby
On Wednesday, November 12, 2014 3:10 PM, Harsha Balasubramanian
<harsha...@gmail.com> wrote:
Hi Bobby,
Thanks for getting back to me. It is encouraging to hear that this will
be a good addition to Storm if done well.
I have not implemented the changes yet. My project just started. It spans
over the next 2 months. In a few days, I will create a JIRA task and submit
my proposal. It would be great to brainstorm with the Storm community and
iron out my design.
Thanks,
Harsha
Thanks,
Harsha
*Contact Details: *
Harsha Balasubramanian
Graduate Student at University of Toronto
Professional Masters in Computer Science
Email 1: harsha...@cs.toronto.edu
Email 2: harsha...@gmail.com
LinkedIn : ca.linkedin.com/in/harshabala/
On Wed, Nov 12, 2014 at 3:47 PM, Bobby Evans <ev...@yahoo-inc.com.invalid>
wrote:
> Yes, this type of a feature would be great to have. I am rally curious
> how you have done the changes, as there are a lot of potential pitfalls
> here. At a minimum it would just be great to have the feedback framework
> in place so we can iterate on these changes.
> - Bobby
>
>
> On Wednesday, November 12, 2014 2:05 PM, Harsha <st...@harsha.io>
> wrote:
>
>
> Hi,
> It will definitely interesting to the storm community. It will be
> great if you can create a JIRA and post your code as PR for others
> to try out and review the code.
> Thanks,
> Harsha
>
> On Wed, Nov 12, 2014, at 11:16 AM, Harsha Balasubramanian wrote:
> > Please let me know if my project (outlined below) will be useful to
> > Storm.
> >
> > Thanks,
> > Harsha
> >
> > *Contact Details: *
> > Harsha Balasubramanian
> > Graduate Student at University of Toronto
> > Professional Masters in Computer Science
> >
> > Email 1: harsha...@cs.toronto.edu
> > Email 2: harsha...@gmail.com
> > LinkedIn : ca.linkedin.com/in/harshabala/
> >
> > ---------- Forwarded message ----------
> > From: Harsha Balasubramanian <harsha...@gmail.com>
> > Date: Tue, Nov 11, 2014 at 8:15 PM
> > Subject: Auto-Scaling
> > To: dev@storm.apache.org
> >
> >
> > Hi,
> >
> > I am a Graduate student at the University of Toronto. As part of my
> > Advanced Database Systems course, I have proposed to implement an Auto
> > Scaling mechanism for Storm topologies (using a Feedback System).
> >
> > I've gone through the pages on how to contribute to the Storm project and
> > have some questions. Please let me know if auto-scaling is being worked
> > on
> > currently. Also, should this be a project in StormCore or StormProcessor
> > ?
> >
> > If this project will be a good addition to Storm, I will create a new
> > JIRA
> > task for it and add the details of my proposed implementation. Please do
> > let me know.
> >
> > Thanks,
> > Harsha
> >
> > *Contact Details: *
> > Harsha Balasubramanian
> > Graduate Student at University of Toronto
> > Professional Masters in Computer Science
> >
> > Email 1: harsha...@cs.toronto.edu
> > Email 2: harsha...@gmail.com
> > LinkedIn : ca.linkedin.com/in/harshabala/
>
>
>
>
