Sure, The main thing that storm is missing right now is an metrics feedback system to the scheduler (or possibly a separate piece of code that decides when to grow/shrink a topology). We have some basic metrics, but they are not exposed to the scheduler. The other question is how do we deal with creating/destroying new bolts, especially around dealing with different groupings. Some groupings make it difficult. There a number of ways to get around this, but I think the simplest way is to not create/destroy instances, but to treat it as a resources problem, and give them more or less resources as needed. But that still needs to be discussed. - Bobby
On Wednesday, November 12, 2014 3:10 PM, Harsha Balasubramanian <harsha...@gmail.com> wrote: Hi Bobby, Thanks for getting back to me. It is encouraging to hear that this will be a good addition to Storm if done well. I have not implemented the changes yet. My project just started. It spans over the next 2 months. In a few days, I will create a JIRA task and submit my proposal. It would be great to brainstorm with the Storm community and iron out my design. Thanks, Harsha Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate Student at University of Toronto Professional Masters in Computer Science Email 1: harsha...@cs.toronto.edu Email 2: harsha...@gmail.com LinkedIn : ca.linkedin.com/in/harshabala/ On Wed, Nov 12, 2014 at 3:47 PM, Bobby Evans <ev...@yahoo-inc.com.invalid> wrote: > Yes, this type of a feature would be great to have. I am rally curious > how you have done the changes, as there are a lot of potential pitfalls > here. At a minimum it would just be great to have the feedback framework > in place so we can iterate on these changes. > - Bobby > > > On Wednesday, November 12, 2014 2:05 PM, Harsha <st...@harsha.io> > wrote: > > > Hi, > It will definitely interesting to the storm community. It will be > great if you can create a JIRA and post your code as PR for others > to try out and review the code. > Thanks, > Harsha > > On Wed, Nov 12, 2014, at 11:16 AM, Harsha Balasubramanian wrote: > > Please let me know if my project (outlined below) will be useful to > > Storm. > > > > Thanks, > > Harsha > > > > *Contact Details: * > > Harsha Balasubramanian > > Graduate Student at University of Toronto > > Professional Masters in Computer Science > > > > Email 1: harsha...@cs.toronto.edu > > Email 2: harsha...@gmail.com > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > ---------- Forwarded message ---------- > > From: Harsha Balasubramanian <harsha...@gmail.com> > > Date: Tue, Nov 11, 2014 at 8:15 PM > > Subject: Auto-Scaling > > To: dev@storm.apache.org > > > > > > Hi, > > > > I am a Graduate student at the University of Toronto. As part of my > > Advanced Database Systems course, I have proposed to implement an Auto > > Scaling mechanism for Storm topologies (using a Feedback System). > > > > I've gone through the pages on how to contribute to the Storm project and > > have some questions. Please let me know if auto-scaling is being worked > > on > > currently. Also, should this be a project in StormCore or StormProcessor > > ? > > > > If this project will be a good addition to Storm, I will create a new > > JIRA > > task for it and add the details of my proposed implementation. Please do > > let me know. > > > > Thanks, > > Harsha > > > > *Contact Details: * > > Harsha Balasubramanian > > Graduate Student at University of Toronto > > Professional Masters in Computer Science > > > > Email 1: harsha...@cs.toronto.edu > > Email 2: harsha...@gmail.com > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > >