Thanks for the detailed explanation Bobby. I will keep this in mind when I create my design document.
I will mostly not deal with adding/removing bolts to the topology and restrict myself to threads and tasks. This is because of the time I have to complete the project. Once I submit my report (early January), I can extend what I have implemented to more use cases. Thanks, Harsha Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate Student at University of Toronto Professional Masters in Computer Science Email 1: harsha...@cs.toronto.edu Email 2: harsha...@gmail.com LinkedIn : ca.linkedin.com/in/harshabala/ On Wed, Nov 12, 2014 at 6:05 PM, Bobby Evans <ev...@yahoo-inc.com.invalid> wrote: > Sure, > The main thing that storm is missing right now is an metrics feedback > system to the scheduler (or possibly a separate piece of code that decides > when to grow/shrink a topology). We have some basic metrics, but they are > not exposed to the scheduler. The other question is how do we deal with > creating/destroying new bolts, especially around dealing with different > groupings. Some groupings make it difficult. There a number of ways to > get around this, but I think the simplest way is to not create/destroy > instances, but to treat it as a resources problem, and give them more or > less resources as needed. But that still needs to be discussed. - Bobby > > > On Wednesday, November 12, 2014 3:10 PM, Harsha Balasubramanian < > harsha...@gmail.com> wrote: > > > Hi Bobby, > > Thanks for getting back to me. It is encouraging to hear that this will > be a good addition to Storm if done well. > > I have not implemented the changes yet. My project just started. It spans > over the next 2 months. In a few days, I will create a JIRA task and submit > my proposal. It would be great to brainstorm with the Storm community and > iron out my design. > > > Thanks, > Harsha > > Thanks, > Harsha > > *Contact Details: * > Harsha Balasubramanian > Graduate Student at University of Toronto > Professional Masters in Computer Science > > Email 1: harsha...@cs.toronto.edu > Email 2: harsha...@gmail.com > LinkedIn : ca.linkedin.com/in/harshabala/ > > On Wed, Nov 12, 2014 at 3:47 PM, Bobby Evans <ev...@yahoo-inc.com.invalid> > wrote: > > > Yes, this type of a feature would be great to have. I am rally curious > > how you have done the changes, as there are a lot of potential pitfalls > > here. At a minimum it would just be great to have the feedback framework > > in place so we can iterate on these changes. > > - Bobby > > > > > > On Wednesday, November 12, 2014 2:05 PM, Harsha <st...@harsha.io> > > wrote: > > > > > > Hi, > > It will definitely interesting to the storm community. It will be > > great if you can create a JIRA and post your code as PR for others > > to try out and review the code. > > Thanks, > > Harsha > > > > On Wed, Nov 12, 2014, at 11:16 AM, Harsha Balasubramanian wrote: > > > Please let me know if my project (outlined below) will be useful to > > > Storm. > > > > > > Thanks, > > > Harsha > > > > > > *Contact Details: * > > > Harsha Balasubramanian > > > Graduate Student at University of Toronto > > > Professional Masters in Computer Science > > > > > > Email 1: harsha...@cs.toronto.edu > > > Email 2: harsha...@gmail.com > > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > > ---------- Forwarded message ---------- > > > From: Harsha Balasubramanian <harsha...@gmail.com> > > > Date: Tue, Nov 11, 2014 at 8:15 PM > > > Subject: Auto-Scaling > > > To: dev@storm.apache.org > > > > > > > > > Hi, > > > > > > I am a Graduate student at the University of Toronto. As part of my > > > Advanced Database Systems course, I have proposed to implement an Auto > > > Scaling mechanism for Storm topologies (using a Feedback System). > > > > > > I've gone through the pages on how to contribute to the Storm project > and > > > have some questions. Please let me know if auto-scaling is being worked > > > on > > > currently. Also, should this be a project in StormCore or > StormProcessor > > > ? > > > > > > If this project will be a good addition to Storm, I will create a new > > > JIRA > > > task for it and add the details of my proposed implementation. Please > do > > > let me know. > > > > > > Thanks, > > > Harsha > > > > > > *Contact Details: * > > > Harsha Balasubramanian > > > Graduate Student at University of Toronto > > > Professional Masters in Computer Science > > > > > > Email 1: harsha...@cs.toronto.edu > > > Email 2: harsha...@gmail.com > > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > > > > > > > >
