Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
Hi Bobby, I seem to get everything I need working except this part. It seems like I'm making a simple mistake. I checked the JIRA post you gave and tried the following. 1. When I tried, c:\pythonpython c:\storm\bin\storm list, I got this error *The

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
Hi Bobby, When you said 'trunk', did you mean the master branch from https://github.com/apache/storm? Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate Student at University of Toronto Masters in Computer Science Mobile : 647-771-3596 Email 1: harsha...@cs.toronto.edu Email

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
I built the storm deliverable from main branch (g drive link to tar and zip files), https://drive.google.com/folderview?id=0B9th7RvhGLuiMHc5LWpobWY1czgusp=sharing Then I deployed it into my c: drive - c:/storm (which is STORM_HOME) When I run the python script from STORM_HOME, I get the same

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
FYI: Running 'storm nimbus', 'storm ui' and 'storm supervisor' works fine from the STORM_HOME dir. The problem comes only when I try to run using python to run these commands. I want to use python only because I need the CLI tool to re-balance the topology that I am using. In windows, storm.cmd

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha
your build is older version of storm. Do a git pull and rebuild the storm. I don't see STORM-487 fix in your build. https://github.com/apache/storm/blob/master/bin/storm that is the latest version and I tested in windows 8 don't see any issues. -Harsha On Fri, Dec 26, 2014, at 02:16 PM, Harsha

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
i'll create a new fork and try again Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate Student at University of Toronto Masters in Computer Science Mobile : 647-771-3596 Email 1: harsha...@cs.toronto.edu Email 2: harsha...@gmail.com LinkedIn : ca.linkedin.com/in/harshabala/ On

Re: Fwd: Auto-Scaling

2014-12-26 Thread Harsha Balasubramanian
ok. I will try again with my new fork now. I compared files from my fork and master branch of storm and they look the same. Quick question: to run the pythin script, do i need to add this env variable in windows PATHEXT = .PY Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate

Re: Fwd: Auto-Scaling

2014-12-26 Thread Nathan Leung
You can't change the number of tasks. Changing the number of tasks will, for example, break any fields grouping you have. On Dec 26, 2014 7:03 PM, Harsha Balasubramanian harsha...@gmail.com wrote: Works Fine Now!! Thanks a ton to everyone who helped me get this going. You folks are incredibly

Re: Fwd: Auto-Scaling

2014-12-24 Thread Harsha Balasubramanian
Thanks Bobby. I setup a local server and CLI, UI and my topology run fine. However there is a feature that doesn't work and I need some help to figure out why. 1. I followed the instructions here to setup the storm system, http://ptgoetz.github.io/blog/2013/12/18/running-apache-storm-on-windows/

Re: Fwd: Auto-Scaling

2014-12-24 Thread Harsha
looks like there is a bug storm.cmd in make_command_arguments method. Its converting -e count=10 to -e count 10 (removing =) which is causing this issue. If you using trunk you can look STORM-487 which makes python storm command work in windows. On Wed, Dec 24, 2014, at 12:09 PM, Harsha

Re: Fwd: Auto-Scaling

2014-12-22 Thread Bobby Evans
Harsha, Sorry about that, I have been out at a conference.  for 1 you can get the number of acks so far, but you are going to have to calculate the rate yourself, it keeps track of the number of acks over several time windows, but reports them as whole numbers. What do you mean by a local

Re: Fwd: Auto-Scaling

2014-12-16 Thread Harsha Balasubramanian
Hi, Can someone pls answer my questions from the above thread. *I have 2 questions for you;* *1. In one of your earlier emails, you mentioned that there is a limited Metrics implementation which is available in the current Storm version. * *Will this provide 'acks threshold', i.e, the number

Re: Fwd: Auto-Scaling

2014-12-15 Thread Harsha Balasubramanian
Hi Bobby, Hope this email finds you well. I.m about to start designing the auto-scaling system. Created a JIRA for it, https://issues.apache.org/jira/browse/STORM-594 Please take a look and let me know if there are any concerns. I have 2 questions for you; 1. In one of your earlier emails,

Re: Fwd: Auto-Scaling

2014-11-12 Thread Bobby Evans
Yes, this type of a feature would be great to have.  I am rally curious how you have done the changes, as there are a lot of potential pitfalls here.  At a minimum it would just be great to have the feedback framework in place so we can iterate on these changes.  - Bobby On Wednesday,

Re: Fwd: Auto-Scaling

2014-11-12 Thread Bobby Evans
Sure, The main thing that storm is missing right now is an metrics feedback system to the scheduler (or possibly a separate piece of code that decides when to grow/shrink a topology).  We have some basic metrics, but they are not exposed to the scheduler.  The other question is how do we deal