I built the storm deliverable from main branch (g drive link to tar and zip files), https://drive.google.com/folderview?id=0B9th7RvhGLuiMHc5LWpobWY1czg&usp=sharing
Then I deployed it into my c: drive - c:/storm (which is STORM_HOME) When I run the python script from STORM_HOME, I get the same error as before. *Is there any setting that distinguishes the build as a release bundle* H@H-PC /c/storm $ python ./bin/storm nimbus ****************************************** The storm client can only be run from within a release. You appear to be trying to run the client from a checkout of Storm's source code. You can download a Storm release at http://storm-project.net/downloads.html ****************************************** H@H-PC /c/storm $ Thanks, Harsha *Contact Details: * Harsha Balasubramanian Graduate Student at University of Toronto Masters in Computer Science Mobile : 647-771-3596 Email 1: [email protected] Email 2: [email protected] LinkedIn : ca.linkedin.com/in/harshabala/ On Fri, Dec 26, 2014 at 12:40 PM, Harsha <[email protected]> wrote: > Yes he meant "master". > On your previous mail > "When I tried, c:\python>python c:\storm\bin\storm list, I got this > error" > you need to build storm and deploy . Running it from source doesn't > work. > and the correct script is storm\bin\storm like you are using above. > > > > On Fri, Dec 26, 2014, at 09:23 AM, Harsha Balasubramanian wrote: > > Hi Bobby, > > > > When you said 'trunk', did you mean the master branch from > > https://github.com/apache/storm? > > > > Thanks, > > Harsha > > > > *Contact Details: * > > Harsha Balasubramanian > > Graduate Student at University of Toronto > > Masters in Computer Science > > > > Mobile : 647-771-3596 > > Email 1: [email protected] > > Email 2: [email protected] > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > On Wed, Dec 24, 2014 at 4:13 PM, Harsha <[email protected]> wrote: > > > > > looks like there is a bug storm.cmd in make_command_arguments method. > > > Its converting -e count=10 to -e count 10 (removing =) which is causing > > > this issue. > > > If you using trunk you can look STORM-487 which makes python storm > > > command work in windows. > > > > > > > > > On Wed, Dec 24, 2014, at 12:09 PM, Harsha Balasubramanian wrote: > > > > Thanks Bobby. > > > > > > > > I setup a local server and CLI, UI and my topology run fine. However > > > > there > > > > is a feature that doesn't work and I need some help to figure out > why. > > > > > > > > 1. I followed the instructions here to setup the storm system, > > > > > > > > http://ptgoetz.github.io/blog/2013/12/18/running-apache-storm-on-windows/ > > > > > > > > 2. My feature requires the usage of the CLI tool to re-balance the > > > > topology > > > > when it needs more resources. For that I thought I could use the > > > > following > > > > CLI command, > > > > > > > > storm rebalance mytopology -n 5 -e blue-spout=3 -e yellow-bolt=10 > > > > > > > > I found this in the storm documentation at the bottom of this page: > > > > > > > > http://storm.apache.org/documentation/Understanding-the-parallelism-of-a-Storm-topology.html > > > > > > > > 3. When I run this command in my setup, I get the error pasted > below. Is > > > > re-balancing supported at the spouts and bolts level? Am I using the > > > > wrong > > > > version of the storm jar. > > > > > > > > This works: > > > > *C:\storm>storm rebalance WordCount -n 15* > > > > *2493 [main] INFO backtype.storm.thrift - Connecting to Nimbus at > > > > localhost:6627* > > > > *3573 [main] INFO backtype.storm.command.rebalance - Topology > WordCount > > > > is > > > > rebalancing* > > > > > > > > This does not work: > > > > *C:\storm>storm rebalance WordCount -n 5 -e Count=10* > > > > *Exception in thread "main" > java.lang.StringIndexOutOfBoundsException: > > > > String index out of range: -1* > > > > * at java.lang.String.substring(Unknown Source)* > > > > * at > > > > > backtype.storm.command.rebalance$parse_executor.invoke(rebalance.clj:24)* > > > > * at clojure.tools.cli$apply_specs.invoke(cli.clj:80)* > > > > * at clojure.tools.cli$cli.doInvoke(cli.clj:130)* > > > > * at clojure.lang.RestFn.invoke(RestFn.java:460)* > > > > * at > > > > backtype.storm.command.rebalance$_main.doInvoke(rebalance.clj:31)* > > > > * at clojure.lang.RestFn.applyTo(RestFn.java:137)* > > > > * at backtype.storm.command.rebalance.main(Unknown Source)* > > > > > > > > > > > > > > > > Thanks, > > > > Harsha > > > > > > > > *Contact Details: * > > > > Harsha Balasubramanian > > > > Graduate Student at University of Toronto > > > > Masters in Computer Science > > > > > > > > Mobile : 647-771-3596 > > > > Email 1: [email protected] > > > > Email 2: [email protected] > > > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > > > > On Mon, Dec 22, 2014 at 4:42 PM, Bobby Evans > > > > <[email protected]> > > > > wrote: > > > > > > > > > Harsha, > > > > > Sorry about that, I have been out at a conference. > > > > > > > > > > for 1 you can get the number of acks so far, but you are going to > have > > > to > > > > > calculate the rate yourself, it keeps track of the number of acks > over > > > > > several time windows, but reports them as whole numbers. > > > > > What do you mean by a local cluster? Do you mean a single node > cluster > > > > > you launched yourself? then yes. If you mean a local-mode cluster, > > > then no. > > > > > - Bobby > > > > > > > > > > > > > > > On Tuesday, December 16, 2014 2:35 PM, Harsha Balasubramanian > < > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > Can someone pls answer my questions from the above thread. > > > > > > > > > > > > > > > *I have 2 questions for you;* > > > > > *1. In one of your earlier emails, you mentioned that there is > > > > > a limited Metrics implementation which is available in the current > > > Storm > > > > > version. * > > > > > > > > > > *Will this provide 'acks threshold', i.e, the number of acks per > > > second (or > > > > > minute)? If not, can you please suggest a way in which I can > implement > > > this > > > > > myself. * > > > > > > > > > > *2. Is it possible to use the CLI and UI tools from a local > cluster?* > > > > > > > > > > Thanks, > > > > > Harsha > > > > > > > > > > *Contact Details: * > > > > > Harsha Balasubramanian > > > > > Graduate Student at University of Toronto > > > > > Masters in Computer Science > > > > > > > > > > Mobile : 647-771-3596 > > > > > Email 1: [email protected] > > > > > Email 2: [email protected] > > > > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > > > > > > On Mon, Dec 15, 2014 at 12:23 PM, Harsha Balasubramanian < > > > > > [email protected]> wrote: > > > > > > > > > > > > Hi Bobby, > > > > > > > > > > > > Hope this email finds you well. I.m about to start designing the > > > > > > auto-scaling system. Created a JIRA for it, > > > > > > https://issues.apache.org/jira/browse/STORM-594 > > > > > > > > > > > > Please take a look and let me know if there are any concerns. > > > > > > > > > > > > I have 2 questions for you; > > > > > > 1. In one of your earlier emails, you mentioned that there is a > > > *limited > > > > > *Metrics > > > > > > implementation which is available in the current Storm version. > > > > > > Will this provide 'acks threshold'? > > > > > > > > > > > > 2. Is it possible to use the CLI tool from a local cluster? > > > > > > > > > > > > Thanks, > > > > > > Harsha > > > > > > > > > > > > *Contact Details: * > > > > > > Harsha Balasubramanian > > > > > > Graduate Student at University of Toronto > > > > > > Masters in Computer Science > > > > > > > > > > > > Mobile : 647-771-3596 > > > > > > Email 1: [email protected] > > > > > > Email 2: [email protected] > > > > > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > > > > > > > > On Wed, Nov 12, 2014 at 6:28 PM, Harsha Balasubramanian < > > > > > > [email protected]> wrote: > > > > > >> > > > > > >> Thanks for the detailed explanation Bobby. I will keep this in > mind > > > when > > > > > >> I create my design document. > > > > > >> > > > > > >> I will mostly not deal with adding/removing bolts to the > topology > > > and > > > > > >> restrict myself to threads and tasks. This is because of the > time I > > > > > have to > > > > > >> complete the project. Once I submit my report (early January), > I can > > > > > extend > > > > > >> what I have implemented to more use cases. > > > > > >> > > > > > >> Thanks, > > > > > >> Harsha > > > > > >> > > > > > >> Thanks, > > > > > >> Harsha > > > > > >> > > > > > >> *Contact Details: * > > > > > >> Harsha Balasubramanian > > > > > >> Graduate Student at University of Toronto > > > > > >> Professional Masters in Computer Science > > > > > >> > > > > > >> Email 1: [email protected] > > > > > >> Email 2: [email protected] > > > > > >> LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > >> > > > > > >> On Wed, Nov 12, 2014 at 6:05 PM, Bobby Evans > > > > > <[email protected] > > > > > >> > wrote: > > > > > >> > > > > > >>> Sure, > > > > > >>> The main thing that storm is missing right now is an metrics > > > feedback > > > > > >>> system to the scheduler (or possibly a separate piece of code > that > > > > > decides > > > > > >>> when to grow/shrink a topology). We have some basic metrics, > but > > > they > > > > > are > > > > > >>> not exposed to the scheduler. The other question is how do we > deal > > > > > with > > > > > >>> creating/destroying new bolts, especially around dealing with > > > different > > > > > >>> groupings. Some groupings make it difficult. There a number > of > > > ways > > > > > to > > > > > >>> get around this, but I think the simplest way is to not > > > create/destroy > > > > > >>> instances, but to treat it as a resources problem, and give > them > > > more > > > > > or > > > > > >>> less resources as needed. But that still needs to be > discussed. - > > > > > Bobby > > > > > >>> > > > > > >>> > > > > > >>> On Wednesday, November 12, 2014 3:10 PM, Harsha > > > Balasubramanian < > > > > > >>> [email protected]> wrote: > > > > > >>> > > > > > >>> > > > > > >>> Hi Bobby, > > > > > >>> > > > > > >>> Thanks for getting back to me. It is encouraging to hear that > this > > > > > will > > > > > >>> be a good addition to Storm if done well. > > > > > >>> > > > > > >>> I have not implemented the changes yet. My project just > started. > > > It > > > > > >>> spans > > > > > >>> over the next 2 months. In a few days, I will create a JIRA > task > > > and > > > > > >>> submit > > > > > >>> my proposal. It would be great to brainstorm with the Storm > > > community > > > > > and > > > > > >>> iron out my design. > > > > > >>> > > > > > >>> > > > > > >>> Thanks, > > > > > >>> Harsha > > > > > >>> > > > > > >>> Thanks, > > > > > >>> Harsha > > > > > >>> > > > > > >>> *Contact Details: * > > > > > >>> Harsha Balasubramanian > > > > > >>> Graduate Student at University of Toronto > > > > > >>> Professional Masters in Computer Science > > > > > >>> > > > > > >>> Email 1: [email protected] > > > > > >>> Email 2: [email protected] > > > > > >>> LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > >>> > > > > > >>> On Wed, Nov 12, 2014 at 3:47 PM, Bobby Evans > > > > > <[email protected] > > > > > >>> > > > > > > >>> wrote: > > > > > >>> > > > > > >>> > Yes, this type of a feature would be great to have. I am > rally > > > > > curious > > > > > >>> > how you have done the changes, as there are a lot of > potential > > > > > pitfalls > > > > > >>> > here. At a minimum it would just be great to have the > feedback > > > > > >>> framework > > > > > >>> > in place so we can iterate on these changes. > > > > > >>> > - Bobby > > > > > >>> > > > > > > >>> > > > > > > >>> > On Wednesday, November 12, 2014 2:05 PM, Harsha < > > > > > [email protected]> > > > > > >>> > wrote: > > > > > >>> > > > > > > >>> > > > > > > >>> > Hi, > > > > > >>> > It will definitely interesting to the storm community. It > > > will be > > > > > >>> > great if you can create a JIRA and post your code as PR > for > > > others > > > > > >>> > to try out and review the code. > > > > > >>> > Thanks, > > > > > >>> > Harsha > > > > > >>> > > > > > > >>> > On Wed, Nov 12, 2014, at 11:16 AM, Harsha Balasubramanian > wrote: > > > > > >>> > > Please let me know if my project (outlined below) will be > > > useful to > > > > > >>> > > Storm. > > > > > >>> > > > > > > > >>> > > Thanks, > > > > > >>> > > Harsha > > > > > >>> > > > > > > > >>> > > *Contact Details: * > > > > > >>> > > Harsha Balasubramanian > > > > > >>> > > Graduate Student at University of Toronto > > > > > >>> > > Professional Masters in Computer Science > > > > > >>> > > > > > > > >>> > > Email 1: [email protected] > > > > > >>> > > Email 2: [email protected] > > > > > >>> > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > >>> > > > > > > > >>> > > ---------- Forwarded message ---------- > > > > > >>> > > From: Harsha Balasubramanian <[email protected]> > > > > > >>> > > Date: Tue, Nov 11, 2014 at 8:15 PM > > > > > >>> > > Subject: Auto-Scaling > > > > > >>> > > To: [email protected] > > > > > >>> > > > > > > > >>> > > > > > > > >>> > > Hi, > > > > > >>> > > > > > > > >>> > > I am a Graduate student at the University of Toronto. As > part > > > of > > > > > my > > > > > >>> > > Advanced Database Systems course, I have proposed to > implement > > > an > > > > > >>> Auto > > > > > >>> > > Scaling mechanism for Storm topologies (using a Feedback > > > System). > > > > > >>> > > > > > > > >>> > > I've gone through the pages on how to contribute to the > Storm > > > > > >>> project and > > > > > >>> > > have some questions. Please let me know if auto-scaling is > > > being > > > > > >>> worked > > > > > >>> > > on > > > > > >>> > > currently. Also, should this be a project in StormCore or > > > > > >>> StormProcessor > > > > > >>> > > ? > > > > > >>> > > > > > > > >>> > > If this project will be a good addition to Storm, I will > > > create a > > > > > new > > > > > >>> > > JIRA > > > > > >>> > > task for it and add the details of my proposed > implementation. > > > > > >>> Please do > > > > > >>> > > let me know. > > > > > >>> > > > > > > > >>> > > Thanks, > > > > > >>> > > Harsha > > > > > >>> > > > > > > > >>> > > *Contact Details: * > > > > > >>> > > Harsha Balasubramanian > > > > > >>> > > Graduate Student at University of Toronto > > > > > >>> > > Professional Masters in Computer Science > > > > > >>> > > > > > > > >>> > > Email 1: [email protected] > > > > > >>> > > Email 2: [email protected] > > > > > >>> > > LinkedIn : ca.linkedin.com/in/harshabala/ > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > >
