[jira] [Commented] (STORM-605) Attempting to call unbound fn during bolt prepare

2015-02-27 Thread Li Chaomo (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341389#comment-14341389
 ] 

Li Chaomo commented on STORM-605:
-

[~sriharsha] [~phyce]
Hello , i found the root cause .
Because we package storm-jar include the clj sources
Storm may load the .clj source or .class , they conflict 

 Attempting to call unbound fn during bolt prepare
 -

 Key: STORM-605
 URL: https://issues.apache.org/jira/browse/STORM-605
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.9.3
Reporter: Philippe Guillebert

 We had a bunch of topologies running very well under Storm 0.8.2 until last
 week when we switched to storm 0.9.2-incubating. We use the clojure DSL,
 and clojure 1.5.1 (only).
 Since the change, we have a large topology (about 30 bolts, parallellism=10
 or 20 per bolt, total 372 tasks on 10 workers) that fails on startup with
 several bolts showing the exception :
 java.lang.RuntimeException: java.lang.IllegalStateException: Attempting to
 call unbound fn: #'entry-dedup.bolt/dedup__ at
 backtype.storm.clojure.ClojureBolt.prepare(ClojureBolt.java:77) ...
 This can occur on one or several bolts at random and is not consistent
 between restarts.
 The topology is indeed quite long to initialize (a dozen seconds) due to 
 several models being loaded but this was OK in 0.8.2.
 Another (shorter) topology works most of the time but shows this behaviour
 on some restarts sometimes.
 We found a workaround that works most of the time : start the topology in
 the INACTIVE state, then wait 200 seconds, then activate it. But this
 doesn't really solve our problem because sometimes Storm tries to rebalance
 the topologies by itself and reassigns the topology without our little trick, 
 effectively crashing them.
 The same behavior is present with storm 0.9.3.
 So maybe something changed in storm that introduces a kind of race
 condition during initializaion of some bolts on larger topologies ? Maybe 
 this is a consequence to the switch to Netty ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: STORM-446: Allow superusers to impersonate oth...

2015-02-27 Thread revans2
Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76430788
  
I haven't gone deeply into the code, but for the most part it looks good.  
Just one request that should hopefully not be too difficult to change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] storm pull request: STORM-446: Allow superusers to impersonate oth...

2015-02-27 Thread revans2
Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/448#discussion_r25521108
  
--- Diff: 
storm-core/src/jvm/backtype/storm/security/auth/authorizer/SimpleACLAuthorizer.java
 ---
@@ -50,6 +50,7 @@
 protected SetString _supervisors;
 protected IPrincipalToLocal _ptol;
 protected IGroupMappingServiceProvider _groupMappingProvider;
+protected ImpersonationAuthorizer _impersonationAuthorizer;
--- End diff --

I don't really like the idea of having all authorizers have to be updated 
to support the impersonation code.  I would much rather have the code spliced 
in at a much higher level that cannot be turned off.  Otherwise a custom 
IAuthorizer instance that does not know to update after this change is now wide 
open for anyone to impersonate anyone else.  The logging changes should also 
probably happen at the same level.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (STORM-446) secure Impersonation in storm

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340382#comment-14340382
 ] 

ASF GitHub Bot commented on STORM-446:
--

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76430788
  
I haven't gone deeply into the code, but for the most part it looks good.  
Just one request that should hopefully not be too difficult to change.


 secure Impersonation in storm
 -

 Key: STORM-446
 URL: https://issues.apache.org/jira/browse/STORM-446
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Sriharsha Chintalapani
Assignee: Parth Brahmbhatt
  Labels: Security

 Storm security adds features of authenticating with kerberos and than uses 
 that principal and TGT as way to authorize user operations, topology 
 operation. Currently Storm UI user needs to be part of nimbus.admins to get 
 details on user submitted topologies. Ideally storm ui needs to take 
 authenticated user  principal to submit requests to nimbus which will than 
 authorize the user rather than storm UI user. This feature will also benefit 
 superusers to impersonate other users to submit topologies in a secured way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (STORM-446) secure Impersonation in storm

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340514#comment-14340514
 ] 

ASF GitHub Bot commented on STORM-446:
--

Github user Parth-Brahmbhatt commented on a diff in the pull request:

https://github.com/apache/storm/pull/448#discussion_r25526790
  
--- Diff: 
storm-core/src/jvm/backtype/storm/security/auth/authorizer/SimpleACLAuthorizer.java
 ---
@@ -50,6 +50,7 @@
 protected SetString _supervisors;
 protected IPrincipalToLocal _ptol;
 protected IGroupMappingServiceProvider _groupMappingProvider;
+protected ImpersonationAuthorizer _impersonationAuthorizer;
--- End diff --

Ok, I am going to add this authorization as part of nimbus 
check-authorization! method. I will also add nimbus.impersonation.authorizer 
config to allow for overriding behavior. I will also add the logging changes 
there. 


 secure Impersonation in storm
 -

 Key: STORM-446
 URL: https://issues.apache.org/jira/browse/STORM-446
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Sriharsha Chintalapani
Assignee: Parth Brahmbhatt
  Labels: Security

 Storm security adds features of authenticating with kerberos and than uses 
 that principal and TGT as way to authorize user operations, topology 
 operation. Currently Storm UI user needs to be part of nimbus.admins to get 
 details on user submitted topologies. Ideally storm ui needs to take 
 authenticated user  principal to submit requests to nimbus which will than 
 authorize the user rather than storm UI user. This feature will also benefit 
 superusers to impersonate other users to submit topologies in a secured way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (STORM-446) secure Impersonation in storm

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340518#comment-14340518
 ] 

ASF GitHub Bot commented on STORM-446:
--

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76446301
  
@revans2 @harshach  For supporting the actual jira issue ui server being 
able to impersonate users we probably need to support wildcards (allow userX 
to impersonate all users(*) from localhost) or allow the nimbus.admin users 
to impersonate all users from all hosts.  Any preferences?


 secure Impersonation in storm
 -

 Key: STORM-446
 URL: https://issues.apache.org/jira/browse/STORM-446
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Sriharsha Chintalapani
Assignee: Parth Brahmbhatt
  Labels: Security

 Storm security adds features of authenticating with kerberos and than uses 
 that principal and TGT as way to authorize user operations, topology 
 operation. Currently Storm UI user needs to be part of nimbus.admins to get 
 details on user submitted topologies. Ideally storm ui needs to take 
 authenticated user  principal to submit requests to nimbus which will than 
 authorize the user rather than storm UI user. This feature will also benefit 
 superusers to impersonate other users to submit topologies in a secured way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: STORM-446: Allow superusers to impersonate oth...

2015-02-27 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76446301
  
@revans2 @harshach  For supporting the actual jira issue ui server being 
able to impersonate users we probably need to support wildcards (allow userX 
to impersonate all users(*) from localhost) or allow the nimbus.admin users 
to impersonate all users from all hosts.  Any preferences?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (STORM-696) As a dev I would like to be able to run unit tests in a particular namespace.

2015-02-27 Thread Kyle Nusbaum (JIRA)
Kyle Nusbaum created STORM-696:
--

 Summary: As a dev I would like to be able to run unit tests in a 
particular namespace.
 Key: STORM-696
 URL: https://issues.apache.org/jira/browse/STORM-696
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Kyle Nusbaum
Assignee: Kyle Nusbaum
Priority: Minor


Right now, we have to run the full test suite through maven. If we want to test 
particular namespaces, we must do so from a REPL.

Some quick way to launch unit tests for a namespace is desirable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (STORM-696) Single Namespace Test Launching

2015-02-27 Thread Kyle Nusbaum (JIRA)

 [ 
https://issues.apache.org/jira/browse/STORM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kyle Nusbaum updated STORM-696:
---
Summary: Single Namespace Test Launching  (was: As a dev I would like to be 
able to run unit tests in a particular namespace.)

 Single Namespace Test Launching
 ---

 Key: STORM-696
 URL: https://issues.apache.org/jira/browse/STORM-696
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Kyle Nusbaum
Assignee: Kyle Nusbaum
Priority: Minor

 Right now, we have to run the full test suite through maven. If we want to 
 test particular namespaces, we must do so from a REPL.
 Some quick way to launch unit tests for a namespace is desirable. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: STORM-446: Allow superusers to impersonate oth...

2015-02-27 Thread Parth-Brahmbhatt
Github user Parth-Brahmbhatt commented on a diff in the pull request:

https://github.com/apache/storm/pull/448#discussion_r25526790
  
--- Diff: 
storm-core/src/jvm/backtype/storm/security/auth/authorizer/SimpleACLAuthorizer.java
 ---
@@ -50,6 +50,7 @@
 protected SetString _supervisors;
 protected IPrincipalToLocal _ptol;
 protected IGroupMappingServiceProvider _groupMappingProvider;
+protected ImpersonationAuthorizer _impersonationAuthorizer;
--- End diff --

Ok, I am going to add this authorization as part of nimbus 
check-authorization! method. I will also add nimbus.impersonation.authorizer 
config to allow for overriding behavior. I will also add the logging changes 
there. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (STORM-446) secure Impersonation in storm

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340606#comment-14340606
 ] 

ASF GitHub Bot commented on STORM-446:
--

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76450968
  
Yes we would need something like that for this to work.  I'm not sure I 
want all administrators to have that power though.


 secure Impersonation in storm
 -

 Key: STORM-446
 URL: https://issues.apache.org/jira/browse/STORM-446
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Sriharsha Chintalapani
Assignee: Parth Brahmbhatt
  Labels: Security

 Storm security adds features of authenticating with kerberos and than uses 
 that principal and TGT as way to authorize user operations, topology 
 operation. Currently Storm UI user needs to be part of nimbus.admins to get 
 details on user submitted topologies. Ideally storm ui needs to take 
 authenticated user  principal to submit requests to nimbus which will than 
 authorize the user rather than storm UI user. This feature will also benefit 
 superusers to impersonate other users to submit topologies in a secured way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (STORM-446) secure Impersonation in storm

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340616#comment-14340616
 ] 

ASF GitHub Bot commented on STORM-446:
--

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76451646
  
@revans2 I agree. I can allow users to specify regex for matching but that 
seems like an overkill. For now   I am just going to make * as special 
character for both groups and hosts. 


 secure Impersonation in storm
 -

 Key: STORM-446
 URL: https://issues.apache.org/jira/browse/STORM-446
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Sriharsha Chintalapani
Assignee: Parth Brahmbhatt
  Labels: Security

 Storm security adds features of authenticating with kerberos and than uses 
 that principal and TGT as way to authorize user operations, topology 
 operation. Currently Storm UI user needs to be part of nimbus.admins to get 
 details on user submitted topologies. Ideally storm ui needs to take 
 authenticated user  principal to submit requests to nimbus which will than 
 authorize the user rather than storm UI user. This feature will also benefit 
 superusers to impersonate other users to submit topologies in a secured way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: STORM-446: Allow superusers to impersonate oth...

2015-02-27 Thread revans2
Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/448#issuecomment-76450968
  
Yes we would need something like that for this to work.  I'm not sure I 
want all administrators to have that power though.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (STORM-696) Single Namespace Test Launching

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340610#comment-14340610
 ] 

ASF GitHub Bot commented on STORM-696:
--

GitHub user knusbaum opened a pull request:

https://github.com/apache/storm/pull/450

[STORM-696] Single Namespace Test Launching

Adding test-ns.py

test-ns.py accepts a single argument, the name of a test namespace to run. 
Run it from the storm project root.
```
~/storm $ dev-tools/test-ns.py backtype.storm.config-test

...

Ran 11 tests containing 52 assertions.
0 failures, 0 errors.
{:type :summary, :fail 0, :error 0, :pass 52, :test 11}
user= Bye for now!
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knusbaum/incubator-storm STORM-696

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/450.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #450


commit b7bfdde8714c39e7556ac34915817dd2a8879356
Author: Kyle Nusbaum knusb...@yahoo-inc.com
Date:   2015-02-27T18:51:04Z

Adding test-ns.py




 Single Namespace Test Launching
 ---

 Key: STORM-696
 URL: https://issues.apache.org/jira/browse/STORM-696
 Project: Apache Storm
  Issue Type: Improvement
Reporter: Kyle Nusbaum
Assignee: Kyle Nusbaum
Priority: Minor

 As a dev I would like to be able to run unit tests in a particular namespace.
 Right now, we have to run the full test suite through maven. If we want to 
 test particular namespaces, we must do so from a REPL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: [STORM-696] Single Namespace Test Launching

2015-02-27 Thread knusbaum
GitHub user knusbaum opened a pull request:

https://github.com/apache/storm/pull/450

[STORM-696] Single Namespace Test Launching

Adding test-ns.py

test-ns.py accepts a single argument, the name of a test namespace to run. 
Run it from the storm project root.
```
~/storm $ dev-tools/test-ns.py backtype.storm.config-test

...

Ran 11 tests containing 52 assertions.
0 failures, 0 errors.
{:type :summary, :fail 0, :error 0, :pass 52, :test 11}
user= Bye for now!
```

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knusbaum/incubator-storm STORM-696

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/450.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #450


commit b7bfdde8714c39e7556ac34915817dd2a8879356
Author: Kyle Nusbaum knusb...@yahoo-inc.com
Date:   2015-02-27T18:51:04Z

Adding test-ns.py




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (STORM-695) storm CLI tool reports zero exit code on error scenario, take 2

2015-02-27 Thread Michael Noll (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14340333#comment-14340333
 ] 

Michael Noll commented on STORM-695:


To fix the storm CLI script:  Essentially we must updated the 
{{exec_storm_class()}} function to understand and pass exit codes of the 
commands it ran back to the caller.

Idea:

{code}
$ diff -u storm storm.fixed
--- storm   2015-02-24 13:59:03.0 +
+++ storm.fixed 2015-02-27 15:47:38.373536529 +
@@ -179,11 +179,13 @@
 -cp, get_classpath(extrajars),
 ] + jvmopts + [klass] + list(args)
 print(Running:  +  .join(all_args))
+return_code = 1 # default return code indicates failure
 if fork:
-os.spawnvp(os.P_WAIT, JAVA_CMD, all_args)
+return_code = os.spawnvp(os.P_WAIT, JAVA_CMD, all_args)
 else:
 # handling whitespaces in JAVA_CMD
-sub.call(all_args)
+return_code = sub.call(all_args)
+return return_code

 def jar(jarfile, klass, *args):
 Syntax: [storm jar topology-jar-path class ...]
@@ -490,7 +492,7 @@
 def unknown_command(*args):
 print(Unknown command: [storm %s] % ' '.join(sys.argv[1:]))
 print_usage()
-sys.exit(254)
+return 254

 COMMANDS = {jar: jar, kill: kill, shell: shell, nimbus: nimbus, ui: 
ui, logviewer: logviewer,
 drpc: drpc, supervisor: supervisor, localconfvalue: 
print_localconfvalue,
@@ -532,7 +534,8 @@
 parse_config(config_list)
 COMMAND = args[0]
 ARGS = args[1:]
-(COMMANDS.get(COMMAND, unknown_command))(*ARGS)
+return_code = (COMMANDS.get(COMMAND, unknown_command))(*ARGS)
+sys.exit(return_code)

 if __name__ == __main__:
 main()
{code}

As I said above this diff/patch alone is not sufficient because the Storm 
commands themselves (e.g. kill_topology.clj) are not returning proper 
success/failure information.

 storm CLI tool reports zero exit code on error scenario, take 2
 ---

 Key: STORM-695
 URL: https://issues.apache.org/jira/browse/STORM-695
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.9.3
Reporter: Michael Noll
Priority: Minor

 Commands such as storm kill non-existing-topology will return an exit code 
 of zero, indicating success when in fact the command failed.
 h3. How to reproduce
 Here is but one example where the {{storm}} CLI tool violates shell best 
 practices:
 {code}
 # Let's kill a topology that is in fact not running in the cluster.
 $ storm kill i-do-not-exist-topo
 snip
 Exception in thread main NotAliveException(msg:i-do-not-exist-topo is not 
 alive)
 # Print the exit code of last command.
 $ echo $?
 0  #  but since the kill command failed this should be non-zero!
 {code}
 Another example is the storm jar command.  If you attempt to submit a 
 topology that has the same name as an existing, running topology, the storm 
 jar command will not submit the topology -- instead it will print an 
 exception (think: the topology FooName is already running), which is ok, 
 but it will then exit with a return code of zero, which indicates success 
 (which is wrong).
 h3. Impact
 This bug prevents automated deployment tools such as Ansible or Puppet as 
 well as ad-hoc CLI scripting (think: fire-fighting Ops teams) to work 
 properly because Storm violates shell conventions by not returning non-zero 
 exit codes in case of failures.
 h3. How to fix
 From what I understand the solution is two-fold:
 # The various Storm commands that are being called by the {{storm}} script 
 must return proper exit codes.
 # The {{storm}} script must store these exit codes and return itself with the 
 respective exit code of the Storm command it actually ran.
 For example, here's the current code that implements the storm kill command:
 {code}
 # In: bin/storm
 def kill(*args):
 Syntax: [storm kill topology-name [-w wait-time-secs]]
 Kills the topology with the name topology-name. Storm will 
 first deactivate the topology's spouts for the duration of 
 the topology's message timeout to allow all messages currently 
 being processed to finish processing. Storm will then shutdown 
 the workers and clean up their state. You can override the length 
 of time Storm waits between deactivation and shutdown with the -w flag.
 
 exec_storm_class(
 backtype.storm.command.kill_topology, 
 args=args, 
 jvmtype=-client, 
 extrajars=[USER_CONF_DIR, STORM_BIN_DIR])
 {code}
 which in turn calls the following code in {{kill_topology.clj}}:
 {code}
 ;; In: backtype.storm.command.kill-topology
 (ns backtype.storm.command.kill-topology
   (:use [clojure.tools.cli :only [cli]])
   (:use [backtype.storm thrift config log])
   (:import [backtype.storm.generated KillOptions])
   (:gen-class))
 (defn 

[jira] [Created] (STORM-695) storm CLI tool reports zero exit code on error scenario, take 2

2015-02-27 Thread Michael Noll (JIRA)
Michael Noll created STORM-695:
--

 Summary: storm CLI tool reports zero exit code on error scenario, 
take 2
 Key: STORM-695
 URL: https://issues.apache.org/jira/browse/STORM-695
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.9.3
Reporter: Michael Noll
Priority: Minor


Commands such as storm kill non-existing-topology will return an exit code of 
zero, indicating success when in fact the command failed.

h3. How to reproduce

Here is but one example where the {{storm}} CLI tool violates shell best 
practices:

{code}
# Let's kill a topology that is in fact not running in the cluster.
$ storm kill i-do-not-exist-topo
snip
Exception in thread main NotAliveException(msg:i-do-not-exist-topo is not 
alive)

# Print the exit code of last command.
$ echo $?
0  #  but since the kill command failed this should be non-zero!
{code}

Another example is the storm jar command.  If you attempt to submit a 
topology that has the same name as an existing, running topology, the storm 
jar command will not submit the topology -- instead it will print an exception 
(think: the topology FooName is already running), which is ok, but it will 
then exit with a return code of zero, which indicates success (which is wrong).

h3. Impact

This bug prevents automated deployment tools such as Ansible or Puppet as well 
as ad-hoc CLI scripting (think: fire-fighting Ops teams) to work properly 
because Storm violates shell conventions by not returning non-zero exit codes 
in case of failures.

h3. How to fix

From what I understand the solution is two-fold:

# The various Storm commands that are being called by the {{storm}} script must 
return proper exit codes.
# The {{storm}} script must store these exit codes and return itself with the 
respective exit code of the Storm command it actually ran.

For example, here's the current code that implements the storm kill command:

{code}
# In: bin/storm
def kill(*args):
Syntax: [storm kill topology-name [-w wait-time-secs]]

Kills the topology with the name topology-name. Storm will 
first deactivate the topology's spouts for the duration of 
the topology's message timeout to allow all messages currently 
being processed to finish processing. Storm will then shutdown 
the workers and clean up their state. You can override the length 
of time Storm waits between deactivation and shutdown with the -w flag.

exec_storm_class(
backtype.storm.command.kill_topology, 
args=args, 
jvmtype=-client, 
extrajars=[USER_CONF_DIR, STORM_BIN_DIR])
{code}

which in turn calls the following code in {{kill_topology.clj}}:

{code}
;; In: backtype.storm.command.kill-topology
(ns backtype.storm.command.kill-topology
  (:use [clojure.tools.cli :only [cli]])
  (:use [backtype.storm thrift config log])
  (:import [backtype.storm.generated KillOptions])
  (:gen-class))

(defn -main [ args]
  (let [[{wait :wait} [name] _] (cli args [-w --wait :default nil :parse-fn 
#(Integer/parseInt %)])
opts (KillOptions.)]
(if wait (.set_wait_secs opts wait))
(with-configured-nimbus-connection nimbus
  (.killTopologyWithOpts nimbus name opts)
  (log-message Killed topology:  name)
  )))
{code}

which in turn calls the following code in {{nimbus.clj}}:

{code}
;; In: backtype.storm.daemon.nimbus
  (^void killTopologyWithOpts [this ^String storm-name ^KillOptions options]
(check-storm-active! nimbus storm-name true)
(let [topology-conf (try-read-storm-conf-from-name conf storm-name 
nimbus)]
  (check-authorization! nimbus storm-name topology-conf killTopology))
(let [wait-amt (if (.is_set_wait_secs options)
 (.get_wait_secs options) 
 )]
  (transition-name! nimbus storm-name [:kill wait-amt] true)
  ))
{code}

As you can see the current implementation does not pass success/failure 
information back to the caller.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[RESULT] [VOTE] Adopt Apache Storm Project Bylaws

2015-02-27 Thread P. Taylor Goetz
This void is now closed and passes with 8 binding +1 votes, no +0 votes, and no 
-1 votes.

Vote results (* indicates a binding vote):

+1:
Bobby Evans*
Andy Feng*
Nathan Marz*
Melan Nimesh
Sriharsha Chintalapani*
Derek Dagit*
P. Taylor Goetz*
Parth Brahmbhatt
Kyle Nusbaum*
Kishor Patil*

+0:

-1:

Thanks to all who voted. I will commit the bylaws and add them to the project 
website.

- Taylor

On Feb 18, 2015, at 4:43 PM, P. Taylor Goetz ptgo...@apache.org wrote:

 As a follow-up to the previous discussion regarding adopting project bylaws, 
 I’d like to start an official VOTE to formally adopt the bylaws as listed 
 below.
 
 Please vote on adopting the proposed bylaws.
 
 [+1] Adopt the bylaws as listed
 [+0] No opinion
 [-1] Do not adopt the bylaws because…
 
 This vote will be 2/3 Majority as described below, and open for 6 days.
 
 -Taylor
 
 -
 
 # Apache Storm Project Bylaws
 
 
 ## Roles and Responsibilities
 
 Apache projects define a set of roles with associated rights and 
 responsibilities. These roles govern what tasks an individual may perform 
 within the project. The roles are defined in the following sections:
 
 ### Users:
 
 The most important participants in the project are people who use our 
 software. The majority of our developers start out as users and guide their 
 development efforts from the user's perspective.
 
 Users contribute to the Apache projects by providing feedback to developers 
 in the form of bug reports and feature suggestions. As well, users 
 participate in the Apache community by helping other users on mailing lists 
 and user support forums.
 
 ### Contributors:
 
 Contributors are all of the volunteers who are contributing time, code, 
 documentation, or resources to the Storm Project. A contributor that makes 
 sustained, welcome contributions to the project may be invited to become a 
 Committer, though the exact timing of such invitations depends on many 
 factors.
 
 ### Committers:
 
 The project's Committers are responsible for the project's technical 
 management. Committers have access to all project source repositories. 
 Committers may cast binding votes on any technical discussion regarding storm.
 
 Committer access is by invitation only and must be approved by lazy consensus 
 of the active PMC members. A Committer is considered emeritus by their own 
 declaration or by not contributing in any form to the project for over six 
 months. An emeritus Committer may request reinstatement of commit access from 
 the PMC. Such reinstatement is subject to lazy consensus approval of active 
 PMC members.
 
 All Apache Committers are required to have a signed Contributor License 
 Agreement (CLA) on file with the Apache Software Foundation. There is a 
 [Committers' FAQ](https://www.apache.org/dev/committers.html) which provides 
 more details on the requirements for Committers.
 
 A Committer who makes a sustained contribution to the project may be invited 
 to become a member of the PMC. The form of contribution is not limited to 
 code. It can also include code review, helping out users on the mailing 
 lists, documentation, testing, etc.
 
 ### Project Management Committee(PMC):
 
 The PMC is responsible to the board and the ASF for the management and 
 oversight of the Apache Storm codebase. The responsibilities of the PMC 
 include:
 
  * Deciding what is distributed as products of the Apache Storm project. In 
 particular all releases must be approved by the PMC.
  * Maintaining the project's shared resources, including the codebase 
 repository, mailing lists, websites.
  * Speaking on behalf of the project.
  * Resolving license disputes regarding products of the project.
  * Nominating new PMC members and Committers.
  * Maintaining these bylaws and other guidelines of the project.
 
 Membership of the PMC is by invitation only and must be approved by a 
 consensus approval of active PMC members. A PMC member is considered 
 emeritus by their own declaration or by not contributing in any form to the 
 project for over six months. An emeritus member may request reinstatement to 
 the PMC. Such reinstatement is subject to consensus approval of the active 
 PMC members.
 
 The chair of the PMC is appointed by the ASF board. The chair is an office 
 holder of the Apache Software Foundation (Vice President, Apache Storm) and 
 has primary responsibility to the board for the management of the projects 
 within the scope of the Storm PMC. The chair reports to the board quarterly 
 on developments within the Storm project.
 
 The chair of the PMC is rotated annually. When the chair is rotated or if the 
 current chair of the PMC resigns, the PMC votes to recommend a new chair 
 using Single Transferable Vote (STV) voting. See 
 http://wiki.apache.org/general/BoardVoting for specifics. The decision must 
 be ratified by the Apache board.
 
 ## Voting
 
 Decisions regarding the project are made by votes on the primary project 

[GitHub] storm pull request: update bylaws for adoption discussion

2015-02-27 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/419


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] storm pull request: Storm-166: Nimbus HA design doc and implementa...

2015-02-27 Thread ptgoetz
Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/354#issuecomment-76474553
  
I'm +1 for merging this, however this feature is targeted for a post-0.10.0 
(AKA security release).

I'd like to propose that this be merged to a feature branch (e.g. 
0.11.x-branch) to allow for further testing and development. That should 
enable others to more easily build and test with these changes. It would also 
relieve @Parth-Brahmbhatt from constantly having to upmerge this pull request.

Granted that would likely shift some of that burden to committers such that 
patches to master will also have to be committed/merged to that branch.

I'd be interested to hear others' thoughts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Comment Edited] (STORM-605) Attempting to call unbound fn during bolt prepare

2015-02-27 Thread Li Chaomo (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339865#comment-14339865
 ] 

Li Chaomo edited comment on STORM-605 at 2/27/15 8:15 AM:
--

[~sriharsha] 

hi, i found something interesting.
when i merge all my clj sources to one clj file , it works fine.
Is this a classpath problem? But it is under the same classpath whether it 
works fine  or failed

i print my classpath when storm load the prepare method

2015-02-26 11:00:14 b.s.c.ClojureBolt [INFO] 
Storm开始加载bolt闭包com.pajk.risk.storm.clj.engine_count-bolt__ 
classpath=/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-devel-1.3
.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/slf4j-api-1.7.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-lang-2.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/
lib/storm-core-0.10.0-SNAPSHOT.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/math.numeric-tower-0.0.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-codec-1.6.jar:/ho
me/admin/apache-storm-0.10.0-SNAPSHOT/lib/crypto-equality-1.0.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/reflectasm-1.07-shaded.jar:/home/admin/apache-storm-0.10.0-SNAPSHO
T/lib/ns-tracker-0.2.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/curator-client-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-exec-1.1.jar:/home/admin/apac
he-storm-0.10.0-SNAPSHOT/lib/tools.namespace-0.2.4.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/asm-4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/chill-java-0.3.5.jar:/
home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-core-1.1.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clj-time-0.8.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-s
ervlet-1.3.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/httpclient-4.3.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jline-2.11.jar:/home/admin/apache-storm-0.10.0-SNAP
SHOT/lib/hadoop-auth-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-io-2.4.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.logging-0.2.3.jar:/home/admin/apa
che-storm-0.10.0-SNAPSHOT/lib/clout-1.0.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/compojure-1.1.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-security-7.6.13.v
20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/disruptor-2.10.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/servlet-api-2.5.jar:/home/admin/apache-storm-0.10.0-SNA
PSHOT/lib/jetty-servlets-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/objenesis-1.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-anti-forgery-1.0.0.j
ar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-http-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/javax.servlet-2.5.0.v201103041518.jar:/home/admin/apac
he-storm-0.10.0-SNAPSHOT/lib/logback-core-1.0.13.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clojure-1.6.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/carbonite-1.4.0.ja
r:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-continuation-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/logback-classic-1.0.13.jar:/home/admin/apache-s
torm-0.10.0-SNAPSHOT/lib/curator-framework-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/snakeyaml-1.11.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/java.classpath-0.
2.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-jetty-adapter-1.3.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-servlet-7.6.13.v20130916.jar:/home/admin/apach
e-storm-0.10.0-SNAPSHOT/lib/json-simple-1.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/minlog-1.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.cli-0.2.4.jar:/home/
admin/apache-storm-0.10.0-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-logging-1.1.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib
/crypto-random-1.2.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/kryo-2.21.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/httpcore-4.3.2.jar:/home/admin/apache-storm-0.10.0
-SNAPSHOT/lib/joda-time-2.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-io-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home
/admin/apache-storm-0.10.0-SNAPSHOT/lib/log4j-over-slf4j-1.6.6.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-client-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-S
NAPSHOT/lib/jetty-util-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-server-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jgrapht-cor
e-0.9.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clj-stacktrace-0.2.7.jar:/home/admin/apache-stor

[jira] [Commented] (STORM-605) Attempting to call unbound fn during bolt prepare

2015-02-27 Thread Li Chaomo (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339865#comment-14339865
 ] 

Li Chaomo commented on STORM-605:
-

[~sriharsha] 

hi, i found something interesting.
when i merge all my clj source to one clj file , it works fine.
Is this a classpath problem? But it is under the same classpath whether it 
works fine  or failed

i print my classpath when storm load the prepare method

2015-02-26 11:00:14 b.s.c.ClojureBolt [INFO] 
Storm开始加载bolt闭包com.pajk.risk.storm.clj.engine_count-bolt__ 
classpath=/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-devel-1.3
.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/slf4j-api-1.7.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-lang-2.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/
lib/storm-core-0.10.0-SNAPSHOT.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/math.numeric-tower-0.0.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-codec-1.6.jar:/ho
me/admin/apache-storm-0.10.0-SNAPSHOT/lib/crypto-equality-1.0.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/reflectasm-1.07-shaded.jar:/home/admin/apache-storm-0.10.0-SNAPSHO
T/lib/ns-tracker-0.2.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/curator-client-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-exec-1.1.jar:/home/admin/apac
he-storm-0.10.0-SNAPSHOT/lib/tools.namespace-0.2.4.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/asm-4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/chill-java-0.3.5.jar:/
home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-core-1.1.5.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clj-time-0.8.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-s
ervlet-1.3.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/httpclient-4.3.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jline-2.11.jar:/home/admin/apache-storm-0.10.0-SNAP
SHOT/lib/hadoop-auth-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-io-2.4.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.logging-0.2.3.jar:/home/admin/apa
che-storm-0.10.0-SNAPSHOT/lib/clout-1.0.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/compojure-1.1.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-security-7.6.13.v
20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/disruptor-2.10.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/servlet-api-2.5.jar:/home/admin/apache-storm-0.10.0-SNA
PSHOT/lib/jetty-servlets-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/objenesis-1.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-anti-forgery-1.0.0.j
ar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-http-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/javax.servlet-2.5.0.v201103041518.jar:/home/admin/apac
he-storm-0.10.0-SNAPSHOT/lib/logback-core-1.0.13.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clojure-1.6.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/carbonite-1.4.0.ja
r:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-continuation-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/logback-classic-1.0.13.jar:/home/admin/apache-s
torm-0.10.0-SNAPSHOT/lib/curator-framework-2.4.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/snakeyaml-1.11.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/java.classpath-0.
2.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/ring-jetty-adapter-1.3.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-servlet-7.6.13.v20130916.jar:/home/admin/apach
e-storm-0.10.0-SNAPSHOT/lib/json-simple-1.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/minlog-1.2.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.cli-0.2.4.jar:/home/
admin/apache-storm-0.10.0-SNAPSHOT/lib/core.incubator-0.1.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-logging-1.1.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib
/crypto-random-1.2.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/kryo-2.21.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/httpcore-4.3.2.jar:/home/admin/apache-storm-0.10.0
-SNAPSHOT/lib/joda-time-2.3.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-io-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/tools.macro-0.1.0.jar:/home
/admin/apache-storm-0.10.0-SNAPSHOT/lib/log4j-over-slf4j-1.6.6.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-client-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-S
NAPSHOT/lib/jetty-util-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jetty-server-7.6.13.v20130916.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/jgrapht-cor
e-0.9.0.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/commons-fileupload-1.2.1.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/lib/clj-stacktrace-0.2.7.jar:/home/admin/apache-stor
m-0.10.0-SNAPSHOT/lib/hiccup-0.3.6.jar:/home/admin/apache-storm-0.10.0-SNAPSHOT/conf:/home/admin/storm/localdir/supervisor/stormdist/risk-engine-4-1424919605/stormjar.jar

[GitHub] storm pull request: [STORM-671] Measure tuple serialization/deseri...

2015-02-27 Thread Lewuathe
Github user Lewuathe commented on the pull request:

https://github.com/apache/storm/pull/442#issuecomment-76501848
  
@revans2 Thank you so much for reviewing. I updated to solve above 
exception and include builtin metrics. Please check it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Commented] (STORM-671) Measure tuple serialization/deserialization latency.

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14341150#comment-14341150
 ] 

ASF GitHub Bot commented on STORM-671:
--

Github user Lewuathe commented on the pull request:

https://github.com/apache/storm/pull/442#issuecomment-76501848
  
@revans2 Thank you so much for reviewing. I updated to solve above 
exception and include builtin metrics. Please check it.


 Measure tuple serialization/deserialization latency.
 

 Key: STORM-671
 URL: https://issues.apache.org/jira/browse/STORM-671
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Robert Joseph Evans
Assignee: Kai Sasaki

 Some times the serialization/deserialization cost can be very high, and it is 
 not currently measured anywhere in storm.  We should measure it, at least in 
 a similar way to how we do execute and process latency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (STORM-694) Compiling storm 0.10.0 results in java.lang.ClassNotFoundException: backtype.storm.daemon.common.SupervisorInfo

2015-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/STORM-694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14339951#comment-14339951
 ] 

ASF GitHub Bot commented on STORM-694:
--

GitHub user duke-bartholomew opened a pull request:

https://github.com/apache/storm/pull/449

STORM-694

Exception in thread main java.lang.ClassNotFoundException: 
backtype.storm.daemon.common.SupervisorInfo, compiling:(supervisor.clj:16:1)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/duke-bartholomew/storm master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/449.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #449


commit c2b23cfb2fd167aa9d53e16cfa49e7665324d46f
Author: Bartholomew duke.bartholo...@gmail.com
Date:   2015-02-27T09:45:49Z

add explicit :require

commit 905c14e10b878194b12646240284cbf014d8eadd
Author: Duke Bartholomew duke.bartholo...@gmail.com
Date:   2015-02-27T09:48:45Z

Merge pull request #1 from duke-bartholomew/STORM-694

add explicit :require




 Compiling storm 0.10.0 results in java.lang.ClassNotFoundException: 
 backtype.storm.daemon.common.SupervisorInfo
 ---

 Key: STORM-694
 URL: https://issues.apache.org/jira/browse/STORM-694
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Duke Bartholomew
Priority: Minor

 When compiling storm I get an exception in supervisor.clj
 {noformat}
 Exception in thread main java.lang.ClassNotFoundException: 
 backtype.storm.daemon.common.SupervisorInfo, compiling:(supervisor.clj:16:1)
   at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3558)
   at clojure.lang.Compiler.compile1(Compiler.java:7226)
   at clojure.lang.Compiler.compile1(Compiler.java:7216)
   at clojure.lang.Compiler.compile(Compiler.java:7292)
   at clojure.lang.RT.compile(RT.java:398)
   at clojure.lang.RT.load(RT.java:438)
   at clojure.lang.RT.load(RT.java:411)
   at clojure.core$load$fn__5066.invoke(core.clj:5641)
   at clojure.core$load.doInvoke(core.clj:5640)
   at clojure.lang.RestFn.invoke(RestFn.java:408)
   at clojure.core$load_one.invoke(core.clj:5446)
   at clojure.core$compile$fn__5071.invoke(core.clj:5652)
   at clojure.core$compile.invoke(core.clj:5651)
   at clojure.lang.Var.invoke(Var.java:379)
   at clojure.lang.Compile.main(Compile.java:81)
 {noformat}
 After adding an explicit {{:require}} in {{supervisor.clj}} the issue seems 
 to be  resolved :
 {noformat}
 diff --git a/storm-core/src/clj/backtype/storm/daemon/supervisor.clj 
 b/storm-core/src/clj/backtype/storm/daemon/supervisor.clj
 index b9f9632..5fc5b7c 100644
 --- a/storm-core/src/clj/backtype/storm/daemon/supervisor.clj
 +++ b/storm-core/src/clj/backtype/storm/daemon/supervisor.clj
 @@ -15,6 +15,7 @@
  ;; limitations under the License.
  (ns backtype.storm.daemon.supervisor
(:import [java.io OutputStreamWriter BufferedWriter IOException])
 +  (:require [backtype.storm.daemon.common :as common])
(:import [backtype.storm.scheduler ISupervisor]
 [backtype.storm.utils LocalState Time Utils]
 [backtype.storm.daemon Shutdownable]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] storm pull request: STORM-694

2015-02-27 Thread duke-bartholomew
GitHub user duke-bartholomew opened a pull request:

https://github.com/apache/storm/pull/449

STORM-694

Exception in thread main java.lang.ClassNotFoundException: 
backtype.storm.daemon.common.SupervisorInfo, compiling:(supervisor.clj:16:1)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/duke-bartholomew/storm master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/449.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #449


commit c2b23cfb2fd167aa9d53e16cfa49e7665324d46f
Author: Bartholomew duke.bartholo...@gmail.com
Date:   2015-02-27T09:45:49Z

add explicit :require

commit 905c14e10b878194b12646240284cbf014d8eadd
Author: Duke Bartholomew duke.bartholo...@gmail.com
Date:   2015-02-27T09:48:45Z

Merge pull request #1 from duke-bartholomew/STORM-694

add explicit :require




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---