[jira] [Closed] (STORM-1080) Compiler SQL literals to LLVM constants

[Haohui Mai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haohui Mai closed STORM-1080.
-
Resolution: Fixed

Thanks [~sriharsha] for the reviews and commits.

> Compiler SQL literals to LLVM constants
> ---
>
> Key: STORM-1080
> URL: https://issues.apache.org/jira/browse/STORM-1080
> Project: Apache Storm
>  Issue Type: New Feature
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> This jira tracks the effort of creating a expression compiler to compile SQL 
> literals down to constants that are represented in LLVM IR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (STORM-1097) Compile IR to Java source code

[Haohui Mai (JIRA)]

Haohui Mai created STORM-1097:
-

 Summary: Compile IR to Java source code
 Key: STORM-1097
 URL: https://issues.apache.org/jira/browse/STORM-1097
 Project: Apache Storm
  Issue Type: Bug
Reporter: Haohui Mai
Assignee: Haohui Mai


The compiler needs to transform the LLVM IR to Java code so that the query can 
be plugged into Storm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1097) Compile IR to Java source code

[Haohui Mai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haohui Mai updated STORM-1097:
--
Issue Type: New Feature  (was: Bug)

> Compile IR to Java source code
> --
>
> Key: STORM-1097
> URL: https://issues.apache.org/jira/browse/STORM-1097
> Project: Apache Storm
>  Issue Type: New Feature
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> The compiler needs to transform the LLVM IR to Java code so that the query 
> can be plugged into Storm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1085. Compile comparison, arithmetic, an...

[haohui]

GitHub user haohui opened a pull request:

https://github.com/apache/storm/pull/789

STORM-1085. Compile comparison, arithmetic, and field reference expressions.

This PR augments the compilers to compile the comparison, arithmetic and 
field references in SQL expressions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/haohui/storm STORM-1085

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/789.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #789


commit 739749196aeacb448fc12f017cbfa9dc7c61d41f
Author: Haohui Mai 
Date:   2015-10-01T21:12:29Z

STORM-1085. Compile comparison, arithmetic, and field reference expressions.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1085) Compile comparison, arithmetic, and field reference expressions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948145#comment-14948145
 ] 

ASF GitHub Bot commented on STORM-1085:
---

GitHub user haohui opened a pull request:

https://github.com/apache/storm/pull/789

STORM-1085. Compile comparison, arithmetic, and field reference expressions.

This PR augments the compilers to compile the comparison, arithmetic and 
field references in SQL expressions.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/haohui/storm STORM-1085

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/789.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #789


commit 739749196aeacb448fc12f017cbfa9dc7c61d41f
Author: Haohui Mai 
Date:   2015-10-01T21:12:29Z

STORM-1085. Compile comparison, arithmetic, and field reference expressions.




> Compile comparison, arithmetic, and field reference expressions
> ---
>
> Key: STORM-1085
> URL: https://issues.apache.org/jira/browse/STORM-1085
> Project: Apache Storm
>  Issue Type: New Feature
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> This jira tracks the effort of implementing the comparison, arithmetic and 
> the field reference expressions in the expression compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1084] - Improve Storm config validation...

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/785#issuecomment-146558153
  
hmmm.  You are right, and there is no inheritance in annotations.  So we 
cannot get that to work.  Let me think about it a bit more.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1096: Fix some issues with impersonation...

[kishorvpatil]

Github user kishorvpatil commented on the pull request:

https://github.com/apache/storm/pull/787#issuecomment-146558088
  
+1. Good catch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1096: Fix some issues with impersonation...

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/787#issuecomment-146548927
  
The test failure was the intermittent kafka issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1096) UI tries to impersonate wrong user when getting topology conf for authorization, impersonation is allowed by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948711#comment-14948711
 ] 

ASF GitHub Bot commented on STORM-1096:
---

Github user kishorvpatil commented on the pull request:

https://github.com/apache/storm/pull/787#issuecomment-146558088
  
+1. Good catch.


> UI tries to impersonate wrong user when getting topology conf for 
> authorization, impersonation is allowed by default
> 
>
> Key: STORM-1096
> URL: https://issues.apache.org/jira/browse/STORM-1096
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.10.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>Priority: Blocker
>
> We have started using 0.10.0 under load and found a few issues around the UI 
> and impersonation.
> The UI when trying to connect to nimbus will impersonate other users.  
> Nimbus, by default allows impersonation and just outputs a warning message 
> that it is allowed.  We really should default to not allowing impersonation.  
> having the authorizer configured by default does not hurt when running 
> insecure because impersonation is not possible, but when security is enabled 
> if someone forgets to set this config we are now insecure by default.
> If you do set all of that up correctly the UI now can impersonate the wrong 
> user when connecting to nimbus.
> The UI decides which user to impersonate by pulling it from the request 
> context.  The requestContext is populated from the HttpRequest when 
> assert-authorized-user is called.  assert-authorized-user takes a 
> topology-conf as a parameter.  The only way to get this topology conf is to 
> talk to nimbus, which will get the wrong user because the request context has 
> not been populated yet.
> This just because a huge pain for users who way too often will not be able to 
> see pages on the UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1096) UI tries to impersonate wrong user when getting topology conf for authorization, impersonation is allowed by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948714#comment-14948714
 ] 

ASF GitHub Bot commented on STORM-1096:
---

Github user kishorvpatil commented on the pull request:

https://github.com/apache/storm/pull/788#issuecomment-146558244
  
+1


> UI tries to impersonate wrong user when getting topology conf for 
> authorization, impersonation is allowed by default
> 
>
> Key: STORM-1096
> URL: https://issues.apache.org/jira/browse/STORM-1096
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.10.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>Priority: Blocker
>
> We have started using 0.10.0 under load and found a few issues around the UI 
> and impersonation.
> The UI when trying to connect to nimbus will impersonate other users.  
> Nimbus, by default allows impersonation and just outputs a warning message 
> that it is allowed.  We really should default to not allowing impersonation.  
> having the authorizer configured by default does not hurt when running 
> insecure because impersonation is not possible, but when security is enabled 
> if someone forgets to set this config we are now insecure by default.
> If you do set all of that up correctly the UI now can impersonate the wrong 
> user when connecting to nimbus.
> The UI decides which user to impersonate by pulling it from the request 
> context.  The requestContext is populated from the HttpRequest when 
> assert-authorized-user is called.  assert-authorized-user takes a 
> topology-conf as a parameter.  The only way to get this topology conf is to 
> talk to nimbus, which will get the wrong user because the request context has 
> not been populated yet.
> This just because a huge pain for users who way too often will not be able to 
> see pages on the UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1084) Improve Storm config validation process to use java annotations instead of *_SCHEMA format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948712#comment-14948712
 ] 

ASF GitHub Bot commented on STORM-1084:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/785#issuecomment-146558153
  
hmmm.  You are right, and there is no inheritance in annotations.  So we 
cannot get that to work.  Let me think about it a bit more.


> Improve Storm config validation process to use java annotations instead of 
> *_SCHEMA format
> --
>
> Key: STORM-1084
> URL: https://issues.apache.org/jira/browse/STORM-1084
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: Boyang Jerry Peng
>Assignee: Boyang Jerry Peng
>
> So currently we specify validators:
>  public static final String STORM_MESSAGING_NETTY_MIN_SLEEP_MS = 
> "storm.messaging.netty.min_wait_ms";
>  public static final Object STORM_MESSAGING_NETTY_MIN_SLEEP_MS_SCHEMA = 
> ConfigValidation.IntegerValidator;
> A better way to do this is using annotations.  Something like:
> @IntegerValidator
>  public static final String STORM_MESSAGING_NETTY_MIN_SLEEP_MS = 
> "storm.messaging.netty.min_wait_ms";
> Do this has many advantages. For one you can stack multiple annotations:
> @IntegerValidator
> @NotNull
>  public static final String STORM_MESSAGING_NETTY_MIN_SLEEP_MS = 
> "storm.messaging.netty.min_wait_ms";
> And we don't have to write another validator for strings that cannot be null
> And we can pass parameters into the annotations: 
> @PositiveIntegerValidator(notNull=true)
>   public static final String DRPC_REQUEST_TIMEOUT_SECS  = 
> "drpc.request.timeout.secs";
> instead of having to write another validator: 
> ConfigValidation.NotNullPosIntegerValidator for checking for not null



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1096) UI tries to impersonate wrong user when getting topology conf for authorization, impersonation is allowed by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948659#comment-14948659
 ] 

ASF GitHub Bot commented on STORM-1096:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/787#issuecomment-146548927
  
The test failure was the intermittent kafka issue.


> UI tries to impersonate wrong user when getting topology conf for 
> authorization, impersonation is allowed by default
> 
>
> Key: STORM-1096
> URL: https://issues.apache.org/jira/browse/STORM-1096
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.10.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>Priority: Blocker
>
> We have started using 0.10.0 under load and found a few issues around the UI 
> and impersonation.
> The UI when trying to connect to nimbus will impersonate other users.  
> Nimbus, by default allows impersonation and just outputs a warning message 
> that it is allowed.  We really should default to not allowing impersonation.  
> having the authorizer configured by default does not hurt when running 
> insecure because impersonation is not possible, but when security is enabled 
> if someone forgets to set this config we are now insecure by default.
> If you do set all of that up correctly the UI now can impersonate the wrong 
> user when connecting to nimbus.
> The UI decides which user to impersonate by pulling it from the request 
> context.  The requestContext is populated from the HttpRequest when 
> assert-authorized-user is called.  assert-authorized-user takes a 
> topology-conf as a parameter.  The only way to get this topology conf is to 
> talk to nimbus, which will get the wrong user because the request context has 
> not been populated yet.
> This just because a huge pain for users who way too often will not be able to 
> see pages on the UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1096) UI tries to impersonate wrong user when getting topology conf for authorization, impersonation is allowed by default

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948761#comment-14948761
 ] 

ASF GitHub Bot commented on STORM-1096:
---

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/787#issuecomment-146566393
  
+1


> UI tries to impersonate wrong user when getting topology conf for 
> authorization, impersonation is allowed by default
> 
>
> Key: STORM-1096
> URL: https://issues.apache.org/jira/browse/STORM-1096
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.10.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>Priority: Blocker
>
> We have started using 0.10.0 under load and found a few issues around the UI 
> and impersonation.
> The UI when trying to connect to nimbus will impersonate other users.  
> Nimbus, by default allows impersonation and just outputs a warning message 
> that it is allowed.  We really should default to not allowing impersonation.  
> having the authorizer configured by default does not hurt when running 
> insecure because impersonation is not possible, but when security is enabled 
> if someone forgets to set this config we are now insecure by default.
> If you do set all of that up correctly the UI now can impersonate the wrong 
> user when connecting to nimbus.
> The UI decides which user to impersonate by pulling it from the request 
> context.  The requestContext is populated from the HttpRequest when 
> assert-authorized-user is called.  assert-authorized-user takes a 
> topology-conf as a parameter.  The only way to get this topology conf is to 
> talk to nimbus, which will get the wrong user because the request context has 
> not been populated yet.
> This just because a huge pain for users who way too often will not be able to 
> see pages on the UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1096: Fix some issues with impersonation...

[d2r]

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/788#issuecomment-146565508
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1079) Batch Puts to HBase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948933#comment-14948933
 ] 

ASF GitHub Bot commented on STORM-1079:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/772#issuecomment-146600504
  
@revans2 @HeartSaVioR I added the default flushIntervalSecs like @dossett 
did in storm-hdfs. Let me know if thats ok or you still want flushIntervalSecs 
to 1.


> Batch Puts to HBase
> ---
>
> Key: STORM-1079
> URL: https://issues.apache.org/jira/browse/STORM-1079
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-hbase
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
> Fix For: 0.11.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1079) Batch Puts to HBase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948957#comment-14948957
 ] 

ASF GitHub Bot commented on STORM-1079:
---

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/772#issuecomment-146607651
  
I have one very minor nit about the logging.  I am +1 either way though. 


> Batch Puts to HBase
> ---
>
> Key: STORM-1079
> URL: https://issues.apache.org/jira/browse/STORM-1079
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-hbase
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
> Fix For: 0.11.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1079. Batch Puts to HBase.

[revans2]

Github user revans2 commented on the pull request:

https://github.com/apache/storm/pull/772#issuecomment-146607651
  
I have one very minor nit about the logging.  I am +1 either way though. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1079. Batch Puts to HBase.

[revans2]

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/772#discussion_r41536283
  
--- Diff: 
external/storm-hbase/src/main/java/org/apache/storm/hbase/bolt/HBaseBolt.java 
---
@@ -53,21 +61,69 @@ public HBaseBolt withConfigKey(String configKey) {
 return this;
 }
 
+public HBaseBolt withBatchSize(int batchSize) {
+this.batchSize = batchSize;
+return this;
+}
+
+public HBaseBolt withFlushIntervalSecs(int flushIntervalSecs) {
+this.flushIntervalSecs = flushIntervalSecs;
+return this;
+}
+
+@Override
+public Map getComponentConfiguration() {
+Map conf = super.getComponentConfiguration();
+if (conf == null) {
+conf = new Config();
+}
+
+if (conf.containsKey("topology.message.timeout.secs") && 
tickTupleInterval == 0) {
+Integer topologyTimeout = 
Integer.parseInt(conf.get("topology.message.timeout.secs").toString());
+flushIntervalSecs = (int)(Math.floor(topologyTimeout / 2));
+LOG.debug("Setting flush interval to [" + flushIntervalSecs + 
"] based on topology.message.timeout.secs");
--- End diff --

minor nit.  Can we use the "{}" for logging instead of using +.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1079. Batch Puts to HBase.

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/772#issuecomment-146600504
  
@revans2 @HeartSaVioR I added the default flushIntervalSecs like @dossett 
did in storm-hdfs. Let me know if thats ok or you still want flushIntervalSecs 
to 1.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1079) Batch Puts to HBase

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14948956#comment-14948956
 ] 

ASF GitHub Bot commented on STORM-1079:
---

Github user revans2 commented on a diff in the pull request:

https://github.com/apache/storm/pull/772#discussion_r41536283
  
--- Diff: 
external/storm-hbase/src/main/java/org/apache/storm/hbase/bolt/HBaseBolt.java 
---
@@ -53,21 +61,69 @@ public HBaseBolt withConfigKey(String configKey) {
 return this;
 }
 
+public HBaseBolt withBatchSize(int batchSize) {
+this.batchSize = batchSize;
+return this;
+}
+
+public HBaseBolt withFlushIntervalSecs(int flushIntervalSecs) {
+this.flushIntervalSecs = flushIntervalSecs;
+return this;
+}
+
+@Override
+public Map getComponentConfiguration() {
+Map conf = super.getComponentConfiguration();
+if (conf == null) {
+conf = new Config();
+}
+
+if (conf.containsKey("topology.message.timeout.secs") && 
tickTupleInterval == 0) {
+Integer topologyTimeout = 
Integer.parseInt(conf.get("topology.message.timeout.secs").toString());
+flushIntervalSecs = (int)(Math.floor(topologyTimeout / 2));
+LOG.debug("Setting flush interval to [" + flushIntervalSecs + 
"] based on topology.message.timeout.secs");
--- End diff --

minor nit.  Can we use the "{}" for logging instead of using +.


> Batch Puts to HBase
> ---
>
> Key: STORM-1079
> URL: https://issues.apache.org/jira/browse/STORM-1079
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-hbase
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
> Fix For: 0.11.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-517: Adding JAVA_SERVICE_NAME to bin/sto...

[d2r]

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/281#issuecomment-146654979
  
@solarce This appears to have been addressed as part of #562 .  We should 
probably close this pull request.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-517) Support for "-Dservice=" in bin/storm, via JAVA_SERVICE_NAME environment variable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949164#comment-14949164
 ] 

ASF GitHub Bot commented on STORM-517:
--

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/281#issuecomment-146654979
  
@solarce This appears to have been addressed as part of #562 .  We should 
probably close this pull request.


> Support for "-Dservice=" in bin/storm, via JAVA_SERVICE_NAME environment 
> variable
> -
>
> Key: STORM-517
> URL: https://issues.apache.org/jira/browse/STORM-517
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: Brandon Burton
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> *Reasoning:*
> Currently the way that _bin/storm_ starts the various storm processes 
> (nimbus, ui, supervisor) results in a process list entry like the following, 
> for each process:
> {code}
> ubuntu   21940  0.4  3.9 2294904 151384 ?  Ssl  Oct03   2:47 java -server 
> -Dstorm.options= -Dstorm.home=/home/ubuntu/apache-storm-0.9.2-incubating 
> -Djava.library.path=/usr/local/lib -Dstorm.conf.file= -cp 
> /home/ubuntu/apache-storm-0.9.2-incubating/lib/asm-4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-logging-1.1.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-devel-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/storm-core-0.9.2-incubating.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/log4j-over-slf4j-1.6.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.cli-0.2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-servlet-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-codec-1.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/logback-classic-1.0.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clj-time-0.4.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/httpclient-4.3.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/slf4j-api-1.6.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/httpcore-4.3.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/carbonite-1.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clout-1.0.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jetty-6.1.26.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clojure-1.5.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-io-2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-exec-1.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jgrapht-core-0.9.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/curator-client-2.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.macro-0.1.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/zookeeper-3.4.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jline-2.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/minlog-1.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/tools.logging-0.2.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/reflectasm-1.07-shaded.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/guava-13.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-core-1.1.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/logback-core-1.0.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/netty-3.6.3.Final.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/ring-jetty-adapter-0.3.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/core.incubator-0.1.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-lang-2.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/netty-3.2.2.Final.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/curator-framework-2.4.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/disruptor-2.10.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/servlet-api-2.5-20081211.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/clj-stacktrace-0.2.4.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/chill-java-0.3.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/compojure-1.1.3.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/kryo-2.21.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/jetty-util-6.1.26.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/servlet-api-2.5.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/json-simple-1.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/commons-fileupload-1.2.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/snakeyaml-1.11.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/math.numeric-tower-0.0.1.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/joda-time-2.0.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/hiccup-0.3.6.jar:/home/ubuntu/apache-storm-0.9.2-incubating/lib/objenesis-1.2.jar:/home/ubuntu/apache-storm-0.9.2-incubating/conf
>  -Xmx1024m -Dlogfile.name=nimbus.log 
> 

[GitHub] storm pull request: STORM-912: Support SSL on Logviewer

[knusbaum]

Github user knusbaum commented on the pull request:

https://github.com/apache/storm/pull/717#issuecomment-146656531
  
@HeartSaVioR Sorry, it took me a while to get back to this. 

The keyword is scheme, not schema. I've fixed it here, and I'll create a 
new pull request to master to fix it there, too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-912) Support SSL on Logviewer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949175#comment-14949175
 ] 

ASF GitHub Bot commented on STORM-912:
--

Github user knusbaum commented on the pull request:

https://github.com/apache/storm/pull/717#issuecomment-146656531
  
@HeartSaVioR Sorry, it took me a while to get back to this. 

The keyword is scheme, not schema. I've fixed it here, and I'll create a 
new pull request to master to fix it there, too.


> Support SSL on Logviewer
> 
>
> Key: STORM-912
> URL: https://issues.apache.org/jira/browse/STORM-912
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: Kyle Nusbaum
>Assignee: Kyle Nusbaum
>Priority: Minor
> Fix For: 0.11.0
>
>
> Support SSL on the logviewer like it is in the UI.
> Also detect what method we're using and make sure logviewer links in the UI 
> are pointing to the appropriate Logviewer endpoint.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-591] exclude logback.xml from jar

[d2r]

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/352#issuecomment-146660961
  
@Lewuathe ,
The file has been converted to log4j2.xml, so all we need to do get your 
branch upmerged.  Would you take a look?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-591) logback.xml in store-core

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949201#comment-14949201
 ] 

ASF GitHub Bot commented on STORM-591:
--

Github user d2r commented on the pull request:

https://github.com/apache/storm/pull/352#issuecomment-146660961
  
@Lewuathe ,
The file has been converted to log4j2.xml, so all we need to do get your 
branch upmerged.  Would you take a look?


> logback.xml in store-core
> -
>
> Key: STORM-591
> URL: https://issues.apache.org/jira/browse/STORM-591
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.3, 0.9.4, 0.9.5
>Reporter: Shaun Thompson
>Assignee: Kai Sasaki
>
> logback.xml is now being included in storm-core.jar.
> This causes issues for any application that might include storm-core in it's 
> classpath, having to override the config file location.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-893) Resource Aware Scheduling

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949438#comment-14949438
 ] 

ASF GitHub Bot commented on STORM-893:
--

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/746#issuecomment-146692455
  
Nice work! +1


> Resource Aware Scheduling
> -
>
> Key: STORM-893
> URL: https://issues.apache.org/jira/browse/STORM-893
> Project: Apache Storm
>  Issue Type: Umbrella
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Boyang Jerry Peng
> Attachments: resource_aware_scheduler_api.pdf
>
>
> At Yahoo we have been working on resource aware scheduling in storm, based 
> off of some work done in academia.  This rollup ticket is to track the 
> complete project.  With several sub tasks.  Some that are already done and 
> need to be pushed back, and others that we have not started on yet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1087: Avoid issues with transfer-queue b...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/782#issuecomment-146677901
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-166:Nimbus HA solution based on Zookeepe...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/61#issuecomment-146684243
  
@d2r Yes, I think this pull request can be closed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-166) Highly available Nimbus

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949387#comment-14949387
 ] 

ASF GitHub Bot commented on STORM-166:
--

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/61#issuecomment-146684243
  
@d2r Yes, I think this pull request can be closed.


> Highly available Nimbus
> ---
>
> Key: STORM-166
> URL: https://issues.apache.org/jira/browse/STORM-166
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>Assignee: Parth Brahmbhatt
>Priority: Minor
> Fix For: 0.11.0
>
>
> https://github.com/nathanmarz/storm/issues/360
> The goal of this feature is to be able to run multiple Nimbus servers so that 
> if one goes down another one will transparently take over. Here's what needs 
> to happen to implement this:
> 1. Everything currently stored on local disk on Nimbus needs to be stored in 
> a distributed and reliable fashion. A DFS is perfect for this. However, as we 
> do not want to make a DFS a mandatory requirement to run Storm, the storage 
> of these artifacts should be pluggable (default to local filesystem, but the 
> interface should support DFS). You would only be able to run multiple NImbus 
> if you use the right storage, and the storage interface chosen should have a 
> flag indicating whether it's suitable for HA mode or not. If you choose local 
> storage and try to run multiple Nimbus, one of the Nimbus's should fail to 
> launch.
> 2. Nimbus's should register themselves in Zookeeper. They should use a leader 
> election protocol to decide which one is currently responsible for launching 
> and monitoring topologies.
> 3. StormSubmitter should find the Nimbus to connect to via Zookeeper. In case 
> the leader changes during submission, it should use a retry protocol to try 
> reconnecting to the new leader and attempting submission again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-150: Replace jar distribution strategy w...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/71#issuecomment-146684414
  
@d2r Thanks for the reminder. Closing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-150) Replace jar distribution strategy with bittorent

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949390#comment-14949390
 ] 

ASF GitHub Bot commented on STORM-150:
--

Github user ptgoetz closed the pull request at:

https://github.com/apache/storm/pull/71


> Replace jar distribution strategy with bittorent
> 
>
> Key: STORM-150
> URL: https://issues.apache.org/jira/browse/STORM-150
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/435
> Consider using http://turn.github.com/ttorrent/
> --
> ptgoetz: I've been looking into implementing this, but have a design question 
> that boils down to this:
> Should the client (storm jar command) be the initial seeder, or should we 
> wait and let nimbus do the seeding?
> The benefit of doing the seeding client-side is that we would only have to 
> transfer a small .torrent through the nimbus thrift API. But I can imagine 
> situations where the network environment would prevent BitTorrent clients 
> from connecting back to the machine that's submitting the topology. This 
> would create an indefinitely "stalled submission" since none of the cluster 
> nodes would be able to connect to the seeder.
> The alternative would be to use the current technique of uploading the jar to 
> nimbus, and have nimbus generate and distribute the .torrent file, and 
> provide the initial seed. If the cluster is properly configured, we're pretty 
> much guaranteed connectivity between nimbus and supervisor nodes.
> I'm leaning toward the latter approach, but would be interested in others' 
> opinions.
> --
> nathanmarz: @ptgoetz I think Nimbus should do the seeding. That ensures that 
> when the client finishes submitting, it can disconnect/go away without having 
> to worry about making the topology unlaunchable.
> --
> jasonjckn: @nathanmarz How does this solve the nimbuses dependency on 
> reliable local disk state (as you talked about in person)?
> What happens when zookeeper is offline for 1 hour? All the workers will die, 
> and nimbus will be continually restarting. The onus is still on nimbus to 
> store topology jars on local disk, so that when the workers and supervisors 
> reboots it can seed all this again.
> You -can- solve the local disk persistence problem with replicated state to 
> the non-elected nimbuses, but that's orthogonal to a distribution strategy. 
> Yes there is some replication going on in bittorrent, but it's not really a 
> protocol that delivers reliable persistence of state.
> I think it's still a good feature if it gives us performant topology submit 
> times even with 500 workers, which take 3 minutes for us.
> Particularly with the worker heartbeat start-up timeout of 120s, you want to 
> be able to start 500 workers within 120s, or even 1500 workers within 120s, 
> the current distribution strategy is not scalable in that way.
> --
> nathanmarz: @jasonjckn On top of the bittorrent stuff we can ensure that a 
> topology is considered submitted only when the artifacts exist on at least N 
> nodes. Nimbus would only be the initial seed for topology files. Also, it 
> wouldn't have to only be Nimbus that acts as a seed, that work could be 
> shared by the supervisors. That's less relevant in the storm-mesos world, but 
> you could still fairly easily run multiple Nimbus's to get replication.
> --
> jasonjckn: This PR might be aided by "topology packages" #557, as it bundles 
> all the state that needs to be replicated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-893] Resource Aware Scheduling

[zhuoliu]

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/746#issuecomment-146692455
  
Nice work! +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1099) worker childopts cannot accept arraylist of strings

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949580#comment-14949580
 ] 

ASF GitHub Bot commented on STORM-1099:
---

GitHub user ppoulosk opened a pull request:

https://github.com/apache/storm/pull/791

[STORM-1099]  Fix worker childopts as arraylist of strings

This fixes the bug by changing supervisor.clj to operate on any sequential, 
not just a list.  

A unit test was added to verify that this works.  I ran the unit test 
without the supervisor change, and it did fail.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppoulosk/storm STORM-1099

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/791.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #791


commit 103125d428e13aed20acbd7881ab8b1aaa423ed9
Author: Paul Poulosky 
Date:   2015-10-08T22:51:34Z

Merge pull request #579 from ppoulosk/YSTORM-2270

[YSTORM-2270]  Fix arraylist of strings as config value for childopts




> worker childopts cannot accept arraylist of strings
> ---
>
> Key: STORM-1099
> URL: https://issues.apache.org/jira/browse/STORM-1099
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Paul Poulosky
>Assignee: Paul Poulosky
>
> This used to work on 0.9, but is broken in 0.10.
> If a customer attempts to add a array of strings as a config for 
> topology.worker.childopts, like the following.
> {noformat}
>   config.put("topology.worker.childopts", Arrays.asList("-Xmx1g", 
> "-Dmy.boxus.parameter=mybogusvalue"));
> {noformat}
> The topology will not launch because "[-Xmx1g" will be an argument to the jvm 
> when the supervisor launches it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1087: Avoid issues with transfer-queue b...

[zhuoliu]

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/782#issuecomment-146674662
  
Looks good to me! +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: [STORM-1093] Launching Workers with resources ...

[zhuoliu]

GitHub user zhuoliu opened a pull request:

https://github.com/apache/storm/pull/790

[STORM-1093] Launching Workers with resources specified in resource-aware 
schedulers

This request provides a set of schemes that allow nimbus to put different 
types of resources to each worker slot, then push the resources within 
assignment to ZooKeepers; also, at the supervisor side, such resources in each 
worker slot should be used by supervisor for launching a worker's JVM 
(initially, the JVM heap size, this can also be called by CGroup or Docker for 
resource segregation).
Such scheme can be used not only by RAS scheduler (STORM-893), but also by 
any customized scheduler for conducting mem/cpu resource aware scheduling.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhuoliu/storm 1093a

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/790.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #790


commit 066fd7aa89a7f50a1f5794a14a68312b5996a112
Author: zhuol 
Date:   2015-10-07T21:08:36Z

Add resource setting API in WorkerSlot

commit 8b327c9ed84ec2b29ad7c807f04b9928f6d25cbc
Author: Kishor Patil 
Date:   2015-08-21T16:37:19Z

Merge pull request #509 from zhuol/1779

[YSTORM-1779] Send/receive resource information from nimbus to supervisor 
for RAS

commit ca8bb24dbffd806ed41e27976552e197608d6627
Author: zhuol 
Date:   2015-10-07T21:48:33Z

Remove generated files

commit 715e64ba05bd691555d53d12c0daa851868fea03
Author: Derek Dagit 
Date:   2015-08-31T16:55:36Z

Cherry-pick YSTORM-1546

commit 3a045bf0f5077df155a00dc11ad859e6b9d6578d
Author: Zhuo Liu 
Date:   2015-09-16T22:02:42Z

Cherry-pick YSTORM-2146

commit 545df27ddd79d2a1175749887c1681abfbd74635
Author: zhuol 
Date:   2015-10-08T15:50:27Z

Minor Fix of the import

commit a684962022ad228397f86c09cb266a56afb75a59
Author: zhuol 
Date:   2015-10-08T21:19:49Z

Push Thrift file changes




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1085. Compile comparison, arithmetic, an...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/789#issuecomment-146702068
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1085) Compile comparison, arithmetic, and field reference expressions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949500#comment-14949500
 ] 

ASF GitHub Bot commented on STORM-1085:
---

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/789#issuecomment-146702068
  
+1


> Compile comparison, arithmetic, and field reference expressions
> ---
>
> Key: STORM-1085
> URL: https://issues.apache.org/jira/browse/STORM-1085
> Project: Apache Storm
>  Issue Type: New Feature
>Reporter: Haohui Mai
>Assignee: Haohui Mai
>
> This jira tracks the effort of implementing the comparison, arithmetic and 
> the field reference expressions in the expression compiler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-150: Replace jar distribution strategy w...

[ptgoetz]

Github user ptgoetz closed the pull request at:

https://github.com/apache/storm/pull/71


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1099) worker childopts cannot accept arraylist of strings

[Paul Poulosky (JIRA)]

Paul Poulosky created STORM-1099:


 Summary: worker childopts cannot accept arraylist of strings
 Key: STORM-1099
 URL: https://issues.apache.org/jira/browse/STORM-1099
 Project: Apache Storm
  Issue Type: Bug
Reporter: Paul Poulosky
Assignee: Paul Poulosky


This used to work on 0.9, but is broken in 0.10.

If a customer attempts to add a array of strings as a config for 
topology.worker.childopts, like the following.

{noformat}

  config.put("topology.worker.childopts", Arrays.asList("-Xmx1g", 
"-Dmy.boxus.parameter=mybogusvalue"));

{noformat}

The topology will not launch because "[-Xmx1g" will be an argument to the jvm 
when the supervisor launches it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1099] Fix worker childopts as arraylist...

[ppoulosk]

GitHub user ppoulosk opened a pull request:

https://github.com/apache/storm/pull/791

[STORM-1099]  Fix worker childopts as arraylist of strings

This fixes the bug by changing supervisor.clj to operate on any sequential, 
not just a list.  

A unit test was added to verify that this works.  I ran the unit test 
without the supervisor change, and it did fail.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppoulosk/storm STORM-1099

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/791.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #791


commit 103125d428e13aed20acbd7881ab8b1aaa423ed9
Author: Paul Poulosky 
Date:   2015-10-08T22:51:34Z

Merge pull request #579 from ppoulosk/YSTORM-2270

[YSTORM-2270]  Fix arraylist of strings as config value for childopts




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1087) Auto Backpressure can be stuck off for a worker

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949312#comment-14949312
 ] 

ASF GitHub Bot commented on STORM-1087:
---

Github user zhuoliu commented on the pull request:

https://github.com/apache/storm/pull/782#issuecomment-146674662
  
Looks good to me! +1


> Auto Backpressure can be stuck off for a worker
> ---
>
> Key: STORM-1087
> URL: https://issues.apache.org/jira/browse/STORM-1087
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> There is a race condition where automatic backpressure can get stuck off when 
> it should be on.  This can cause it to OOM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1087) Auto Backpressure can be stuck off for a worker

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949343#comment-14949343
 ] 

ASF GitHub Bot commented on STORM-1087:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/782#issuecomment-146677901
  
+1


> Auto Backpressure can be stuck off for a worker
> ---
>
> Key: STORM-1087
> URL: https://issues.apache.org/jira/browse/STORM-1087
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> There is a race condition where automatic backpressure can get stuck off when 
> it should be on.  This can cause it to OOM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1090) Nimbus HA should support `storm.local.hostname`

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949340#comment-14949340
 ] 

ASF GitHub Bot commented on STORM-1090:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/783#issuecomment-146677802
  
+1 Nice catch.


> Nimbus HA should support `storm.local.hostname`
> ---
>
> Key: STORM-1090
> URL: https://issues.apache.org/jira/browse/STORM-1090
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.11.0
>Reporter: Michael Schonfeld
>
> Nimbus HA's `NimbusInfo` class relies on each nimbus's hostname for network 
> reachability. This is a show-stopper in situations that utilize 
> Config.STORM_LOCAL_HOSTNAME /  `storm.local.hostname`.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1093) Launching Workers with resources specified in resource-aware schedulers

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949416#comment-14949416
 ] 

ASF GitHub Bot commented on STORM-1093:
---

GitHub user zhuoliu opened a pull request:

https://github.com/apache/storm/pull/790

[STORM-1093] Launching Workers with resources specified in resource-aware 
schedulers

This request provides a set of schemes that allow nimbus to put different 
types of resources to each worker slot, then push the resources within 
assignment to ZooKeepers; also, at the supervisor side, such resources in each 
worker slot should be used by supervisor for launching a worker's JVM 
(initially, the JVM heap size, this can also be called by CGroup or Docker for 
resource segregation).
Such scheme can be used not only by RAS scheduler (STORM-893), but also by 
any customized scheduler for conducting mem/cpu resource aware scheduling.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zhuoliu/storm 1093a

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/790.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #790


commit 066fd7aa89a7f50a1f5794a14a68312b5996a112
Author: zhuol 
Date:   2015-10-07T21:08:36Z

Add resource setting API in WorkerSlot

commit 8b327c9ed84ec2b29ad7c807f04b9928f6d25cbc
Author: Kishor Patil 
Date:   2015-08-21T16:37:19Z

Merge pull request #509 from zhuol/1779

[YSTORM-1779] Send/receive resource information from nimbus to supervisor 
for RAS

commit ca8bb24dbffd806ed41e27976552e197608d6627
Author: zhuol 
Date:   2015-10-07T21:48:33Z

Remove generated files

commit 715e64ba05bd691555d53d12c0daa851868fea03
Author: Derek Dagit 
Date:   2015-08-31T16:55:36Z

Cherry-pick YSTORM-1546

commit 3a045bf0f5077df155a00dc11ad859e6b9d6578d
Author: Zhuo Liu 
Date:   2015-09-16T22:02:42Z

Cherry-pick YSTORM-2146

commit 545df27ddd79d2a1175749887c1681abfbd74635
Author: zhuol 
Date:   2015-10-08T15:50:27Z

Minor Fix of the import

commit a684962022ad228397f86c09cb266a56afb75a59
Author: zhuol 
Date:   2015-10-08T21:19:49Z

Push Thrift file changes




> Launching Workers with resources specified in resource-aware schedulers
> ---
>
> Key: STORM-1093
> URL: https://issues.apache.org/jira/browse/STORM-1093
> Project: Apache Storm
>  Issue Type: Story
>  Components: storm-core
>Reporter: Zhuo Liu
>Assignee: Zhuo Liu
>
> Currently, we have Resource Aware Scheduler (STORM-894) in nimbus, which can 
> allocate different types of resource (CPU, onheap-memory, offheap-memory) to 
> the workers assigned to each topology's tasks.
> However, such resources are not visible to the supervisor, therefore, 
> supervisor will still launch workers with fixed amount of heap size memory 
> (e.g., -Xmx=768M). 
> Therefore, we need a whole set of schemes that allow nimbus to put different 
> types of resources to each worker slot, then push the resources with 
> assignment to ZooKeepers; also, at the supervisor side, such resources in 
> each worker slot should be used by supervisor for launching a worker's JVM 
> (initially, the JVM heap size).
> This scheme can be used not only by RAS scheduler (STORM-893), but also by 
> any customized scheduler for conducting mem/cpu/network resource 
> specified-scheduling.
> In the future, the resources of memory, CPU and network can also be used by 
> supervisor to launch a worker in a resource-segregated container, such as a 
> CGroup or Docker, with isolated Memory/CPU/Network resources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1091) Add unit test for tick tuples to HiveBolt and HdfsBolt

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949335#comment-14949335
 ] 

ASF GitHub Bot commented on STORM-1091:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/784#issuecomment-146677423
  
+1


> Add unit test for tick tuples to HiveBolt and HdfsBolt
> --
>
> Key: STORM-1091
> URL: https://issues.apache.org/jira/browse/STORM-1091
> Project: Apache Storm
>  Issue Type: Test
>  Components: storm-hdfs, storm-hive
>Reporter: Aaron Dossett
>Assignee: Aaron Dossett
>Priority: Minor
>
> Add tick tuple unit tests to HiveBolt and HdfsBolt.
> The storm-starter module contains a helpful MockTupleHelpers class.  
> Relocating it to the storm-core test suite will make these tests easier to 
> implement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1090] Nimbus HA should support `storm.l...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/783#issuecomment-146677802
  
+1 Nice catch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1091: Add tick tuple unit tests for hdfs...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/784#issuecomment-146677423
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-89) Can't use more complex types in the topology configuration

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-89:
--
Component/s: storm-core

> Can't use more complex types in the topology configuration
> --
>
> Key: STORM-89
> URL: https://issues.apache.org/jira/browse/STORM-89
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/441
> Storm's use of json-simple means that you can't use any type more complex 
> than Map/Array for values in the topology configuration. It would be nice to 
> switch to something like Jackson for json writing/parsing (and 
> coincidentally, it also supports Yaml) of the configuration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-80) NPE caused by TridentBoltExecutor reusing TrackedBatches between batch groups

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-80:
--
Component/s: storm-core

> NPE caused by TridentBoltExecutor reusing TrackedBatches between batch groups
> -
>
> Key: STORM-80
> URL: https://issues.apache.org/jira/browse/STORM-80
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/421
> I'm seeing intermittent errors caused by SubtopologyBolt.execute being called 
> with a BatchInfo whose ProcessorContext is set up for a different Batch 
> Group. In particular I'm seeing null pointer exceptions from 
> PartitionPersistProcessor because its state fields were never set up 
> correctly.
> The best I can tell the id key (IBatchID) being used for the _batches map in 
> TridentBoltExecutor is not unique between batch groups. As a result the 
> tracked batch will have been initialized for a different Batch Group and set 
> of processors.
> I hoped to be able to track down the source of this issue but can't determine 
> where the BatchIDs are being added to the tuples.
> If it matters, my topology has two streams each reading from their own 
> OpaqueTransactionalKafka spout w/different topics.
> Backtrace:
> 65108 [Thread-25] ERROR backtype.storm.daemon.executor - 
> java.lang.RuntimeException: java.lang.NullPointerException
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.daemon.executor$fn__3551$fn__3563$fn__3610.invoke(executor.clj:712)
>  ~[storm-0.9.0-wip4.jar:na]
> at backtype.storm.util$async_loop$fn__436.invoke(util.clj:377) 
> ~[storm-0.9.0-wip4.jar:na]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:722) [na:1.7.0_09]
> Caused by: java.lang.NullPointerException: null
> at 
> storm.trident.planner.processor.PartitionPersistProcessor.execute(PartitionPersistProcessor.java:59)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> storm.trident.planner.SubtopologyBolt$InitialReceiver.receive(SubtopologyBolt.java:189)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> storm.trident.planner.SubtopologyBolt.execute(SubtopologyBolt.java:129) 
> ~[storm-0.9.0-wip4.jar:na]
> at 
> storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:352)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.daemon.executor$fn__3551$tuple_action_fn__3553.invoke(executor.clj:607)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.daemon.executor$mk_task_receiver$fn__3474.invoke(executor.clj:379)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.disruptor$clojure_handler$reify__3011.onEvent(disruptor.clj:43)
>  ~[storm-0.9.0-wip4.jar:na]
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:84)
>  ~[storm-0.9.0-wip4.jar:na]
> ... 6 common frames omitted
> Also, I'm only seeing this in LocalCluster mode, not in production.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-82) Please support settings for changing the worker launching command

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-82?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-82:
--
Component/s: storm-core

> Please support settings for changing the worker launching command
> -
>
> Key: STORM-82
> URL: https://issues.apache.org/jira/browse/STORM-82
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/124
> The `collect' tool in Oracle Solaris Studio can be used to profile Java 
> applications, but it requires the command line to transform from
> java 
> to
> collect -j on java 
> Currently, the only simple, non-compilation way to use collect' with Storm is 
> to write a wrapper aroundjava' to invoke `collect' as needed. Please support 
> settings for changing the command to launch a worker, so that there is no 
> need of such a wrapper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-91) Registering already registered serializations causes strange runtime errors

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-91:
--
Component/s: storm-core

> Registering already registered serializations causes strange runtime errors
> ---
>
> Key: STORM-91
> URL: https://issues.apache.org/jira/browse/STORM-91
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/279
> e.g. if registering String:
> java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: 
> java.lang.IllegalArgumentException: Class is not registered: char[]
> Note: To register this class use: kryo.register(char[].class);
> Serialization trace:
> value (java.lang.String)
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:82)
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:55)
> at 
> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:56)
> at 
> backtype.storm.disruptor$consume_loop_STAR_$fn__1596.invoke(disruptor.clj:67)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: com.esotericsoftware.kryo.KryoException: 
> java.lang.IllegalArgumentException: Class is not registered: char[]
> Note: To register this class use: kryo.register(char[].class);
> Serialization trace:
> value (java.lang.String)
> at 
> com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:495)
> at 
> com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:213)
> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:554)
> at 
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:77)
> at 
> com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:18)
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:472)
> at 
> backtype.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:27)
> at 
> backtype.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:27)
> at 
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4120$fn__4124.invoke(worker.clj:99)
> at backtype.storm.util$fast_list_map.invoke(util.clj:770)
> at 
> backtype.storm.daemon.worker$mk_transfer_fn$fn__4120.invoke(worker.clj:99)
> at 
> backtype.storm.daemon.executor$start_batch_transfer__GT_worker_handler_BANG_$fn__3898.invoke(executor.clj:205)
> at 
> backtype.storm.disruptor$clojure_handler$reify__1584.onEvent(disruptor.clj:43)
> at 
> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:79)
> ... 6 more
> Caused by: java.lang.IllegalArgumentException: Class is not registered: char[]
> Note: To register this class use: kryo.register(char[].class);
> at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:426)
> at com.esotericsoftware.kryo.Kryo.getSerializer(Kryo.java:446)
> at 
> com.esotericsoftware.kryo.serializers.FieldSerializer$ObjectField.write(FieldSerializer.java:477)
> ... 19 more
> This error catching should happen at registration time (ideally in Kryo 
> itself). Or, even better, Storm should allow duplicate registrations and skip 
> any duplicates when initializing Kryo. This is consistent with how 
> serialization registration works w.r.t. component configs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (STORM-692) KakfaSpout storage of offsets in Zookeeper lost

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg closed STORM-692.
--

> KakfaSpout storage of offsets in Zookeeper lost
> ---
>
> Key: STORM-692
> URL: https://issues.apache.org/jira/browse/STORM-692
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-kafka
>Affects Versions: 0.9.3
>Reporter: Rick Kellogg
>Assignee: Rick Kellogg
>
> Constructor for SpoutConfig and inadequate documentation in 
> storm-kafka/README.md leads to lost persistence of Kafka offset information.
> The SpoutConfig class constructor takes four parameters:
> hosts - Zookeeper host & root path for Kafka 
> topic - Kafka topic name
> zkRoot - Zookeeper path for Kafka Spout offset
> id -- Identifier for KafkaSpout 
> Unfortunately it also exposes two public instance variables that must also be 
> set: zkServers & zkPort.  If these two variables are not set it uses Storm 
> default values(localhost & 2000) which cause the offset information to be 
> lost.
> Suggest we replace the existing constructor with one that supplies all 6 
> required values.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-78) Storm UI Needs HTTP Auth

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-78:
--
Component/s: storm-core

> Storm UI Needs HTTP Auth
> 
>
> Key: STORM-78
> URL: https://issues.apache.org/jira/browse/STORM-78
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/452
> When we start storm ui, any one can access it by ip / port. It would be good 
> if there is a HTTP Auth for basic security.
> --
> Jagaran: We are also implementing Storm for one of our Client.
> We also need to secure HTTP based Storm UI ? Any updates or future road map 
> for implementing this Security ??
> --
> Jargaran: I can also contribute to this part in Storm ? let me know the steps
> --
> lockwobr: I would also really find the feature useful. when you are in cloud 
> you can't just let this kind of thing hanging in the wind.
> --
> tvpavan: @Jagaran Do you have a fork which has this fix ?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-60) Document Thrift APIs

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-60:
--
Issue Type: Documentation  (was: Improvement)

> Document Thrift APIs
> 
>
> Key: STORM-60
> URL: https://issues.apache.org/jira/browse/STORM-60
> Project: Apache Storm
>  Issue Type: Documentation
>  Components: documentation
>Reporter: James Xu
>  Labels: newbie
>
> https://github.com/nathanmarz/storm/issues/61



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-68) Need multiple output streams for a bolt in Trident

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-68:
--
Component/s: storm-core

> Need multiple output streams for a bolt in Trident
> --
>
> Key: STORM-68
> URL: https://issues.apache.org/jira/browse/STORM-68
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>
> https://github.com/nathanmarz/storm/issues/638
> Nathan suggested to open an issue for this:
> Transactional Topologies gave us the flexibility to have multiple output 
> streams from a bolt from which different downstreamer bolts can subscribe per 
> bolt per stream; while in Trident, though the bolt can take multiple input 
> streams, it is not possible (as per our understanding) to emit different 
> streams. All downstreamer bolts have to subscribe to the same output stream 
> from the bolt.
> The only way to solve this in the current system is to tunnel multiple 
> streams in the outgoing stream, which all downstreamers receive and then 
> demultiplex it in each of those bolts and select whatever stream the bolt is 
> interested in. This will have perf implications for our usecases which deals 
> with good amount of traffic.
> The discussion:
> https://groups.google.com/forum/#!topic/storm-user/G8POD1Hb89I



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-982) Random test failures on backtype.storm.grouping-test

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-982:
---
Component/s: storm-core

> Random test failures on backtype.storm.grouping-test
> 
>
> Key: STORM-982
> URL: https://issues.apache.org/jira/browse/STORM-982
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Jungtaek Lim
>
> {code}
> Error parsing 
> /home/travis/build/HeartSaVioR/storm/storm-core/target/test-reports/backtype.storm.grouping-test.xml
> 
> 
> 
>  classname="backtype.storm.grouping-test">
> 
> 

[jira] [Updated] (STORM-1089) Add tool for recording and playback of storm traces

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-1089:

Component/s: storm-core

> Add tool for recording and playback of storm traces
> ---
>
> Key: STORM-1089
> URL: https://issues.apache.org/jira/browse/STORM-1089
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: Robert Joseph Evans
>
> We currently report information about latency and throughput of tuples 
> flowing through storm. It would really be nice to have a tool that can gather 
> this information over a period of time, anonymizes it if needed, and records 
> it to a file.
> Then have another tool that will replay that data so we can test how changes 
> to storm would impact a given topology or set of topologies.
> It would be great if we could capture the entire cluster, And potentially add 
> in more details around CPU, memory, Network, tuple size, etc as we are able 
> to gather more information.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-979) [storm-elasticsearch] Introduces BaseQueryFunction to query to ES while using Trident

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-979:
---
Component/s: storm-elasticsearch

> [storm-elasticsearch] Introduces BaseQueryFunction to query to ES while using 
> Trident
> -
>
> Key: STORM-979
> URL: https://issues.apache.org/jira/browse/STORM-979
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-elasticsearch
>Reporter: Jungtaek Lim
>
> storm-elasticsearch has features on storing document, not querying something.
> It would be better to have BaseQueryFunction for querying to ES and emit 
> matched documents, as other external modules did.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-128) Topology fails to start if a configured DRPC server is down

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-128:
---
Component/s: storm-core

> Topology fails to start if a configured DRPC server is down
> ---
>
> Key: STORM-128
> URL: https://issues.apache.org/jira/browse/STORM-128
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/696
> In our environment we have 3 DRPC servers running. This was done mainly for 
> availability and capacity. However, we noticed that when even one of these 
> servers is down, topologies fail to start with the following exception:
> java.lang.RuntimeException: org.apache.thrift7.transport.TTransportException: 
> java.net.NoRouteToHostException: No route to host
> at backtype.storm.drpc.DRPCInvocationsClient.(DRPCInvocationsClient.java:23)
> at backtype.storm.drpc.DRPCSpout.open(DRPCSpout.java:65)
> at 
> storm.trident.spout.RichSpoutBatchTriggerer.open(RichSpoutBatchTriggerer.java:41)
> at backtype.storm.daemon.executor$fn__3985$fn__3997.invoke(executor.clj:460)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:375)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.thrift7.transport.TTransportException: 
> java.net.NoRouteToHostException: No route to host
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
> at 
> org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
> at 
> backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:30)
> at backtype.storm.drpc.DRPCInvocationsClient.(DRPCInvocationsClient.java:21)
> ... 6 more
> Caused by: java.net.NoRouteToHostException: No route to host
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
> ... 9 more
> I was wondering if it makes sense to make Storm handle this gracefully 
> instead of failing fast. Otherwise, the DRPC servers become a SPOF.
> If the topologies are already running the topology usually just logs an error 
> message and continues.
> --
> dkador: +1 on figuring out how to make the DRPC stuff not a SOP. I'd be happy 
> to look into it myself but not sure where to start. Any guidance?
> --
> rijuk: For reference, the stack trace I see when a DRPC server goes down 
> while a topology is running is the following. In this case, the topology 
> continues to function normally.
> [backtype.storm.drpc.DRPCSpout Thread-65]: Failed to fetch DRPC result from 
> DRPC server
> org.apache.thrift7.transport.TTransportException: java.net.ConnectException: 
> Connection refused
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:183)
> at 
> org.apache.thrift7.transport.TFramedTransport.open(TFramedTransport.java:81)
> at 
> backtype.storm.drpc.DRPCInvocationsClient.connect(DRPCInvocationsClient.java:30)
> at 
> backtype.storm.drpc.DRPCInvocationsClient.fetchRequest(DRPCInvocationsClient.java:53)
> at backtype.storm.drpc.DRPCSpout.nextTuple(DRPCSpout.java:89)
> at 
> storm.trident.spout.RichSpoutBatchTriggerer.nextTuple(RichSpoutBatchTriggerer.java:68)
> at 
> backtype.storm.daemon.executor$fn__3985$fn__3997$fn__4026.invoke(executor.clj:502)
> at backtype.storm.util$async_loop$fn__465.invoke(util.clj:377)
> at clojure.lang.AFn.run(AFn.java:24)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.thrift7.transport.TSocket.open(TSocket.java:178)
> ... 9 more
> In this case I'd the host up, but the DRPC server process was down. Hence the 
> ConnectException. But, the behavior is the same even when the host is 
> unreachable, except for the Exception type.
> @dkador, I'm not sure what the right solution is. One naive solution I can 
> think of is to make DRPCInvocationsClient constructor rethrow TException 
> instead of throwing a RuntimeException. Obviously, 

[jira] [Updated] (STORM-30) Modularized Storm artifacts for using Storm as a library

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-30?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-30:
--
Component/s: storm-core

> Modularized Storm artifacts for using Storm as a library
> 
>
> Key: STORM-30
> URL: https://issues.apache.org/jira/browse/STORM-30
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/313
> When using Storm as a library (for example, the DRPC client), storm-lib 
> brings in a lot of dependencies you probably don't need, such as all the 
> UI-related libraries. It would be nice if the build were more sophisticated, 
> so that rather than have one giant storm-lib, you'd have dependencies for:
> Just the Thrift code + helpful wrapper like DRPCClient
> Storm itself
> The UI stuff
> Leiningen 2 may be able to do this via submodules



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-131) Intermittent Zookeper errors when shutting down local Topology

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-131:
---
Component/s: storm-core

> Intermittent Zookeper errors when shutting down local Topology
> --
>
> Key: STORM-131
> URL: https://issues.apache.org/jira/browse/STORM-131
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/259
> We have a great deal of Storm integration tests in our project (Storm version 
> 0.7.3) using local topology. We have only one topology operational at any 
> moment in time. As tests run they are organized in groups. Each group works 
> within the boundaries of a topology. When the tests finish executing they 
> shutdown their local cluster, then the new group of tests launches its own 
> cluster.
> We see with some remarkable regularity failures related to, what looks like, 
> incorrect Zookeeper shutdown, which leads to a JVM exit (which is a disaster 
> as no test information is recorded at the end). Here is what we see in the 
> main error log (log level: WARN and higher):
> {code}
> 2012-07-07 00:22:58,420 WARN [ConnectionStateManager-0|]@jenkins 
> com.netflix.curator.framework.state.ConnectionStateManager
> => There are no ConnectionStateListeners registered.
> 2012-07-07 00:22:58,534 WARN [Thread-23-EventThread|]@jenkins 
> backtype.storm.cluster
> => Received event :disconnected::none: with disconnected Zookeeper.
> 2012-07-07 00:23:00,013 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-07-07 00:23:01,527 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-07-07 00:23:03,510 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-07-07 00:23:04,687 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-07-07 00:23:05,961 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-07-07 00:23:07,588 WARN [Thread-23-SendThread(localhost:2000)|]@jenkins 
> org.apache.zookeeper.ClientCnxn
> => Session 0x1385ece8f1b0017 for server null, unexpected error, closing 
> socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method)
> at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java)
> at 

[jira] [Updated] (STORM-136) [Trident] Bug when working with two spouts with stateful operator

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-136:
---
Component/s: storm-core

> [Trident] Bug when working with two spouts with stateful operator
> -
>
> Key: STORM-136
> URL: https://issues.apache.org/jira/browse/STORM-136
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/711
> Hi,
> I'm working with Trident and basically what i am trying to do is a windowed 
> join across batches using partitionPersist and stateQuery on two different 
> stream that come from TWO DIFFERENT SPOUTS.
> Both the spouts implement the IBatchSpout interface.
> The error i get is a NPE on StateQueryProcessor or on 
> PartitionPersistProcessor depending on which one of the two spouts starts 
> early.
> I try to debug this and what i have understand is that Trident use the same 
> BatchID(txid) for the two different spouts and this take to a wrong 
> initialization of state in the core processing nodes.
> If i use the same throughput for the two spout and i make one spout starts 
> with a delay the problem doesn't occur (we don't have an overlap between the 
> BachID(txid) of the different spouts).
> --
> liesrock: Sorry for the delay.
> I send to you the source code.
> https://dl.dropboxusercontent.com/u/49470846/TridentBug.tar.gz
> In this example i am using two spout:
> 1) the first one starts emitting tuples at a certain rate.
> 2) the second one is delayed by 5 seconds and emits at an higher rate than 
> the first one.
> I print on the sterr some debug info, the most important is the Batch ID 
> (txid).
> I print on the stdout info about the state window.
> You can see that this is exactly the case that i was explain in my first 
> message.
> Thank you for your attention, i can't figure out on this, so i left Trident 
> and i start working with Storm(i didn't have problem in implementing this on 
> Storm).
> Let me know what is your opinion about it.
> Luca Muratore
> --
> xumingming: @liesrock try to minimize your test case so it can reproduce the 
> issue but contains the least source files(preferably only one source file), 
> the least lines of source code --- that makes us more easier to see whether 
> the issue is in your app code or in storm core.
> --
> liesrock: As you requested, in the following link you can find the test case 
> in one source file and with the minimum lines of code:
> https://dl.dropboxusercontent.com/u/49470846/SimpleTridentBug.zip
> Thanks for your patience.
> --
> xumingming: paste the case source code here to make it easier for others to 
> follow:
> package storm.starter.trident;
> import java.io.IOException;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.Map;
> import storm.trident.Stream;
> import storm.trident.TridentState;
> import storm.trident.TridentTopology;
> import storm.trident.operation.TridentCollector;
> import storm.trident.spout.IBatchSpout;
> import storm.trident.state.BaseQueryFunction;
> import storm.trident.state.BaseStateUpdater;
> import storm.trident.state.State;
> import storm.trident.state.StateFactory;
> import storm.trident.tuple.TridentTuple;
> import backtype.storm.Config;
> import backtype.storm.LocalCluster;
> import backtype.storm.task.IMetricsContext;
> import backtype.storm.task.TopologyContext;
> import backtype.storm.tuple.Fields;
> import backtype.storm.tuple.Values;
> public class BugTest {
> //Spout that simulates an ATM
> @SuppressWarnings("serial")
> public static class ATMSpout implements IBatchSpout {   
> private int batchSize;
> private int initialSleepMilliTime;
> private int rate;
> private String name;
> private List currentCCIDList;
> private long withdrawalID = 0;
> private final static String[] LOCATIONS = { "Madrid", "Barcelona", 
> "Siviglia", "Granada", "Toledo", "Ibiza", "Valladolid", "Valencia" };
> public ATMSpout(int batchSize, int initialSleepMilliTime, int rate, 
> String name) throws IOException {
> this.batchSize = batchSize;
> this.initialSleepMilliTime = initialSleepMilliTime;
> this.rate = rate;
> this.name = name;
> }
> //generate a list of "size" CCID
> private List generateCCID(int size){
> //check the input parameter
> if(size < 0){
> System.err.println("Negative CCID list size");
> return null;
> }
> //initialize some variables
> List aux = new ArrayList();
> Integer randDigit = 0;
> String currentCCID = "";
> 

[jira] [Updated] (STORM-46) Allow scheduler to configure memory per worker

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-46:
--
Component/s: storm-core

> Allow scheduler to configure memory per worker
> --
>
> Key: STORM-46
> URL: https://issues.apache.org/jira/browse/STORM-46
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>Assignee: SHAILESH PILARE
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/272
> Useful if different workers have different memory needs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-38) Should be able to do state updates through DRPC

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-38?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-38:
--
Component/s: storm-core

> Should be able to do state updates through DRPC
> ---
>
> Key: STORM-38
> URL: https://issues.apache.org/jira/browse/STORM-38
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/307
> The batch size of 1 might make it inefficient, but it should still be 
> possible. In addition, there should be a "batch DRPC" in which a single batch 
> contains multiple DRPC request tuples. Each tuple would be [request id, 
> args], and the result should be `[[request id, ...] ...]"
> -
> childs: @nathanmarz can you given any hint as to wether this is nice to have 
> or we need to do this from your point of view?
> -
> nathanmarz: This is something we most definitely want to implement.
> -
> danielpizarro: I've just hit the wall with this today. @nathanmarz, do you 
> have any hints of a workaround to get same functionality?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-126) Add Lifecycle support API for worker nodes

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-126:
---
Component/s: storm-core

> Add Lifecycle support API for worker nodes
> --
>
> Key: STORM-126
> URL: https://issues.apache.org/jira/browse/STORM-126
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/155
> Storm is already used in variety of environments. It is important that Storm 
> provides some form of "lifecycle" API specified at Topology builder level to 
> be called when worker nodes start and stop.
> It is a very crucial functional piece that is missing from Storm. Many 
> project have to integrate, for example, with various container-like 
> frameworks like Spring or Google Guice that need to be started at stopped in 
> a controlled fashion before worker nodes begin or finish their work.
> I think something like a WorkerContextListener interface with two methods: 
> onStartup(SomeContextClass) and onShutdown(SomeContextClass) could go a very 
> long way for allowing to user's to plugin various third-party libraries 
> easily.
> Then, the TopologyBuilder needs to be modified to accept classes that 
> implement this interface.
> SomeContextClass does not need to be much more than a Map for now. But it is 
> important to have it as it allows propagation ofl information between those 
> lifecycle context listeners.
> Nathan, it would interesting to hear your opinion.
> Thanks!
> --
> nathanmarz: I agree, this should be added to Storm. The lifecycle methods 
> should be parameterized with which tasks are running in this worker.
> Additionally, I think lifecycle methods should be added for bolt/spouts in 
> the context of workers. Sometimes there's some code you want to run for a 
> spout/bolt within a worker only one time, regardless of how many tasks for 
> that bolt are within the worker. Then individual tasks should be able to 
> access that "global state" within the worker for that spout/bolt.
> --
> kyrill007: Thank you, Nathan, I think it would be relatively simple to 
> implement and would have big impact. Now we're forced to manage container 
> initializations through lazy static fields. You'd love to see that code. :-)
> --
> nathanmarz: Yup, this should be fairly easy to implement. I encourage you to 
> submit a patch for this.
> --
> kyrill007: Oh, well, my Clojure is unfortunately too weak for this kind of 
> work. But I am working on it... Any pointers as to where in Storm code 
> workers are started and stopped?
> --
> nathanmarz: Here's the function that's called to start a worker: 
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/daemon/worker.clj#L315
> And here's the code in the same file that shuts down a worker:
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/daemon/worker.clj#L352
> I think the interface for the lifecycle stuff should look something like this:
> interface WorkerHook extends Serializable {
> void start(Map conf, TopologyContext context, List taskIds);
> void shutdown();
> }
> You'll need to add a definition for worker hooks into the topology definition 
> Thrift structure:
> https://github.com/nathanmarz/storm/blob/master/src/storm.thrift#L91
> I think for the first go it's ok to make this a Java-only feature by adding 
> something like "4: list worker_hooks" to the StormTopology structure 
> (where the "binary" refers to a Java-serialized object).
> Then TopologyBuilder can have a simple "addWorkerHook" method that will 
> serialize the object and add it to the Thrift struct.
> --
> danehammer: I've started working on this. I've followed Nathan's proposed 
> design, but I keep hitting snags with calls to ThriftTopologyUtils, now that 
> there is an optional list on StormTopology.
> I would like to add some unit tests for what I change there, would it make 
> more sense for those to be in Java instead of Clojure? If so, are there any 
> strong preferences on what dependencies I add and how I go about adding Java 
> unit tests to storm-core?
> --
> nathanmarz: No... unit tests should remain in Clojure. You can run Java code 
> in Clojure very easily. Here's a good example of testing Java code in 
> Clojure: 
> https://github.com/nathanmarz/storm/blob/master/storm-core/test/clj/backtype/storm/security/auth/AuthUtils_test.clj
> --
> danehammer: For this design:
> interface WorkerHook extends Serializable {
> void start(Map conf, TopologyContext context, List taskIds);
> void shutdown();
> }
> When mk-worker has the worker hooks and goes to call start on them, where 
> should it get the topology context? I started with what was 

[jira] [Updated] (STORM-49) Make State implement Closeable

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-49?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-49:
--
Component/s: storm-core

> Make State implement Closeable
> --
>
> Key: STORM-49
> URL: https://issues.apache.org/jira/browse/STORM-49
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/644
> It would be useful to be able to 'close' a State object when the topology is 
> being shut down, so that it can cleanly close any connections & resources it 
> is using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-41) show worker's working directory in storm UI

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-41?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-41:
--
Component/s: storm-core

> show worker's working directory in storm UI
> ---
>
> Key: STORM-41
> URL: https://issues.apache.org/jira/browse/STORM-41
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/345
> Show worker's working directory in storm UI. This helps you locate log files 
> quickly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-178) sending metrics to Ganglia

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-178:
---
Component/s: storm-core

> sending metrics to Ganglia
> --
>
> Key: STORM-178
> URL: https://issues.apache.org/jira/browse/STORM-178
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: caofangkun
>Assignee: caofangkun
>Priority: Minor
> Attachments: storm-ganglia.png
>
>
> provide something like ./conf/storm-metrics.properties 
> and GangliaContext.java 
> Collect and sent metrics to Ganglia



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-254) one Spout/Bolt can register metric twice with same name

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-254:
---
Component/s: storm-core

> one Spout/Bolt can register metric twice with same name
> ---
>
> Key: STORM-254
> URL: https://issues.apache.org/jira/browse/STORM-254
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: DashengJu
>Assignee: DashengJu
> Fix For: 0.9.3
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> In a Bolt's prepare method, we can register metrics twice with the same name, 
> using different timeBucketSizeInSecs parameter, like this:
> public void prepare(Map stormConf, TopologyContext context) {
>   mapper = new ObjectMapper();
>   
>   cMetric = new MTCountMetric();
>   context.registerMetric("JavaBoltCount", cMetric, 120);
>   ccMetric = new MTCountMetric();
>   context.registerMetric("JavaBoltCount", ccMetric, 60);
> }
> 
> This is caused by TopologyContext's registerMetric. In TopologyContext, all 
> registered metrics holds in a map defined below:
> private Map>> _registeredMetrics;
> timeBucketSizeInSecs > __taskId > metricName > metirc



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-342) Contention in Disruptor Queue which may cause message loss or out of order

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-342:
---
Component/s: storm-core

> Contention in Disruptor Queue which may cause message loss or out of order
> --
>
> Key: STORM-342
> URL: https://issues.apache.org/jira/browse/STORM-342
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Sean Zhong
>Assignee: Sean Zhong
>Priority: Blocker
> Fix For: 0.9.2-incubating
>
>
> h2. STORM-342: Message loss, executor hang, or message disorder
> Disruptor helper class contains a potential contention bug between consumer 
> and producer. It can cause consume queue hang, message loss, or message 
> disorder.
> {code:title=Disruptor.java|borderStyle=solid}
> class Disruptor {
> ...
> public void publish(Object obj, boolean block) throws 
> InsufficientCapacityException {
> if(consumerStartedFlag) {
> final long id;
> if(block) {
> id = _buffer.next();
> } else {
> id = _buffer.tryNext(1);
> }
> final MutableObject m = _buffer.get(id);
> m.setObject(obj);
> _buffer.publish(id);
> } else {
> _cache.add(obj);
> if(consumerStartedFlag) flushCache();
> }
> }
> public void consumerStarted() {
> if(!consumerStartedFlag) {
> consumerStartedFlag = true;
> flushCache();
> }
> }
> }
> {code}
> Consumer
> {code:title=Task Executor Thread|borderStyle=solid}
>   (disruptor/consumer-started! receive-queue)
>   (fn []
>  (disruptor/consume-batch-when-available receive-queue event-handler)
> {code}
> h3. Howto: Executor Hang, message loss:
> 1. [Consumer Thread] consumer not started.
> 2. [Producer A Thread] publish message "1", as "consumerStartedFlag" == 
> false, it will be added it into cache.
> 3. [Consumer Thread] consumerStarted() is called. consumerStartedFlag is set 
> to true, but flushCache() is not called yet.
> 4. As "consumerStartedFlag" is true now, new produced message will be 
> published to RingBuffer.
> 5. [Producer B Thread] generates enough message, and make RingBuffer full.
> 6. [Consumer Thread] flushCache() is called in consumerStarted()
> 7. [Consumer Thread] FLUSH_CACHE object is published RingBuffer in blocking 
> way, As now RingBuffer is full, the consumer thread will be blocked.
> 8. [Consumer Thread] consumeBatch() will never called, so the RingBuffer is 
> always full, and the consumer thread is always blocked.
> h3. Howto: Message Disorder
> 1. [Consumer Thread] consumer not started.
> 2. [Producer A Thread] publish message "1", as "consumerStartedFlag" == 
> false, it will be added it into cache.
> 3. [Consumer Thread] consumerStarted() is called. consumerStartedFlag is set 
> to true, but flushCache() is not called yet.
> 4. As "consumerStartedFlag" is true now, new produced message will be 
> published to RingBuffer.
> 5. [Producer A Thread] publish a new message "2", it will be published 
> directly in RingBuffer.
> 6. [Consumer Thread] flushCache() is called in consumerStarted()
> 7. [Consumer Thread] FLUSH_CACHE message is published RingBuffer, FLUSH_CACHE 
> message is written after message "2".
> 8. [Consumer Thread] consumeBatch() is called, first it picks "2", then it 
> picks FLUSH_CACHE, will represents "1"
> 9. We produce in Producer A Thread in order "1", "2", but we received in 
> consumer thread "2", "1"
> 10. Message order is wrong.
> I found this after troubleshooting a tricky random failure(1 in 100 times). 
> It usually happen when producer and consumer colocated in same process, for 
> example,  the task send queue thread as producer, produce message to local 
> task receive queue in same worker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-187) From ZMQ to Netty "java.lang.IllegalArgumentException: timeout value is negative"

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-187:
---
Component/s: storm-core

> From ZMQ to Netty "java.lang.IllegalArgumentException: timeout value is 
> negative"
> -
>
> Key: STORM-187
> URL: https://issues.apache.org/jira/browse/STORM-187
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Jose Ignacio Honrado
>Assignee: P. Taylor Goetz
>Priority: Blocker
>  Labels: netty, storm, zeromq
> Fix For: 0.9.2-incubating
>
> Attachments: STORM-187.patch
>
>
> Hi,
> I am trying to use Netty as the transport layer in storm 0.9.0.1 but I am 
> getting the following error trace:
> {code}
> java.lang.IllegalArgumentException: timeout value is negative
> at java.lang.Thread.sleep(Native Method)
> at backtype.storm.messaging.netty.Client.reconnect(Client.java:78)
> at 
> backtype.storm.messaging.netty.StormClientHandler.exceptionCaught(StormClientHandler.java:108)
> at 
> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377)
> at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
> at 
> org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:109)
> at 
> org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:78)
> at 
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> at 
> org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:41)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> I am using the following supervisor config:
> {code}
> storm.zookeeper.servers:
> - "son-rtl-dev-zook1"
> storm.zookeeper.port: 9000
> storm.local.dir: "/mnt/storm"
> storm.local.hostname: "son-rtl-dev-superv1"
> java.library.path: "/usr/local/lib"
> nimbus.host: "son-rtl-dev-nimbus"
> nimbus.task.launch.secs: 240
> supervisor.worker.start.timeout.secs: 240
> supervisor.worker.timeout.secs: 240
> supervisor.childopts: "-Xmx512m -Djava.net.preferIPv4Stack=true"
> supervisor.slots.ports:
> - 6700
> - 6701
> - 6702
> - 6703
> - 6704
> - 6705
> - 6706
> - 6707
> worker.childopts: "-Xmx512m -Djava.net.preferIPv4Stack=true"
> nimbus.childopts: "-Xmx512m -Djava.net.preferIPv4Stack=true"
> topology.message.timeout.secs: 1
> storm.messaging.transport: "backtype.storm.messaging.netty.Context"
> storm.messaging.netty.server_worker_threads: 1
> storm.messaging.netty.client_worker_threads: 1
> storm.messaging.netty.buffer_size: 5242880
> storm.messaging.netty.max_retries: 30
> storm.messaging.netty.max_wait_ms: 1000
> storm.messaging.netty.min_wait_ms: 100
> {code}
> Without these last config lines, topologies are working fine using ZMQ.
> Any idea?
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-273) Error while running storm topologies on Windows using "storm jar"

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-273:
---
Component/s: storm-core

> Error while running storm topologies on Windows using "storm jar"
> -
>
> Key: STORM-273
> URL: https://issues.apache.org/jira/browse/STORM-273
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Ramya Sunil
>Assignee: Suresh Srinivas
>  Labels: windows
> Fix For: 0.9.2-incubating
>
> Attachments: STORM-273.patch
>
>
> While running some example topologies from storm-starter on Windows, I 
> noticed the following error during file copy:
> D:\storm\bin\storm.cmd jar D:\storm\contrib\storm-starter\storm-starter.jar 
> storm.starter.BasicDRPCTopology
> {code}
> 23402 [Thread-6] INFO  backtype.storm.daemon.supervisor - Downloading code 
> for storm id drpc-demo-1-1396297060 from 
> C:\Users\hadoopqa\AppData\Local\Temp\60b71f41-e4b4-4f76-843a-7cb4d1aadcb3\nimbus\stormdist\drpc-demo-1-1396297060
> 24485 [Thread-6] INFO  backtype.storm.daemon.supervisor - Copying resources 
> at jar:file:/D:\storm\contrib\storm-starter\storm-starter.jar!/resources to 
> C:\Users\hadoopqa\AppData\Local\Temp\220d4bc5-7833-40f1-ad61-73ba62d6035a\supervisor\stormdist\drpc-demo-1-1396297060\resources
> 24488 [Thread-6] ERROR backtype.storm.event - Error when processing event
> java.io.FileNotFoundException: Source 
> 'file:\D:\storm\contrib\storm-starter\storm-starter.jar!\resources' does not 
> exist
> at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1368) 
> ~[commons-io-2.4.jar:2.4]
> at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1261) 
> ~[commons-io-2.4.jar:2.4]
> at org.apache.commons.io.FileUtils.copyDirectory(FileUtils.java:1230) 
> ~[commons-io-2.4.jar:2.4]
> at 
> backtype.storm.daemon.supervisor$eval6588$fn__6589.invoke(supervisor.clj:493) 
> ~[na:na]
> at clojure.lang.MultiFn.invoke(MultiFn.java:172) ~[clojure-1.4.0.jar:na]
> at 
> backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__6491.invoke(supervisor.clj:325)
>  ~[na:na]
> at backtype.storm.event$event_manager$fn__2583.invoke(event.clj:39) ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) [clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:722) [na:1.7.0_21]
> 24507 [Thread-6] INFO  backtype.storm.util - Halting process: ("Error when 
> processing an event")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-196) When JVM_OPTS are set, storm jar fails to detect storm.jar from environment

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-196:
---
Component/s: storm-core

> When JVM_OPTS are set, storm jar fails to detect storm.jar from environment
> ---
>
> Key: STORM-196
> URL: https://issues.apache.org/jira/browse/STORM-196
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Alexander Yerenkow
>Assignee: Alexander Yerenkow
>Priority: Blocker
> Fix For: 0.9.2-incubating
>
>
> Pull request:
> https://github.com/apache/incubator-storm/pull/27
> More info:
> http://mail-archives.apache.org/mod_mbox/storm-user/201312.mbox/%3cCAPJF9w=pz3tab+g_2s9peqbrkgmkwhwaosrm6yfnk-ivfqg...@mail.gmail.com%3e
> Current pull request uses "shlib" and parse JVM parameters correctly



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-270) Why ahead-of-time compiled clojure "classes" are not used in storm-core

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-270:
---
Component/s: storm-core

> Why ahead-of-time compiled clojure "classes" are not used in storm-core
> ---
>
> Key: STORM-270
> URL: https://issues.apache.org/jira/browse/STORM-270
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-core
>Reporter: Rafael Bagmanov
>Assignee: Robert Joseph Evans
> Fix For: 0.9.2-incubating
>
>
> Hi
> I do see that storm-core.jar contains both *.clj sources and clojure compiler 
> generated  *.class files. But in my observation in storm UI daemon, every 
> request triggers some clj compilation on the-flight.
> This leads to:
> - high latency. (it takes about 5 seconds to fully load storm UI index page)
> - under very modest load (10 concurrent requests) index page rendering fails 
> with exceptions for some requests:
> java.lang.RuntimeException: Unable to resolve symbol: nimbus in this context, 
> compiling:(backtype/storm/ui/core.clj:1038)
>   at clojure.lang.Compiler.analyze(Compiler.java:6281)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3548)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6457)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
>   at clojure.lang.Compiler$LetExpr$Parser.parse(Compiler.java:5919)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:5618)
>   at clojure.lang.Compiler$FnMethod.parse(Compiler.java:5054)
>   at clojure.lang.Compiler$FnExpr.parse(Compiler.java:3674)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6453)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3548)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6457)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6443)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:3548)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6457)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.access$100(Compiler.java:37)
>   at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:518)
>   at clojure.lang.Compiler.analyzeSeq(Compiler.java:6455)
>   at clojure.lang.Compiler.analyze(Compiler.java:6262)
>   at clojure.lang.Compiler.analyze(Compiler.java:6223)
>   at clojure.lang.Compiler.eval(Compiler.java:6515)
>   at clojure.lang.Compiler.load(Compiler.java:6952)
>   at clojure.lang.RT.loadResourceScript(RT.java:359)
>   at clojure.lang.RT.loadResourceScript(RT.java:350)
>   at clojure.lang.RT.load(RT.java:429)
>   at clojure.lang.RT.load(RT.java:400)
>   at clojure.core$load$fn__4890.invoke(core.clj:5415)
>   at clojure.core$load.doInvoke(core.clj:5414)
>   at clojure.lang.RestFn.invoke(RestFn.java:408)
>   at clojure.core$load_one.invoke(core.clj:5227)
>   at clojure.core$load_lib.doInvoke(core.clj:5264)
>   at clojure.lang.RestFn.applyTo(RestFn.java:142)
>   at clojure.core$apply.invoke(core.clj:603)
>   at clojure.core$load_libs.doInvoke(core.clj:5298)
>   at clojure.lang.RestFn.applyTo(RestFn.java:137)
>   at clojure.core$apply.invoke(core.clj:603)
>   at clojure.core$require.doInvoke(core.clj:5381)
>   at clojure.lang.RestFn.invoke(RestFn.java:421)
>   at ring.middleware.reload$wrap_reload$fn__7318.invoke(reload.clj:13)
>   at backtype.storm.ui.core$catch_errors$fn__2082.invoke(core.clj:1075)
>   at 
> ring.middleware.keyword_params$wrap_keyword_params$fn__6327.invoke(keyword_params.clj:27)
>   at 
> ring.middleware.nested_params$wrap_nested_params$fn__6364.invoke(nested_params.clj:65)
>   at 

[jira] [Updated] (STORM-274) Add support for command remoteconfvalue in storm.cmd

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-274:
---
Component/s: storm-core

> Add support for command remoteconfvalue in storm.cmd
> 
>
> Key: STORM-274
> URL: https://issues.apache.org/jira/browse/STORM-274
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: Suresh Srinivas
>Assignee: Suresh Srinivas
>  Labels: windows
> Fix For: 0.9.3
>
> Attachments: STORM-274.patch
>
>
> Currently storm.cmd does not support remoteconfvalue as supported by its 
> linux counterpart storm.py.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-262) Add capability to remove keys from MemoryMapState

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-262:
---
Component/s: storm-core

> Add capability to remove keys from MemoryMapState
> -
>
> Key: STORM-262
> URL: https://issues.apache.org/jira/browse/STORM-262
> Project: Apache Storm
>  Issue Type: Improvement
>  Components: storm-core
>Reporter: Nathan Marz
>Assignee: Nathan Marz
>
> https://github.com/apache/incubator-storm/pull/48



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-275) Topologies, Spouts and Bolts with special characters in name cannot be viewed in ui

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-275:
---
Component/s: storm-core

> Topologies, Spouts and Bolts with special characters in name cannot be viewed 
> in ui
> ---
>
> Key: STORM-275
> URL: https://issues.apache.org/jira/browse/STORM-275
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Fix For: 0.9.2-incubating
>
>
> If you run a topology with a spout or bolt that contains a whitespace in the 
> name (i.e. "Do some thing") then the URL towards the component page will look 
> something like this:
> /topology/TOPONAME-12-12345678/component/Do+some+thing
> The page that is opened on that link will try to show a component called
> Do+some+thing
> which it doesn't have.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-351) multilang python process fall into endless loop

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-351:
---
Component/s: storm-multilang

> multilang python process fall into endless loop
> ---
>
> Key: STORM-351
> URL: https://issues.apache.org/jira/browse/STORM-351
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-multilang
>Affects Versions: 0.9.3
> Environment: storm 0.9.3-incubating
>Reporter: DashengJu
>Assignee: DashengJu
>Priority: Blocker
> Fix For: 0.9.3
>
>
> 1. steps to reproduce
> 1)  write a topology with a python bolt, run the topology on storm; then 
> there will be two process for the bolt: the worker(java process for 
> ShellBolt), python process.
> 2）kill -9  the worker(java process for ShellBolt);
> 2. expected behavior
> the worker exit and the python process exist
> 3. actual, incorrect behavior
> the worker exit, but the python process never exist and fall into endless 
> loop
> 4. analyse
> in storm.py，read tuple from stdin with follow function:
> def readMsg():
> msg = ""
> while True:
> line = sys.stdin.readline()[0:-1]
> if line == "end":
> break
> msg = msg + line + "\n"
> return json_decode(msg[0:-1])
> when sys.stdin is closed, EOF is encountered, readline() return None, so 
> readMsg fall into endless loop.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-224) Storm should use stricter ACLs within zookeeper

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-224:
---
Component/s: storm-core

> Storm should use stricter ACLs within zookeeper
> ---
>
> Key: STORM-224
> URL: https://issues.apache.org/jira/browse/STORM-224
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Derek Dagit
> Fix For: 0.10.0
>
>
> In a stand alone environment storm stores everything wide open in ZK.  We 
> really should lock this down with ACLs so that individual topologies cannot 
> modify data that the storm system uses, and so that other topologies cannot 
> modify/interfere with each other.
> The current code from Yahoo will generate a random username/password for each 
> topology that is launched.  This works great for most topologies, but for 
> trident topologies because they store long lived data in ZK the user has to 
> keep the credentials around themselves.  We would love to switch ZK access 
> over to use a forwarded TGT, but have not finished the work to do this yet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-219) DRPC HTTP Support

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-219:
---
Component/s: storm-core

> DRPC HTTP Support
> -
>
> Key: STORM-219
> URL: https://issues.apache.org/jira/browse/STORM-219
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Andy Feng
> Fix For: 0.10.0
>
>
> Though this is not strictly needed for security this is part of the work in 
> the Yahoo multi-tenant patch.  It provides a RESTFUL HTTP API to DRPC.
> HTTP Authentication should be included with this too and should probably wait 
> until STORM-218 is in.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-218) HTTP Filters + HTTP Auth

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-218:
---
Component/s: storm-core

> HTTP Filters + HTTP Auth
> 
>
> Key: STORM-218
> URL: https://issues.apache.org/jira/browse/STORM-218
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Derek Dagit
> Fix For: 0.10.0
>
>
> HTTP needs a way to authenticate who a user is and limit access to topologies 
> to only those people who are allowed to see them.
> In the version Yahoo released we also added a config to optionally disable 
> the kill/Rebalance UI actions.  We can either do that or also add in 
> authentication checks on the restful APIs while we are at it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-229) Lock Down File Downloads

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-229:
---
Component/s: storm-core

> Lock Down File Downloads
> 
>
> Key: STORM-229
> URL: https://issues.apache.org/jira/browse/STORM-229
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Derek Dagit
> Fix For: 0.10.0
>
>
> The file upload API for storm will put files in a specific directory, but the 
> download API is fairly wide open and could potentially be used to download 
> more then it should.
> We should lock this down do that only files that have been uploaded can be 
> downloaded again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-217) Thrift Authorization

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-217:
---
Component/s: storm-core

> Thrift Authorization
> 
>
> Key: STORM-217
> URL: https://issues.apache.org/jira/browse/STORM-217
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Andy Feng
> Fix For: 0.10.0
>
>
> We currently have a framework for authentication, but no real way to reject 
> anyone from doing anything.  We should provide pluggable authorizer classes 
> for NIMBUS and DRPC that can enforce different security policies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-251) Command-Line config options with '=' in values are silently ignored

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-251:
---
Component/s: storm-core

> Command-Line config options with '=' in values are silently ignored
> ---
>
> Key: STORM-251
> URL: https://issues.apache.org/jira/browse/STORM-251
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Derek Dagit
>Priority: Minor
>  Labels: newbie
>
> As a user submitting a topology, I would like to override a topology 
> configuration with a value containing an = character, so that I can set keys 
> like 
> topology.worker.childopts="-Djava.security.auth.login.config=/path/to/file".
> See 
> https://github.com/apache/incubator-storm/blob/22ddd6e6d5c78e36610366e71ea879b283575e01/storm-core/src/jvm/backtype/storm/utils/Utils.java#L165



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-264) Remove deprecated topology.optimize config?

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-264:
---
Component/s: storm-core

> Remove deprecated topology.optimize config?
> ---
>
> Key: STORM-264
> URL: https://issues.apache.org/jira/browse/STORM-264
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Derek Dagit
>Assignee: Derek Dagit
>Priority: Minor
> Fix For: 0.9.2-incubating
>
>
> There continues to be some confusion around topology.optimize.  Do we want to 
> remove references to it?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-210) Add storm-hbase module

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-210:
---
Component/s: storm-hbase

> Add storm-hbase module
> --
>
> Key: STORM-210
> URL: https://issues.apache.org/jira/browse/STORM-210
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-hbase
>Reporter: P. Taylor Goetz
>Assignee: P. Taylor Goetz
> Fix For: 0.9.3
>
>
> There are a few projects out there that provide HBase integration, in various 
> states of maintenance.
> This is a placeholder for adding some type of HBase integration as a storm 
> component.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-227) allow worker.gc.childopts to be configured separately

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-227:
---
Component/s: storm-core

> allow worker.gc.childopts to be configured separately
> -
>
> Key: STORM-227
> URL: https://issues.apache.org/jira/browse/STORM-227
> Project: Apache Storm
>  Issue Type: Sub-task
>  Components: storm-core
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Fix For: 0.10.0
>
>
> For some JDKs gc options cannot be overridden by later options on the command 
> line.  They result in an error instead.  So if you want to provide default gc 
> options for end users, but give them the ability to override them, we need to 
> separate them out into a separate config that is handled differently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-260) Race condition in backtype.storm.utils.Time

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-260:
---
Component/s: storm-core

> Race condition in backtype.storm.utils.Time
> ---
>
> Key: STORM-260
> URL: https://issues.apache.org/jira/browse/STORM-260
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: Adrian Petrescu
>Assignee: Adrian Petrescu
>Priority: Minor
>  Labels: patch, test
> Fix For: 0.9.2-incubating
>
> Attachments: patch_commit_ae6a0eaf21e1.patch
>
>
> Some of my test runs were occasionally failing with a NullPointerException on 
> [backtype.storm.utils.Time.java:64|https://github.com/apache/incubator-storm/blob/master/storm-core/src/jvm/backtype/storm/utils/Time.java#L64].
> After a bit of investigation, it seems there's a race condition here; if we 
> disable simulating mode while a thread is currently sleeping, then when it 
> wakes up it won't re-check if it's still in "simulating" mode, it'll try to 
> remove the sleep time, and get the NPE.
> The attached patch is a fairly straightforward fix.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-276) Add support for logviewer in storm.cmd

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-276:
---
Component/s: storm-core

> Add support for logviewer in storm.cmd
> --
>
> Key: STORM-276
> URL: https://issues.apache.org/jira/browse/STORM-276
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>Priority: Minor
>  Labels: windows
> Fix For: 0.9.2-incubating
>
> Attachments: STORM-276.patch, STORM-276_1.PATCH
>
>
> storm.cmd doesn't have logviewer functionality as it is in storm.py



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-454) Storm Rest UI shows window in milliseconds but should be seconds

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-454:
---
Component/s: storm-core

> Storm Rest UI shows window in milliseconds but should be seconds
> 
>
> Key: STORM-454
> URL: https://issues.apache.org/jira/browse/STORM-454
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 0.9.2-incubating
>Reporter: Ashton Webster
>Priority: Trivial
> Fix For: 0.9.3
>
>
> In the STORM-UI-REST-API.md doc file, the parameter for "window" appears to 
> be in seconds, but the documentation says milliseconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-245) Implement localOrShuffle() on Stream for performance

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-245:
---
Component/s: storm-core

>  Implement localOrShuffle() on Stream for performance
> -
>
> Key: STORM-245
> URL: https://issues.apache.org/jira/browse/STORM-245
> Project: Apache Storm
>  Issue Type: New Feature
>  Components: storm-core
>Reporter: Rob Turner
>Assignee: P. Taylor Goetz
>Priority: Minor
>  Labels: performance, trident
> Fix For: 0.9.2-incubating
>
>
> Stream has a shuffle() method but does not have a localOrShuffle() method to 
> prevent cross worker serialization if possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-258) Upgrading Apache Commons IO to 2.x

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-258:
---
Component/s: storm-core

> Upgrading Apache Commons IO to 2.x
> --
>
> Key: STORM-258
> URL: https://issues.apache.org/jira/browse/STORM-258
> Project: Apache Storm
>  Issue Type: Dependency upgrade
>  Components: storm-core
>Reporter: Daniel Cruver
>Assignee: P. Taylor Goetz
>Priority: Minor
> Fix For: 0.9.2-incubating
>
>
> The version of commons-io is 1.4 in Storm is very old, it was release about 6 
> years ago.  This can causes issues when deploying applications to Storm that 
> are using the newer version of common-io.  For example Hadoop is currently 
> depending on commons-io-2.1.  We currently have issues running our 
> application on Storm and we currently are using commons-io 2.4.
> According to the release documentation:
> https://commons.apache.org/proper/commons-io/upgradeto2_0.html
> https://commons.apache.org/proper/commons-io/upgradeto2_2.html
> https://commons.apache.org/proper/commons-io/upgradeto2_4.html
> It only looks like their is some incompatibilities when upgrading to 2.4 but 
> these may not apply to Storm.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1070) Additional JIRA Components

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-1070:

Component/s: storm-core

> Additional JIRA Components
> --
>
> Key: STORM-1070
> URL: https://issues.apache.org/jira/browse/STORM-1070
> Project: Apache Storm
>  Issue Type: Task
>  Components: storm-core
>Reporter: Rick Kellogg
>Assignee: Rick Kellogg
>Priority: Minor
>
> Create additional JIRA components in Storm project:
> storm-core
> storm-multilang
> storm-nimbus
> storm-supervisor
> storm-drpc
> storm-elasticsearch
> storm-hive
> storm-jdbc
> storm-solr
> storm-examples (for storm-starter and others)
> storm-ui
> For consideration:
> documentation
> build
> website
> security
> Would also consider making component required when creating new bugs.  
> Default would be storm-core.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-188) Allow user to specify full configuration path when running storm command

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-188:
---
Component/s: storm-core

> Allow user to specify full configuration path when running storm command
> 
>
> Key: STORM-188
> URL: https://issues.apache.org/jira/browse/STORM-188
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Sean Zhong
>Assignee: Sriharsha Chintalapani
>Priority: Minor
> Fix For: 0.10.0
>
> Attachments: search_local_path_for_config.patch, storm-188.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Currently, storm will only look up configuration path in java classpath. We 
> should also allow user to specify full configuration path. This is very 
> important for a shared cluster environment, like YARN. Multiple storm cluster 
> may runs with different configuration, but share same binary folder.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-242) Rotated metrics.log.* should use ${storm.home} in cluster.xml

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-242:
---
Component/s: storm-core

> Rotated metrics.log.* should use ${storm.home} in cluster.xml
> -
>
> Key: STORM-242
> URL: https://issues.apache.org/jira/browse/STORM-242
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Tsz Ming
>Priority: Minor
>
> Current value in cluster.xml:
> metrics.log.%i
> Should be similar to others:
> ${storm.home}/logs/metrics.log.%i



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-285) Fix storm-core shade plugin config

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-285:
---
Component/s: storm-core

> Fix storm-core shade plugin config
> --
>
> Key: STORM-285
> URL: https://issues.apache.org/jira/browse/STORM-285
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: Bryan Baugher
>Assignee: Bryan Baugher
>Priority: Trivial
> Fix For: 0.9.2-incubating
>
>
> In the storm-core pom for the shade plugin config it has,
> {code}
> odg.apache.storm:*
> {code}
> which should likely be
> {code}
> org.apache.storm:*
> {code}
> instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-604) Storm Bug List

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg updated STORM-604:
---
Component/s: storm-core

> Storm Bug List
> --
>
> Key: STORM-604
> URL: https://issues.apache.org/jira/browse/STORM-604
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: TonyLee
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (STORM-754) Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java (default-cli) on project fyp-storm-try: The parameters 'mainClass' for goal org.codehaus.mojo:exec-m

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg reopened STORM-754:


> Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java 
> (default-cli) on project fyp-storm-try: The parameters 'mainClass' for goal 
> org.codehaus.mojo:exec-maven-plugin:1.2.1:java are missing or invalid -> 
> [Help 1]
> ---
>
> Key: STORM-754
> URL: https://issues.apache.org/jira/browse/STORM-754
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-core
>Affects Versions: 0.9.3
> Environment: Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-44-generic x86_64)
> at Amazon Web Services EC2
>Reporter: Yuen
>Assignee: Yuen
>  Labels: build, maven, newbie
> Fix For: 0.9.3
>
>
> ubuntu@ip-10-0-0-101:~/storm/examples/fyp-storm-try$ sudo mvn compile 
> exec:java  
> -D storm.topology=fyp.storm.try.Topology
> [INFO] Scanning for projects...
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.storm:fyp-storm-try:jar:0.9.4
> [WARNING] 'reporting.plugins.plugin.version' for 
> org.apache.maven.plugins:maven-javadoc-plugin is missing. @ 
> org.apache.storm:storm:0.9.4, 
> /root/.m2/repository/org/apache/storm/storm/0.9.4/storm-0.9.4.pom, line 694, 
> column 21
> [WARNING] 'reporting.plugins.plugin.version' for 
> org.apache.maven.plugins:maven-surefire-report-plugin is missing. @ 
> org.apache.storm:storm:0.9.4, 
> /root/.m2/repository/org/apache/storm/storm/0.9.4/storm-0.9.4.pom, line 660, 
> column 21
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> [WARNING] 
> [WARNING] For this reason, future Maven versions might no longer support 
> building such malformed projects.
> [WARNING] 
> [INFO]
>  
> [INFO] 
> 
> [INFO] Building fyp-storm-try 0.9.4
> [INFO] 
> 
> [INFO] 
> [INFO] --- maven-remote-resources-plugin:1.2.1:process (default) @ 
> fyp-storm-try ---
> [INFO] 
> [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
> fyp-storm-try ---
> [debug] execute contextualize
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory 
> /home/ubuntu/storm/examples/fyp-storm-try/multilang
> [INFO] Copying 3 resources
> [INFO] 
> [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
> fyp-storm-try ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 5 source files to 
> /home/ubuntu/storm/examples/fyp-storm-try/target/classes
> [INFO] 
> [INFO] --- clojure-maven-plugin:1.3.18:compile (compile) @ fyp-storm-try ---
> [INFO] 
> [INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) @ fyp-storm-try >>>
> [INFO] 
> [INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) @ fyp-storm-try <<<
> [INFO] 
> [INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ fyp-storm-try ---
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 10.364s
> [INFO] Finished at: Wed Apr 01 12:43:59 UTC 2015
> [INFO] Final Memory: 19M/51M
> [INFO] 
> 
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:java 
> (default-cli) on project fyp-storm-try: The parameters 'mainClass' for goal 
> org.codehaus.mojo:exec-maven-plugin:1.2.1:java are missing or invalid -> 
> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> [ERROR] For more information about the errors and possible solutions, please 
> read the following articles:
> [ERROR] [Help 1] 
> http://cwiki.apache.org/confluence/display/MAVEN/PluginParameterException



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (STORM-604) Storm Bug List

[Rick Kellogg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rick Kellogg reopened STORM-604:


> Storm Bug List
> --
>
> Key: STORM-604
> URL: https://issues.apache.org/jira/browse/STORM-604
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: TonyLee
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)