[jira] [Commented] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261488#comment-15261488
 ] 

ASF GitHub Bot commented on STORM-1736:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1376#issuecomment-215303147
  
The patch looks okay to me, but I would like to know why it's necessary. I 
can't reproduce the issuue from the branch or rc2 build.


> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>
> 1.x-branch failing with following error
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
> (default-testCompile) on project storm-kafka: Compilation failure
> [ERROR] 
> /Users/harsha/code/harshach/incubator-storm/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java:[70,16]
>  constructor KafkaConfig in class kafka.server.KafkaConfig cannot be applied 
> to given types;
> [ERROR] required: java.util.Map,boolean
> [ERROR] found: java.util.Properties
> [ERROR] reason: actual and formal argument lists differ in length
> [ERROR] -> [Help 1]
> [ERROR] 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1736. Change KafkaTestBroker.buildKafkaC...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1376#issuecomment-215303147
  
The patch looks okay to me, but I would like to know why it's necessary. I 
can't reproduce the issuue from the branch or rc2 build.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1740) Storm on mesos and yarn

[Renjie Liu (JIRA)]

Renjie Liu created STORM-1740:
-

 Summary: Storm on mesos and yarn
 Key: STORM-1740
 URL: https://issues.apache.org/jira/browse/STORM-1740
 Project: Apache Storm
  Issue Type: New Feature
  Components: storm-core
Reporter: Renjie Liu
Priority: Minor


Storm should delegate resource management to mesos or yarn rather than use it 
own schemanism.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261468#comment-15261468
 ] 

ASF GitHub Bot commented on STORM-1736:
---

GitHub user harshach opened a pull request:

https://github.com/apache/storm/pull/1376

STORM-1736. Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/harshach/incubator-storm STORM-1736

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1376


commit 68bdcc53d5ea88e6ee84bcea3a0aa5fcfe0979d5
Author: Sriharsha Chintalapani 
Date:   2016-04-28T02:50:28Z

STORM-1736. Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.




> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>
> 1.x-branch failing with following error
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
> (default-testCompile) on project storm-kafka: Compilation failure
> [ERROR] 
> /Users/harsha/code/harshach/incubator-storm/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java:[70,16]
>  constructor KafkaConfig in class kafka.server.KafkaConfig cannot be applied 
> to given types;
> [ERROR] required: java.util.Map,boolean
> [ERROR] found: java.util.Properties
> [ERROR] reason: actual and formal argument lists differ in length
> [ERROR] -> [Help 1]
> [ERROR] 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1736. Change KafkaTestBroker.buildKafkaC...

[harshach]

GitHub user harshach opened a pull request:

https://github.com/apache/storm/pull/1376

STORM-1736. Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/harshach/incubator-storm STORM-1736

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1376.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1376


commit 68bdcc53d5ea88e6ee84bcea3a0aa5fcfe0979d5
Author: Sriharsha Chintalapani 
Date:   2016-04-28T02:50:28Z

STORM-1736. Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-1737) storm-kaka-client has compilation errors with Apache Kafka 0.10

[Hugo Louro (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hugo Louro updated STORM-1737:
--
Summary: storm-kaka-client has compilation errors with Apache Kafka 0.10  
(was: storm-kaka-client has compilation errors with Apache Kafka 0.10 branch)

> storm-kaka-client has compilation errors with Apache Kafka 0.10
> ---
>
> Key: STORM-1737
> URL: https://issues.apache.org/jira/browse/STORM-1737
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Hugo Louro
>Priority: Critical
>
> when compiled with Apache Kafka 0.10 branch getting following errors
> {code}
> [ERROR] 
> /Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[163,51]
>  incompatible types: org.apache.kafka.common.TopicPartition cannot be 
> converted to java.util.Collection
> [ERROR] 
> /Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[166,45]
>  incompatible types: org.apache.kafka.common.TopicPartition cannot be 
> converted to java.util.Collection
> [ERROR] 
> /Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[175,51]
>  incompatible types: org.apache.kafka.common.TopicPartition cannot be 
> converted to java.util.Collection
> [ERROR] 
> /Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[177,45]
>  incompatible types: org.apache.kafka.common.TopicPartition cannot be 
> converted to java.util.Collection
> [ERROR] 
> /Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[252,41]
>  incompatible types: org.apache.kafka.common.TopicPartition cannot be 
> converted to java.util.Collection
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[P. Taylor Goetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261455#comment-15261455
 ] 

P. Taylor Goetz commented on STORM-1736:


{quote}
I hope this doesn't require reverse engineer the code to understand the intent 
of the JIRA.
{quote}

Of course it doesn't.

I just feel that JIRAs should be descriptive, and enable other developers to 
better understand what's being proposed.

> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>
> 1.x-branch failing with following error
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
> (default-testCompile) on project storm-kafka: Compilation failure
> [ERROR] 
> /Users/harsha/code/harshach/incubator-storm/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java:[70,16]
>  constructor KafkaConfig in class kafka.server.KafkaConfig cannot be applied 
> to given types;
> [ERROR] required: java.util.Map,boolean
> [ERROR] found: java.util.Properties
> [ERROR] reason: actual and formal argument lists differ in length
> [ERROR] -> [Help 1]
> [ERROR] 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: minor: update command line client documentatio...

[arinto]

GitHub user arinto opened a pull request:

https://github.com/apache/storm/pull/1375

minor: update command line client documentation

Fix broken link to setup dev env link and complete options for `storm 
rebalance` command for Storm 0.9.6

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arinto/storm minor-fix-documentation

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1375.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1375


commit e2149c2eb952f648a8f57e07ce7f16b446436851
Author: Arinto Murdopo 
Date:   2016-04-28T02:44:08Z

Fix broken link to setup dev env link

And complete options for `storm rebalance` command




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani updated STORM-1736:
--
Description: 
1.x-branch failing with following error

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
(default-testCompile) on project storm-kafka: Compilation failure
[ERROR] 
/Users/harsha/code/harshach/incubator-storm/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java:[70,16]
 constructor KafkaConfig in class kafka.server.KafkaConfig cannot be applied to 
given types;
[ERROR] required: java.util.Map,boolean
[ERROR] found: java.util.Properties
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] -> [Help 1]
[ERROR] 


> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>
> 1.x-branch failing with following error
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
> (default-testCompile) on project storm-kafka: Compilation failure
> [ERROR] 
> /Users/harsha/code/harshach/incubator-storm/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java:[70,16]
>  constructor KafkaConfig in class kafka.server.KafkaConfig cannot be applied 
> to given types;
> [ERROR] required: java.util.Map,boolean
> [ERROR] found: java.util.Properties
> [ERROR] reason: actual and formal argument lists differ in length
> [ERROR] -> [Help 1]
> [ERROR] 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[Sriharsha Chintalapani (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261428#comment-15261428
 ] 

Sriharsha Chintalapani commented on STORM-1736:
---

[~ptgoetz] updated the title to reflect the intended change. I hope this 
doesn't require reverse engineer the code to understand the intent of the JIRA.

> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1736) Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani updated STORM-1736:
--
Summary: Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.  
(was: Change KafkaTestBroker.)

> Change KafkaTestBroker.buildKafkaConfig to new KafkaConfig api.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1736) Change KafkaTestBroker.

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani updated STORM-1736:
--
Summary: Change KafkaTestBroker.  (was: Change storm-kafka KafkaTestBroker 
to newer api)

> Change KafkaTestBroker.
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[Sriharsha Chintalapani (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261422#comment-15261422
 ] 

Sriharsha Chintalapani commented on STORM-1736:
---

[~ptgoetz] I am not sure what context you are missing here .The title is good 
enough and I posted clear enough details in my comment above. Its a one-line 
fix which shouldn't require a JIRA let alone long explanation about the fix. If 
someone is not willing to look at the code they shouldn't worried about JIRA at 
all.

> Change storm-kafka KafkaTestBroker to newer api
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[P. Taylor Goetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261417#comment-15261417
 ] 

P. Taylor Goetz commented on STORM-1736:


[~harsha_ch] From a dev community perspective, we have no context about your 
intention here. Pointing to a line number makes us dig into code, essentially 
reverse engineer, and context switch.

Ideally a JIRA should have enough why/how information in the description to let 
others determine quickly whether or not the issue, bug, or patch is something 
they should be interested in. 

I'm sure I've been guilty of the same, but it is something we should strive to 
correct.

> Change storm-kafka KafkaTestBroker to newer api
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (STORM-1739) Mini JAVA version dependency is wrong in 0.10.0 and above.

[Carey Tzou (JIRA)]

Carey Tzou created STORM-1739:
-

 Summary: Mini JAVA version dependency is wrong in 0.10.0 and above.
 Key: STORM-1739
 URL: https://issues.apache.org/jira/browse/STORM-1739
 Project: Apache Storm
  Issue Type: Documentation
Reporter: Carey Tzou
Assignee: Carey Tzou
Priority: Minor


Storm drops supporting JDK 1.6 in Apache Storm 0.10.0 and above. need update 
the document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261397#comment-15261397
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215286456
  
@dsKarthick Seems like we don't have a plan to make a 0.9.x release. So the 
PR for 0.9.x-branch should be closed.


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 Flush stdout and stderr before call...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215286456
  
@dsKarthick Seems like we don't have a plan to make a 0.9.x release. So the 
PR for 0.9.x-branch should be closed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1738) Add Storm Alarm component (API and an implementation)

[Xin Wang (JIRA)]

Xin Wang created STORM-1738:
---

 Summary: Add Storm Alarm component (API and an implementation)
 Key: STORM-1738
 URL: https://issues.apache.org/jira/browse/STORM-1738
 Project: Apache Storm
  Issue Type: New Feature
Reporter: Xin Wang
Assignee: Xin Wang






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-143) Launching a process throws away standard out; can hang

[Erik Weathers (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erik Weathers resolved STORM-143.
-
   Resolution: Fixed
Fix Version/s: 0.10.0

> Launching a process throws away standard out; can hang
> --
>
> Key: STORM-143
> URL: https://issues.apache.org/jira/browse/STORM-143
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
> Fix For: 0.10.0
>
>
> https://github.com/nathanmarz/storm/issues/489
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/util.clj#L349
> When we launch a process, standard out is written to a system buffer and does 
> not appear to be read. Also, nothing is redirected to standard in. This can 
> have the following effects:
> A worker can hang when initializing (e.g. UnsatisfiedLinkError looking for 
> jzmq), and it will be unable to communicate the error as standard out is 
> being swallowed.
> A process that writes too much to standard out will block if the buffer fills
> A process that tries to read form standard in for any reason will block.
> Perhaps we can redirect standard out to an .out file, and redirect /dev/null 
> to the standard in stream of the process?
> --
> nathanmarz: Storm redirects stdout to the logging system. It's worked fine 
> for us in our topologies.
> --
> d2r: We see in worker.clj, in mk-worker, where there is a call to 
> redirect-stdio-to-slf4j!. This would not seem to help in cases such as we are 
> seeing when there is a problem launching the worker itself.
> (defn -main [storm-id assignment-id port-str worker-id]
>   (let [conf1 (read-storm-config)
> login_conf_file (System/getProperty "java.security.auth.login.config")
> conf (if login_conf_file (merge conf1 
> {"java.security.auth.login.config" login_conf_file}) conf1)]
> (validate-distributed-mode! conf)
> (mk-worker conf nil (java.net.URLDecoder/decode storm-id) assignment-id 
> (Integer/parseInt port-str) worker-id)))
> If anything were to go wrong (CLASSPATH, jvm opts, misconfiguration...) 
> before -main or before mk-worker, then any output would be lost. The symptom 
> we saw was that the topology sat around apparently doing nothing, yet there 
> was no log indicating that the workers were failing to start.
> Is there other redirection to logs that I'm missing?
> --
> xiaokang: we use bash to launch worker process and redirect its stdout to 
> woker-port.out file. it heleped us find the zeromq jni problem that cause the 
> jvm crash without any log.
> --
> nathanmarz: @d2r Yea, that's all I was referring to. If we redirect stdout, 
> will the code that redirects stdout to the logging system still take effect? 
> This is important because we can control the size of the logfiles (via the 
> logback config) but not the size of the redirected stdout file.
> --
> d2r: My hunch is that it will work as it does now, except that any messages 
> that are getting thrown away before that point would go to a file instead. I 
> can play with it and find out. We wouldn't want to change the redirection, 
> just restore visibility to any output that might occur prior to the 
> redirection. There should be some safety valve to control the size of any new 
> .out in case something goes berserk.
> @xiaokang I see how that would work. We also need to make sure redirection 
> continues to work as it currently does for the above reason.
> --
> xiaokang: @d2r @nathanmarz In out cluster, storm's stdout redirection still 
> works for any System.out output while JNI errors goes to worker-port.out 
> file. I think it will be nice to use the same worker-port.log file for bash 
> stdout redirection since logback can control log file size. But it is a 
> little bit ugly to use bash to launch worker java process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-143) Launching a process throws away standard out; can hang

[Erik Weathers (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261323#comment-15261323
 ] 

Erik Weathers commented on STORM-143:
-

Aha (hadn't clicked the {{...}} on the GitHub UI)!  I'll mark this ticket as 
closed then, thanks!

> Launching a process throws away standard out; can hang
> --
>
> Key: STORM-143
> URL: https://issues.apache.org/jira/browse/STORM-143
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/489
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/util.clj#L349
> When we launch a process, standard out is written to a system buffer and does 
> not appear to be read. Also, nothing is redirected to standard in. This can 
> have the following effects:
> A worker can hang when initializing (e.g. UnsatisfiedLinkError looking for 
> jzmq), and it will be unable to communicate the error as standard out is 
> being swallowed.
> A process that writes too much to standard out will block if the buffer fills
> A process that tries to read form standard in for any reason will block.
> Perhaps we can redirect standard out to an .out file, and redirect /dev/null 
> to the standard in stream of the process?
> --
> nathanmarz: Storm redirects stdout to the logging system. It's worked fine 
> for us in our topologies.
> --
> d2r: We see in worker.clj, in mk-worker, where there is a call to 
> redirect-stdio-to-slf4j!. This would not seem to help in cases such as we are 
> seeing when there is a problem launching the worker itself.
> (defn -main [storm-id assignment-id port-str worker-id]
>   (let [conf1 (read-storm-config)
> login_conf_file (System/getProperty "java.security.auth.login.config")
> conf (if login_conf_file (merge conf1 
> {"java.security.auth.login.config" login_conf_file}) conf1)]
> (validate-distributed-mode! conf)
> (mk-worker conf nil (java.net.URLDecoder/decode storm-id) assignment-id 
> (Integer/parseInt port-str) worker-id)))
> If anything were to go wrong (CLASSPATH, jvm opts, misconfiguration...) 
> before -main or before mk-worker, then any output would be lost. The symptom 
> we saw was that the topology sat around apparently doing nothing, yet there 
> was no log indicating that the workers were failing to start.
> Is there other redirection to logs that I'm missing?
> --
> xiaokang: we use bash to launch worker process and redirect its stdout to 
> woker-port.out file. it heleped us find the zeromq jni problem that cause the 
> jvm crash without any log.
> --
> nathanmarz: @d2r Yea, that's all I was referring to. If we redirect stdout, 
> will the code that redirects stdout to the logging system still take effect? 
> This is important because we can control the size of the logfiles (via the 
> logback config) but not the size of the redirected stdout file.
> --
> d2r: My hunch is that it will work as it does now, except that any messages 
> that are getting thrown away before that point would go to a file instead. I 
> can play with it and find out. We wouldn't want to change the redirection, 
> just restore visibility to any output that might occur prior to the 
> redirection. There should be some safety valve to control the size of any new 
> .out in case something goes berserk.
> @xiaokang I see how that would work. We also need to make sure redirection 
> continues to work as it currently does for the above reason.
> --
> xiaokang: @d2r @nathanmarz In out cluster, storm's stdout redirection still 
> works for any System.out output while JNI errors goes to worker-port.out 
> file. I think it will be nice to use the same worker-port.log file for bash 
> stdout redirection since logback can control log file size. But it is a 
> little bit ugly to use bash to launch worker java process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[Sriharsha Chintalapani (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261319#comment-15261319
 ] 

Sriharsha Chintalapani commented on STORM-1736:
---

[~ptgoetz] 
https://github.com/apache/storm/blob/1.x-branch/external/storm-kafka/src/test/org/apache/storm/kafka/KafkaTestBroker.java#L70
 
need to change above line to return KafkaConfig.fromProps(p)

> Change storm-kafka KafkaTestBroker to newer api
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261274#comment-15261274
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215270518
  
@vesense ported this PR on the versions you listed plus 0.9.x


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 Flush stdout and stderr before call...

[dsKarthick]

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215270518
  
@vesense ported this PR on the versions you listed plus 0.9.x


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1737) storm-kaka-client has compilation errors with Apache Kafka 0.10 branch

[Sriharsha Chintalapani (JIRA)]

Sriharsha Chintalapani created STORM-1737:
-

 Summary: storm-kaka-client has compilation errors with Apache 
Kafka 0.10 branch
 Key: STORM-1737
 URL: https://issues.apache.org/jira/browse/STORM-1737
 Project: Apache Storm
  Issue Type: Bug
Reporter: Sriharsha Chintalapani
Assignee: Hugo Louro
Priority: Critical


when compiled with Apache Kafka 0.10 branch getting following errors
{code}
[ERROR] 
/Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[163,51]
 incompatible types: org.apache.kafka.common.TopicPartition cannot be converted 
to java.util.Collection
[ERROR] 
/Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[166,45]
 incompatible types: org.apache.kafka.common.TopicPartition cannot be converted 
to java.util.Collection
[ERROR] 
/Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[175,51]
 incompatible types: org.apache.kafka.common.TopicPartition cannot be converted 
to java.util.Collection
[ERROR] 
/Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[177,45]
 incompatible types: org.apache.kafka.common.TopicPartition cannot be converted 
to java.util.Collection
[ERROR] 
/Users/harsha/code/hwx/storm/external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java:[252,41]
 incompatible types: org.apache.kafka.common.TopicPartition cannot be converted 
to java.util.Collection
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[P. Taylor Goetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261255#comment-15261255
 ] 

P. Taylor Goetz commented on STORM-1736:


Can you elaborate on your plan?

There's a recent pull request that may do something similar. I'll try to dig it 
up.

> Change storm-kafka KafkaTestBroker to newer api
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: Flush stdout and stderr before calling "os.exe...

[dsKarthick]

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1374

Flush stdout and stderr before calling "os.execvp" to prevent log loss.

Related to #1360

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.1.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1374.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1374


commit 7a7deaefec503e48131852b5e5da54b67e81b712
Author: Karthick Duraisamy Soundararaj 
Date:   2016-04-25T08:54:39Z

Flush stdout and stderr before calling "os.execvp" to prevent log loss.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261195#comment-15261195
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215264278
  
@satishd @roshannaik 
Yeah, I guess we should know about the whole things affecting performance 
as Storm claims very high performance, but we don't, and there's small 
materials to refer for Clojure.
That's another point of why we should move outside of Clojure or even we 
should implement critical path to Java or at least other well-known and popular 
JVM languages (I mean Scala).


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 0.10.1, 1.0.1
>
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (1.x) Avoid looking up debug / back...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215264278
  
@satishd @roshannaik 
Yeah, I guess we should know about the whole things affecting performance 
as Storm claims very high performance, but we don't, and there's small 
materials to refer for Clojure.
That's another point of why we should move outside of Clojure or even we 
should implement critical path to Java or at least other well-known and popular 
JVM languages (I mean Scala).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261192#comment-15261192
 ] 

ASF GitHub Bot commented on STORM-1733:
---

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1373

STORM-1733 (0.10.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss

Related to #1360

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.10.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1373


commit 01c90265cbbcff8bd5629d2c643090c442312024
Author: Karthick Duraisamy Soundararaj 
Date:   2016-04-25T08:54:39Z

Flush stdout and stderr before calling "os.execvp" to prevent log loss.




> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 (0.10.x) Flush stdout and stderr be...

[dsKarthick]

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1373

STORM-1733 (0.10.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss

Related to #1360

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.10.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1373.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1373


commit 01c90265cbbcff8bd5629d2c643090c442312024
Author: Karthick Duraisamy Soundararaj 
Date:   2016-04-25T08:54:39Z

Flush stdout and stderr before calling "os.execvp" to prevent log loss.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[Erik Weathers (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261166#comment-15261166
 ] 

Erik Weathers commented on STORM-1733:
--

[gigantic auto-comment 
above|https://issues.apache.org/jira/browse/STORM-1733?focusedCommentId=15261156=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15261156]
 is an example of why I want to disable the automatic uploading of all GitHub 
stuff into JIRA.

> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261168#comment-15261168
 ] 

ASF GitHub Bot commented on STORM-1733:
---

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1372

STORM-1733 (0.9.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss.

Related to https://github.com/apache/storm/pull/1360/#issuecomment-215073019

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.9.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1372


commit 150d959aac646eac8267fb102e5d434cca4b3093
Author: Karthick Duraisamy Soundararaj 
Date:   2016-04-27T23:12:16Z

Flush stdout and stderr before calling os.execvp to prevent log loss.




> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 (0.9.x) Flush stdout and stderr bef...

[dsKarthick]

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1372

STORM-1733 (0.9.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss.

Related to https://github.com/apache/storm/pull/1360/#issuecomment-215073019

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.9.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1372


commit 150d959aac646eac8267fb102e5d434cca4b3093
Author: Karthick Duraisamy Soundararaj 
Date:   2016-04-27T23:12:16Z

Flush stdout and stderr before calling os.execvp to prevent log loss.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261160#comment-15261160
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1371#issuecomment-215259583
  
Invalid. Please ignore. 


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261161#comment-15261161
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user dsKarthick closed the pull request at:

https://github.com/apache/storm/pull/1371


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 (0.9.x) Flush stdout and stderr bef...

[dsKarthick]

Github user dsKarthick closed the pull request at:

https://github.com/apache/storm/pull/1371


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1733 (0.9.x) Flush stdout and stderr bef...

[dsKarthick]

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1371#issuecomment-215259583
  
Invalid. Please ignore. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1733 (0.9.x) Flush stdout and stderr bef...

[dsKarthick]

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1371

STORM-1733 (0.9.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss.

Please see https://github.com/apache/storm/pull/1360

@arunmahadevan @vesense

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.9.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1371.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1371


commit 98f4e619d54052b73a309d23ab7214953e4c7774
Author: Sriharsha Chintalapani 
Date:   2015-02-05T18:08:05Z

STORM-130: Supervisor getting killed due to java.io.FileNotFoundException: 
File '../stormconf.ser' does not exist.

Signed-off-by: P. Taylor Goetz 

commit a1e5893e1b94c224d39fedf11583b216c21351c8
Author: P. Taylor Goetz 
Date:   2015-02-24T20:46:12Z

update changelog for STORM-130

commit 62788f295bb1fb1cc83b99c30f82beb40eea5f25
Author: P. Taylor Goetz 
Date:   2015-02-24T23:03:40Z

port STORM-329 fix to 0.9.x

commit 81016c2ed7222da99138bc9971e335533d4cb518
Author: Michael G. Noll 
Date:   2015-02-16T09:01:27Z

Track how many messages are being dropped when a connection is unavailable

Signed-off-by: P. Taylor Goetz 

commit 97a76fc896de508f015dbe32f1473ddbf10d736b
Author: Michael G. Noll 
Date:   2015-02-16T09:03:07Z

Clarify name of method for dropping messages

Signed-off-by: P. Taylor Goetz 

commit 9138d9fc255639b4d0d43657379ce467591e8ef2
Author: Michael G. Noll 
Date:   2015-02-16T09:07:35Z

Change log level for intentionally dropping messages from WARN to ERROR

This change makes the log level for dropping messages consistent in
Client.java.

Signed-off-by: P. Taylor Goetz 

commit 6b06d8468ff5e743fb12b85dd84fe0931041c2c3
Author: P. Taylor Goetz 
Date:   2015-02-24T23:18:43Z

add STORM-329 to changelog

commit e63fb2af9086e2b2e688662ca42a4b4d0112274b
Author: Parth Brahmbhatt 
Date:   2015-03-03T00:06:58Z

STORM-693: when bolt fails to write tuple, it should report error instead 
of silently acking.

Signed-off-by: P. Taylor Goetz 

commit 92836de540ec8ab90d7591b96ba02126e80b5c3a
Author: P. Taylor Goetz 
Date:   2015-03-18T14:59:56Z

add STORM-693 to changelog

commit c19e482b70f18d690ad165c78551860506486095
Author: Parth Brahmbhatt 
Date:   2015-02-20T19:56:22Z

STORM-682: supervisor should handle worker state corruption gracefully.

Signed-off-by: P. Taylor Goetz 

commit f0de11a20fe2f20dc1dc2f485549e0dc342f8680
Author: P. Taylor Goetz 
Date:   2015-03-18T15:05:30Z

add STORM-682 to changelog

commit 835a410c879dc1eb02d9670410f65fe0be6f28c6
Author: Parth Brahmbhatt 
Date:   2015-01-14T20:27:35Z

STORM-559: ZkHosts in README should use 2181 as port.

Signed-off-by: P. Taylor Goetz 

commit 30e0be8616c89cb1f8a51fcf462f76a075e6e964
Author: P. Taylor Goetz 
Date:   2015-03-18T15:11:16Z

add STORM-559 to changelog

commit b1bbacb7134d17ff47c2e8b8857a66244a4d1d4f
Author: P. Taylor Goetz 
Date:   2015-03-18T15:28:11Z

add missing import in supervisor.clj

commit edf596bac8feab0c8721f7de94474e5549858355
Author: P. Taylor Goetz 
Date:   2015-03-18T16:21:38Z

[maven-release-plugin] prepare release v0.9.4

commit 48d10e20eb3c750fc41fcf0bef3d49501cf6d5a4
Author: P. Taylor Goetz 
Date:   2015-03-18T16:21:45Z

[maven-release-plugin] prepare for next development iteration

commit 41f44f9914d4f27d0db3f211a85f88301533f09b
Author: P. Taylor Goetz 
Date:   2015-03-18T17:59:39Z

Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 48d10e20eb3c750fc41fcf0bef3d49501cf6d5a4.

commit 233603c3cbd729fdfabd2759bfa7705811996aa4
Author: P. Taylor Goetz 
Date:   2015-03-18T18:00:11Z

Revert "[maven-release-plugin] prepare release v0.9.4"

This reverts commit edf596bac8feab0c8721f7de94474e5549858355.

commit 61e1b5c3e226122143c91bbd7527b605f8ed8727
Author: P. Taylor Goetz 
Date:   2015-03-18T18:05:51Z

add Apache license header to ConnectionWithStatus.java

commit 00091d7952681a39281aa171adfad133a5e26330
Author: P. Taylor Goetz 
Date:   2015-03-18T18:13:57Z

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261156#comment-15261156
 ] 

ASF GitHub Bot commented on STORM-1733:
---

GitHub user dsKarthick opened a pull request:

https://github.com/apache/storm/pull/1371

STORM-1733 (0.9.x) Flush stdout and stderr before calling "os.execvp" to 
prevent log loss.

Please see https://github.com/apache/storm/pull/1360

@arunmahadevan @vesense

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dsKarthick/storm-1 
flush_launch_command_to_stderr_0.9.x-branch

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1371.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1371


commit 98f4e619d54052b73a309d23ab7214953e4c7774
Author: Sriharsha Chintalapani 
Date:   2015-02-05T18:08:05Z

STORM-130: Supervisor getting killed due to java.io.FileNotFoundException: 
File '../stormconf.ser' does not exist.

Signed-off-by: P. Taylor Goetz 

commit a1e5893e1b94c224d39fedf11583b216c21351c8
Author: P. Taylor Goetz 
Date:   2015-02-24T20:46:12Z

update changelog for STORM-130

commit 62788f295bb1fb1cc83b99c30f82beb40eea5f25
Author: P. Taylor Goetz 
Date:   2015-02-24T23:03:40Z

port STORM-329 fix to 0.9.x

commit 81016c2ed7222da99138bc9971e335533d4cb518
Author: Michael G. Noll 
Date:   2015-02-16T09:01:27Z

Track how many messages are being dropped when a connection is unavailable

Signed-off-by: P. Taylor Goetz 

commit 97a76fc896de508f015dbe32f1473ddbf10d736b
Author: Michael G. Noll 
Date:   2015-02-16T09:03:07Z

Clarify name of method for dropping messages

Signed-off-by: P. Taylor Goetz 

commit 9138d9fc255639b4d0d43657379ce467591e8ef2
Author: Michael G. Noll 
Date:   2015-02-16T09:07:35Z

Change log level for intentionally dropping messages from WARN to ERROR

This change makes the log level for dropping messages consistent in
Client.java.

Signed-off-by: P. Taylor Goetz 

commit 6b06d8468ff5e743fb12b85dd84fe0931041c2c3
Author: P. Taylor Goetz 
Date:   2015-02-24T23:18:43Z

add STORM-329 to changelog

commit e63fb2af9086e2b2e688662ca42a4b4d0112274b
Author: Parth Brahmbhatt 
Date:   2015-03-03T00:06:58Z

STORM-693: when bolt fails to write tuple, it should report error instead 
of silently acking.

Signed-off-by: P. Taylor Goetz 

commit 92836de540ec8ab90d7591b96ba02126e80b5c3a
Author: P. Taylor Goetz 
Date:   2015-03-18T14:59:56Z

add STORM-693 to changelog

commit c19e482b70f18d690ad165c78551860506486095
Author: Parth Brahmbhatt 
Date:   2015-02-20T19:56:22Z

STORM-682: supervisor should handle worker state corruption gracefully.

Signed-off-by: P. Taylor Goetz 

commit f0de11a20fe2f20dc1dc2f485549e0dc342f8680
Author: P. Taylor Goetz 
Date:   2015-03-18T15:05:30Z

add STORM-682 to changelog

commit 835a410c879dc1eb02d9670410f65fe0be6f28c6
Author: Parth Brahmbhatt 
Date:   2015-01-14T20:27:35Z

STORM-559: ZkHosts in README should use 2181 as port.

Signed-off-by: P. Taylor Goetz 

commit 30e0be8616c89cb1f8a51fcf462f76a075e6e964
Author: P. Taylor Goetz 
Date:   2015-03-18T15:11:16Z

add STORM-559 to changelog

commit b1bbacb7134d17ff47c2e8b8857a66244a4d1d4f
Author: P. Taylor Goetz 
Date:   2015-03-18T15:28:11Z

add missing import in supervisor.clj

commit edf596bac8feab0c8721f7de94474e5549858355
Author: P. Taylor Goetz 
Date:   2015-03-18T16:21:38Z

[maven-release-plugin] prepare release v0.9.4

commit 48d10e20eb3c750fc41fcf0bef3d49501cf6d5a4
Author: P. Taylor Goetz 
Date:   2015-03-18T16:21:45Z

[maven-release-plugin] prepare for next development iteration

commit 41f44f9914d4f27d0db3f211a85f88301533f09b
Author: P. Taylor Goetz 
Date:   2015-03-18T17:59:39Z

Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 48d10e20eb3c750fc41fcf0bef3d49501cf6d5a4.

commit 233603c3cbd729fdfabd2759bfa7705811996aa4
Author: P. Taylor Goetz 
Date:   2015-03-18T18:00:11Z

Revert "[maven-release-plugin] prepare release v0.9.4"

This reverts commit edf596bac8feab0c8721f7de94474e5549858355.

commit 61e1b5c3e226122143c91bbd7527b605f8ed8727
Author: P. Taylor Goetz 

[jira] [Assigned] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani reassigned STORM-1736:
-

Assignee: Sriharsha Chintalapani

> Change storm-kafka KafkaTestBroker to newer api
> ---
>
> Key: STORM-1736
> URL: https://issues.apache.org/jira/browse/STORM-1736
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Sriharsha Chintalapani
>Assignee: Sriharsha Chintalapani
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (STORM-1736) Change storm-kafka KafkaTestBroker to newer api

[Sriharsha Chintalapani (JIRA)]

Sriharsha Chintalapani created STORM-1736:
-

 Summary: Change storm-kafka KafkaTestBroker to newer api
 Key: STORM-1736
 URL: https://issues.apache.org/jira/browse/STORM-1736
 Project: Apache Storm
  Issue Type: Bug
Reporter: Sriharsha Chintalapani






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1707) Improve supervisor latency by removing 2-min wait

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260896#comment-15260896
 ] 

ASF GitHub Bot commented on STORM-1707:
---

GitHub user ppoulosk opened a pull request:

https://github.com/apache/storm/pull/1370

[STORM-1707]  Remove two minute timeout after worker launch

This is a logic change the removes the two minute wait after worker launch 
in sync processes.  Instead of sitting in a wait loop for all of the workers to 
come up, SyncProcessEvent now checks to see if a worker is in the NOT-STARTED 
state, and has elapsed the Config.SUPERVISOR_WORKER_START_TIMEOUT_SECS time.  
If it has, it kills and cleans up the worker. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppoulosk/storm STORM-1707

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1370.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1370


commit 8c8f2126446b932c51cce66dd93ffcfb7e16fc8e
Author: Paul Poulosky 
Date:   2016-04-27T20:41:38Z

Remove two minute timeout after worker launch




> Improve supervisor latency by removing 2-min wait
> -
>
> Key: STORM-1707
> URL: https://issues.apache.org/jira/browse/STORM-1707
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Paul Poulosky
>Assignee: Paul Poulosky
>
> After launching workers, the supervisor waits up to 2 minutes synchronously 
> for the workers to be "launched".
> We should remove this, and instead keep track of launch time, making the 
> "killer" function smart enough to determine the difference between a worker 
> that's still launching, one that's timed out, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1707] Remove two minute timeout after w...

[ppoulosk]

GitHub user ppoulosk opened a pull request:

https://github.com/apache/storm/pull/1370

[STORM-1707]  Remove two minute timeout after worker launch

This is a logic change the removes the two minute wait after worker launch 
in sync processes.  Instead of sitting in a wait loop for all of the workers to 
come up, SyncProcessEvent now checks to see if a worker is in the NOT-STARTED 
state, and has elapsed the Config.SUPERVISOR_WORKER_START_TIMEOUT_SECS time.  
If it has, it kills and cleans up the worker. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ppoulosk/storm STORM-1707

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1370.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1370


commit 8c8f2126446b932c51cce66dd93ffcfb7e16fc8e
Author: Paul Poulosky 
Date:   2016-04-27T20:41:38Z

Remove two minute timeout after worker launch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260871#comment-15260871
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user roshannaik commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215217692
  
@satishd i had seen a clojure blog where the author believed clojure maps 
are very efficient (and even faster than the Java map) for read-only operations 
.. but didnt provide any numbers substantiate it.  IMO it does not appear to be 
an accurate claim and Java HashMap would hold up much better (relatively) even 
in critical paths.. those clojure lookups sometimes lead to reflection.


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 0.10.1, 1.0.1
>
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[VOTE] Release Apache Storm 1.0.1 (rc2)

[P. Taylor Goetz]

***Please note the version! There is another VOTE open that could easily be 
confused with this one.***

This is a call to vote on releasing Apache Storm 1.0.1 (rc2)

Full list of changes in this release:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=34ca28a4dbab794c8e9adf7ceb1ee7c145f92a76

The tag/commit to be voted upon is v1.0.1:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=25fa9704664de860bdaacf223fbc8215f2b802eb;hb=34ca28a4dbab794c8e9adf7ceb1ee7c145f92a76

The source archive being voted upon can be found here:

https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/apache-storm-1.0.1-src.tar.gz

Other release files, signatures and digests can be found here:

https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/

The release artifacts are signed with the following key:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd

The Nexus staging repository for this release is:

https://repository.apache.org/content/repositories/orgapachestorm-1032/

Please vote on releasing this package as Apache Storm 1.0.1.

When voting, please list the actions taken to verify the release.

This vote will be open for at least 72 hours.

[ ] +1 Release this package as Apache Storm 1.0.1
[ ]  0 No opinion
[ ] -1 Do not release this package because...

Thanks to everyone who contributed to this release.

-Taylor


signature.asc
Description: Message signed with OpenPGP using GPGMail

[jira] [Commented] (STORM-1729) Get rid of reflections while recording stats

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260691#comment-15260691
 ] 

ASF GitHub Bot commented on STORM-1729:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1361#issuecomment-215184340
  
+1 (forgot to do so before merging)


> Get rid of reflections while recording stats
> 
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 1.0.1
>
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
>  - call to method record can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
>  - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but 
> it doesn't work.
> (I don't know why it doesn't work. Does generic type matter?)
> Anyway, defining them into let makes them avoid reflection.
> Even though we sample while recording stats, it does hurt performance with 
> fast & high-traffic components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-1729) Get rid of reflections while recording stats

[P. Taylor Goetz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

P. Taylor Goetz resolved STORM-1729.

   Resolution: Fixed
Fix Version/s: 1.0.1

Merged to 1.x-branch.

> Get rid of reflections while recording stats
> 
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 1.0.1
>
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
>  - call to method record can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
>  - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but 
> it doesn't work.
> (I don't know why it doesn't work. Does generic type matter?)
> Anyway, defining them into let makes them avoid reflection.
> Even though we sample while recording stats, it does hurt performance with 
> fast & high-traffic components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1729 (1.x) Get rid of reflections while ...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1361#issuecomment-215184340
  
+1 (forgot to do so before merging)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260686#comment-15260686
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1362


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 0.10.1, 1.0.1
>
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[P. Taylor Goetz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

P. Taylor Goetz resolved STORM-1731.

   Resolution: Fixed
Fix Version/s: 1.0.1
   0.10.1

Thanks [~kabhwan]. Merged to 1.x-branch and 0.10.x-branch.

> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
> Fix For: 0.10.1, 1.0.1
>
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (1.x) Avoid looking up debug / back...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1362


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1729) Get rid of reflections while recording stats

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260685#comment-15260685
 ] 

ASF GitHub Bot commented on STORM-1729:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1361


> Get rid of reflections while recording stats
> 
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
>  - call to method incBy can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
>  - call to method record can't be resolved (target class is unknown).
> Reflection warning, 
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
>  - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but 
> it doesn't work.
> (I don't know why it doesn't work. Does generic type matter?)
> Anyway, defining them into let makes them avoid reflection.
> Even though we sample while recording stats, it does hurt performance with 
> fast & high-traffic components.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1729 (1.x) Get rid of reflections while ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1361


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[VOTE] Release Apache Storm 0.10.1 (rc2)

[P. Taylor Goetz]

This is a call to vote on releasing Apache Storm 0.10.1 (rc2)

Full list of changes in this release:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=8179921a569b6cf1d97798eed8e7b03b131bc495

The tag/commit to be voted upon is v0.10.1:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=ddf051149f3c386342937efdabdaf45694602dd1;hb=8179921a569b6cf1d97798eed8e7b03b131bc495;a=tree;h=26291835f22474506f0fe90b0459eab0d00bf4a9;hb=f0d3eae7395b3ee036b94b922707f74868ba6190

The source archive being voted upon can be found here:

https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.1-rc2/apache-storm-0.10.1-src.tar.gz

Other release files, signatures and digests can be found here:

https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.1-rc2/

The release artifacts are signed with the following key:

https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd

The Nexus staging repository for this release is:

https://repository.apache.org/content/repositories/orgapachestorm-1031/

Please vote on releasing this package as Apache Storm 0.10.1.

When voting, please list the actions taken to verify the release.

This vote will be open for at least 72 hours.

[ ] +1 Release this package as Apache Storm 0.10.1
[ ]  0 No opinion
[ ] -1 Do not release this package because...

Thanks to everyone who contributed to this release.

-Taylor


signature.asc
Description: Message signed with OpenPGP using GPGMail

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260602#comment-15260602
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1364


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (0.10.x) Avoid looking up debug / b...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/1364


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260502#comment-15260502
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215150674
  
+1


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (1.x) Avoid looking up debug / back...

[ptgoetz]

Github user ptgoetz commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215150674
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[CANCELED] [VOTE] Release Apache Storm 0.10.1 (rc1)

[P. Taylor Goetz]

Canceling in order to include the proposed performance fixes.

-Taylor

> On Apr 26, 2016, at 3:52 PM, P. Taylor Goetz  wrote:
> 
> ***Please note the version! There is another VOTE open that could easily be 
> confused with this one.***
> 
> This is a call to vote on releasing Apache Storm 0.10.1 (rc1)
> 
> Full list of changes in this release:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=f0d3eae7395b3ee036b94b922707f74868ba6190
> 
> The tag/commit to be voted upon is v0.10.1:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=26291835f22474506f0fe90b0459eab0d00bf4a9;hb=f0d3eae7395b3ee036b94b922707f74868ba6190
> 
> The source archive being voted upon can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.1-rc1/apache-storm-0.10.1-src.tar.gz
> 
> Other release files, signatures and digests can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-0.10.1-rc1/
> 
> The release artifacts are signed with the following key:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> 
> The Nexus staging repository for this release is:
> 
> https://repository.apache.org/content/repositories/orgapachestorm-1029/
> 
> Please vote on releasing this package as Apache Storm 0.10.1.
> 
> When voting, please list the actions taken to verify the release.
> 
> This vote will be open for at least 72 hours.
> 
> [ ] +1 Release this package as Apache Storm 0.10.1
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
> 
> Thanks to everyone who contributed to this release.
> 
> -Taylor



signature.asc
Description: Message signed with OpenPGP using GPGMail

[CANCELED] [VOTE] Release Apache Storm 1.0.1

[P. Taylor Goetz]

Canceling this RC in order to include some performance enhancements.

-Taylor

> On Apr 26, 2016, at 12:23 PM, P. Taylor Goetz  wrote:
> 
> This is a call to vote on releasing Apache Storm 1.0.1 (rc1)
> 
> Full list of changes in this release:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=CHANGELOG.md;hb=7520b9ae13c4a4fcd823e2d636e9b2cc1cbb7e6f
> 
> The tag/commit to be voted upon is v1.0.1:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=9cc5c11896dd83402f6a6836d63d50564a94c932;hb=7520b9ae13c4a4fcd823e2d636e9b2cc1cbb7e6f
> The source archive being voted upon can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/apache-storm-1.0.1-src.tar.gz
> 
> Other release files, signatures and digests can be found here:
> 
> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/
> 
> The release artifacts are signed with the following key:
> 
> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> 
> The Nexus staging repository for this release is:
> 
> https://repository.apache.org/content/repositories/orgapachestorm-1029/
> 
> Please vote on releasing this package as Apache Storm 1.0.1.
> 
> When voting, please list the actions taken to verify the release.
> 
> This vote will be open for at least 72 hours.
> 
> [ ] +1 Release this package as Apache Storm 1.0.1
> [ ]  0 No opinion
> [ ] -1 Do not release this package because...
> 
> Thanks to everyone who contributed to this release.
> 
> -Taylor



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: 答复: [VOTE] Release Apache Storm 1.0.1

[Xin Wang]

also +1 on doing another release candidate with Jungtaek's fixes in.

2016-04-27 10:57 GMT+08:00 John Fang :

> Thank you for Jungtaek, I also hope we can do another candidate with these
> fixed in.
>
> -邮件原件-
> 发件人: Harsha [mailto:st...@harsha.io]
> 发送时间: 2016年4月27日 10:20
> 收件人: dev@storm.apache.org
> 主题: Re: [VOTE] Release Apache Storm 1.0.1
>
> I am +1 on doing another release candidate with these fixes in. Nice work
> Jungtaek.
>
> Thanks,
> Harsha
>
> On Tue, Apr 26, 2016, at 05:23 PM, Jungtaek Lim wrote:
> > I've observed the points of performance hit (STORM-1729
> > , STORM-1731
> > ), and it seems huge.
> >
> > Since patches are small and straightforward, I would love to include
> > this as 1.0.1 so that users maintaining high-traffic topologies can
> > enjoy the benefit.
> > But it's only when performance hit / recovery is observed with stable
> > environment. I provided some test reports there, but I want to see the
> > test reports from other environments.
> >
> > Thanks,
> > Jungtaek Lim
> >
> > 2016년 4월 27일 (수) 오전 4:46, P. Taylor Goetz 님이 작성:
> >
> > > Two of the links were wrong, please use the following links:
> > >
> > > The source archive being voted upon can be found here:
> > >
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc1/
> > > apache-storm-1.0.1-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc1/
> > >
> > > Sorry for the inconvenience.
> > >
> > > -Taylor
> > >
> > > On Apr 26, 2016, at 12:23 PM, P. Taylor Goetz 
> wrote:
> > >
> > > This is a call to vote on releasing Apache Storm 1.0.1 (rc1)
> > >
> > > Full list of changes in this release:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=C
> > > HANGELOG.md;hb=7520b9ae13c4a4fcd823e2d636e9b2cc1cbb7e6f
> > >
> > > The tag/commit to be voted upon is v1.0.1:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h=9cc5c11
> > > 896dd83402f6a6836d63d50564a94c932;hb=7520b9ae13c4a4fcd823e2d636e9b2c
> > > c1cbb7e6f The source archive being voted upon can be found here:
> > >
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/
> > > apache-storm-1.0.1-src.tar.gz
> > >
> > > Other release files, signatures and digests can be found here:
> > >
> > > https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.1-rc2/
> > >
> > > The release artifacts are signed with the following key:
> > >
> > >
> > > https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_plain;f=K
> > > EYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd
> > >
> > > The Nexus staging repository for this release is:
> > >
> > > https://repository.apache.org/content/repositories/orgapachestorm-10
> > > 29/
> > >
> > > Please vote on releasing this package as Apache Storm 1.0.1.
> > >
> > > When voting, please list the actions taken to verify the release.
> > >
> > > This vote will be open for at least 72 hours.
> > >
> > > [ ] +1 Release this package as Apache Storm 1.0.1 [ ]  0 No opinion
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks to everyone who contributed to this release.
> > >
> > > -Taylor
> > >
> > >
> > >
>
>

[jira] [Commented] (STORM-1676) NullPointerException while serializing ClusterWorkerHearbeat

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260180#comment-15260180
 ] 

ASF GitHub Bot commented on STORM-1676:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1298#issuecomment-215089647
  
LGTM


> NullPointerException while serializing ClusterWorkerHearbeat
> 
>
> Key: STORM-1676
> URL: https://issues.apache.org/jira/browse/STORM-1676
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 2.0.0
>Reporter: Abhishek Agarwal
>Assignee: Abhishek Agarwal
> Fix For: 2.0.0
>
>
> `Map executor_stats` had null value in the key 
> which was causing NPE during serialization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1676] Filter null executor stats from w...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1298#issuecomment-215089647
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1646) Intermittent test failures in storm-kafka unit tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260174#comment-15260174
 ] 

ASF GitHub Bot commented on STORM-1646:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1335#issuecomment-215088822
  
LGTM


> Intermittent test failures in storm-kafka unit tests
> 
>
> Key: STORM-1646
> URL: https://issues.apache.org/jira/browse/STORM-1646
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Paul Poulosky
>Assignee: Paul Poulosky
>
> We have been seeing intermittent test failures in KafkaUtilsTest on slow 
> hardware and lightly resourced VMs, as well as an intermittent race condition 
> when running the storm-kafka ExponentialBackoffManager  unit test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-1646] Fix Kafka unit tests

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1335#issuecomment-215088822
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-143) Launching a process throws away standard out; can hang

[Robert Joseph Evans (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260170#comment-15260170
 ] 

Robert Joseph Evans commented on STORM-143:
---

If you know the file that was changed, or some code that was changed as a part 
of it, both blame and history can be big helps in getting the traceability.  We 
don't squash commits though so it can sometimes not give you what you want.  
But in this case it works out OK, but you may need to expand the commit comment 
to find the JIRA number

https://github.com/apache/storm/commits/0.10.x-branch/storm-core/src/jvm/backtype/storm/LogWriter.java

STORM-833

and the pull request for it shows LogWriter being added.

https://github.com/apache/storm/pull/562/files



> Launching a process throws away standard out; can hang
> --
>
> Key: STORM-143
> URL: https://issues.apache.org/jira/browse/STORM-143
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/489
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/util.clj#L349
> When we launch a process, standard out is written to a system buffer and does 
> not appear to be read. Also, nothing is redirected to standard in. This can 
> have the following effects:
> A worker can hang when initializing (e.g. UnsatisfiedLinkError looking for 
> jzmq), and it will be unable to communicate the error as standard out is 
> being swallowed.
> A process that writes too much to standard out will block if the buffer fills
> A process that tries to read form standard in for any reason will block.
> Perhaps we can redirect standard out to an .out file, and redirect /dev/null 
> to the standard in stream of the process?
> --
> nathanmarz: Storm redirects stdout to the logging system. It's worked fine 
> for us in our topologies.
> --
> d2r: We see in worker.clj, in mk-worker, where there is a call to 
> redirect-stdio-to-slf4j!. This would not seem to help in cases such as we are 
> seeing when there is a problem launching the worker itself.
> (defn -main [storm-id assignment-id port-str worker-id]
>   (let [conf1 (read-storm-config)
> login_conf_file (System/getProperty "java.security.auth.login.config")
> conf (if login_conf_file (merge conf1 
> {"java.security.auth.login.config" login_conf_file}) conf1)]
> (validate-distributed-mode! conf)
> (mk-worker conf nil (java.net.URLDecoder/decode storm-id) assignment-id 
> (Integer/parseInt port-str) worker-id)))
> If anything were to go wrong (CLASSPATH, jvm opts, misconfiguration...) 
> before -main or before mk-worker, then any output would be lost. The symptom 
> we saw was that the topology sat around apparently doing nothing, yet there 
> was no log indicating that the workers were failing to start.
> Is there other redirection to logs that I'm missing?
> --
> xiaokang: we use bash to launch worker process and redirect its stdout to 
> woker-port.out file. it heleped us find the zeromq jni problem that cause the 
> jvm crash without any log.
> --
> nathanmarz: @d2r Yea, that's all I was referring to. If we redirect stdout, 
> will the code that redirects stdout to the logging system still take effect? 
> This is important because we can control the size of the logfiles (via the 
> logback config) but not the size of the redirected stdout file.
> --
> d2r: My hunch is that it will work as it does now, except that any messages 
> that are getting thrown away before that point would go to a file instead. I 
> can play with it and find out. We wouldn't want to change the redirection, 
> just restore visibility to any output that might occur prior to the 
> redirection. There should be some safety valve to control the size of any new 
> .out in case something goes berserk.
> @xiaokang I see how that would work. We also need to make sure redirection 
> continues to work as it currently does for the above reason.
> --
> xiaokang: @d2r @nathanmarz In out cluster, storm's stdout redirection still 
> works for any System.out output while JNI errors goes to worker-port.out 
> file. I think it will be nice to use the same worker-port.log file for bash 
> stdout redirection since logback can control log file size. But it is a 
> little bit ugly to use bash to launch worker java process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: update the java version.

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1366#issuecomment-215087638
  
@ziyunhx  the change looks good.
Could you also raise a jira (at 
https://issues.apache.org/jira/browse/STORM) and update the PR title ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: minor: improved documenation layout

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1363#issuecomment-215085997
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: update the java version.

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1366#issuecomment-215085760
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260148#comment-15260148
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1364#issuecomment-215084480
  
+1 Great Finding.


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260147#comment-15260147
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215084353
  
+1 Great Finding.


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (1.x) Avoid looking up debug / back...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215084353
  
+1 Great Finding.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-1731 (0.10.x) Avoid looking up debug / b...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1364#issuecomment-215084480
  
+1 Great Finding.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260080#comment-15260080
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215073019
  
LGTM
@dsKarthick Could you address this to 0.10.x-branch and 1.x-branch, too?


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 Flush stdout and stderr before call...

[vesense]

Github user vesense commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-215073019
  
LGTM
@dsKarthick Could you address this to 0.10.x-branch and 1.x-branch, too?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-1732) Resources are deleted when worker dies

[gareth smith (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

gareth smith updated STORM-1732:

Description: 
*Lets say a worker has been started by the supervisor*

2016-04-26 16:11:48.716 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x10bac1e4 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
supervisor 477ae22e-1a2b-4ea3-afd5-cb969f25e732 on port 6700 with id 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:11:48.727 [o.a.s.d.supervisor] INFO: Launching worker with 
command: 'C:\LightningDeployment\Java\bin\java' '-cp'  
2016-04-26 16:11:48.910 [o.a.s.config] INFO: SET worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 LIGHTNINGVM14$


*note this bit is is new for storm 1.0.0*


2016-04-26 16:11:49.405 [o.a.s.d.supervisor] INFO: Creating symlinks for 
worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
Lightning-1-1461683473 to its port artifacts directory
2016-04-26 16:11:50.251 [o.a.s.d.supervisor] INFO: Creating symlinks for 
worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
Lightning-1-1461683473 for files(1): ("resources")



*When a worker dies we correctly see some clean up and a new worker started...*



2016-04-26 16:15:35.520 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 exited with code: 20
2016-04-26 16:15:39.674 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.675 [o.a.s.d.supervisor] INFO: Shutting down and clearing 
state for id a5d51626-6e9f-4614-9ebb-a6263c140ca2. Current supervisor time: 
1461683739. State: :timed-out, Heartbeat: {:time-secs 1461683734, :storm-id 
"Lightning-1-1461683473", :executors [[12 12] [54 54] [42 42] [24 24] [18 18] 
[6 6] [48 48] [30 30] [-1 -1] [36 36]], :port 6700}
2016-04-26 16:15:39.676 [o.a.s.d.supervisor] INFO: Shutting down 
477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:39.676 [o.a.s.config] INFO: GET worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:39.677 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.681 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.857 [o.a.s.util] INFO: Error when trying to kill 1352. 
Process is probably already dead.
2016-04-26 16:15:39.955 [o.a.s.util] INFO: Error when trying to kill 2372. 
Process is probably already dead.
2016-04-26 16:15:40.009 [o.a.s.util] INFO: Error when trying to kill 4932. 
Process is probably already dead.
2016-04-26 16:15:40.009 [o.a.s.d.supervisor] INFO: Sleep 10 seconds for 
execution of cleanup threads on worker.
2016-04-26 16:15:49.677 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:49.679 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:50.056 [o.a.s.util] INFO: Error when trying to kill 1352. 
Process is probably already dead.
2016-04-26 16:15:50.119 [o.a.s.util] INFO: Error when trying to kill 2372. 
Process is probably already dead.
2016-04-26 16:15:50.175 [o.a.s.util] INFO: Error when trying to kill 4932. 
Process is probably already dead.
2016-04-26 16:15:50.257 [o.a.s.config] INFO: REMOVE worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Shut down 
477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x20e1ad4f 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
supervisor 477ae22e-1a2b-4ea3-afd5-cb969f25e732 on port 6700 with id 
e413447b-c9ca-417d-8e55-e10dd0edc6a4


*When the worker has been cleaned up, it seems the folders that the symlinks 
are pointing to are also cleaned (this maybe a windows only problem)*

*This is bad as it deletes the contents of the "resources" directory and hence 
any multilang stuff that was in those directories*

*also I think STORM-876 introduced this problem*

  was:
*Lets say a worker has been started by the supervisor*

2016-04-26 16:11:48.716 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x10bac1e4 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, 

[jira] [Updated] (STORM-1732) Resources are deleted when worker dies

[gareth smith (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

gareth smith updated STORM-1732:

Attachment: potentalFix.patch

I've added what I think would be a potential fix
I've tested this locally and it fixes the bug
note The deleteRecursiveIfExists function (that tests for symlinks and doesn't 
recurse through them) may have an equivalent in some other library...

> Resources are deleted when worker dies
> --
>
> Key: STORM-1732
> URL: https://issues.apache.org/jira/browse/STORM-1732
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
> Environment: Windows
>Reporter: gareth smith
>Priority: Critical
> Attachments: potentalFix.patch
>
>
> *Lets say a worker has been started by the supervisor*
> 2016-04-26 16:11:48.716 [o.a.s.d.supervisor] INFO: Launching worker with 
> assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] 
> [42 42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
> #object[org.apache.storm.generated.WorkerResources 0x10bac1e4 
> "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
> supervisor 477ae22e-1a2b-4ea3-afd5-cb969f25e732 on port 6700 with id 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2
> 2016-04-26 16:11:48.727 [o.a.s.d.supervisor] INFO: Launching worker with 
> command: 'C:\LightningDeployment\Java\bin\java' '-cp'  
> 2016-04-26 16:11:48.910 [o.a.s.config] INFO: SET worker-user 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 LIGHTNINGVM14$
> *note this bit is is new for storm 1.0.0*
> 2016-04-26 16:11:49.405 [o.a.s.d.supervisor] INFO: Creating symlinks for 
> worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
> Lightning-1-1461683473 to its port artifacts directory
> 2016-04-26 16:11:50.251 [o.a.s.d.supervisor] INFO: Creating symlinks for 
> worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
> Lightning-1-1461683473 for files(1): ("resources")
> *When a worker dies we correctly see some clean up and a new worker 
> started...*
> 2016-04-26 16:15:35.520 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 exited with code: 20
> 2016-04-26 16:15:39.674 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
> 2016-04-26 16:15:39.675 [o.a.s.d.supervisor] INFO: Shutting down and clearing 
> state for id a5d51626-6e9f-4614-9ebb-a6263c140ca2. Current supervisor time: 
> 1461683739. State: :timed-out, Heartbeat: {:time-secs 1461683734, :storm-id 
> "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 42] [24 24] [18 18] 
> [6 6] [48 48] [30 30] [-1 -1] [36 36]], :port 6700}
> 2016-04-26 16:15:39.676 [o.a.s.d.supervisor] INFO: Shutting down 
> 477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
> 2016-04-26 16:15:39.676 [o.a.s.config] INFO: GET worker-user 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2
> 2016-04-26 16:15:39.677 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
> 2016-04-26 16:15:39.681 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
> 2016-04-26 16:15:39.857 [o.a.s.util] INFO: Error when trying to kill 1352. 
> Process is probably already dead.
> 2016-04-26 16:15:39.955 [o.a.s.util] INFO: Error when trying to kill 2372. 
> Process is probably already dead.
> 2016-04-26 16:15:40.009 [o.a.s.util] INFO: Error when trying to kill 4932. 
> Process is probably already dead.
> 2016-04-26 16:15:40.009 [o.a.s.d.supervisor] INFO: Sleep 10 seconds for 
> execution of cleanup threads on worker.
> 2016-04-26 16:15:49.677 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
> 2016-04-26 16:15:49.679 [o.a.s.d.supervisor] INFO: Worker Process 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
> 2016-04-26 16:15:50.056 [o.a.s.util] INFO: Error when trying to kill 1352. 
> Process is probably already dead.
> 2016-04-26 16:15:50.119 [o.a.s.util] INFO: Error when trying to kill 2372. 
> Process is probably already dead.
> 2016-04-26 16:15:50.175 [o.a.s.util] INFO: Error when trying to kill 4932. 
> Process is probably already dead.
> 2016-04-26 16:15:50.257 [o.a.s.config] INFO: REMOVE worker-user 
> a5d51626-6e9f-4614-9ebb-a6263c140ca2
> 2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Shut down 
> 477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
> 2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Launching worker with 
> assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] 
> [42 42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
> #object[org.apache.storm.generated.WorkerResources 0x20e1ad4f 
> "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
> 

[jira] [Commented] (STORM-1730) LocalCluster#shutdown() does not terminate all storm threads/thread pools.

[Fergus Byrne (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260037#comment-15260037
 ] 

Fergus Byrne commented on STORM-1730:
-

It seems like it would be safe to change the thread pool executor to create 
daemon threads as this class also contains a timer which is created as daemon.

> LocalCluster#shutdown() does not terminate all storm threads/thread pools.
> --
>
> Key: STORM-1730
> URL: https://issues.apache.org/jira/browse/STORM-1730
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
> Environment: Windows 7 x64
> Oracle Java 1.8.0 u92 x64
>Reporter: Fergus Byrne
>Priority: Minor
> Attachments: Thread Pool '47' remaining..png, storm-shutdown-issue.zip
>
>
> When using the LocalCluster in test setup.  LocalCluster#shutdown() does not 
> shutdown all executor services it starts.  In my test case, there is a single 
> thread pool executor service that is not shutdown and not daemon.  This keeps 
> the jvm alive when it is expected to terminate.
> Please see attached test case.  In my example, thread pool 47 is not 
> shutdown.  Naming here is conditional on threading.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1730) LocalCluster#shutdown() does not terminate all storm threads/thread pools.

[Fergus Byrne (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260036#comment-15260036
 ] 

Fergus Byrne commented on STORM-1730:
-

Hi, 

I have tracked down the offending thread pool to the field *'_exec'* of 
*'org.apache.storm.utils.DisruptorQueue.FlusherPool'*.

The thread pool executor is not named, it is also not daemon, it is also never 
shutdown.

Kind regards,
Fergus

> LocalCluster#shutdown() does not terminate all storm threads/thread pools.
> --
>
> Key: STORM-1730
> URL: https://issues.apache.org/jira/browse/STORM-1730
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
> Environment: Windows 7 x64
> Oracle Java 1.8.0 u92 x64
>Reporter: Fergus Byrne
>Priority: Minor
> Attachments: Thread Pool '47' remaining..png, storm-shutdown-issue.zip
>
>
> When using the LocalCluster in test setup.  LocalCluster#shutdown() does not 
> shutdown all executor services it starts.  In my test case, there is a single 
> thread pool executor service that is not shutdown and not daemon.  This keeps 
> the jvm alive when it is expected to terminate.
> Please see attached test case.  In my example, thread pool 47 is not 
> shutdown.  Naming here is conditional on threading.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1732) Resources are deleted when worker dies

[gareth smith (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

gareth smith updated STORM-1732:

Description: 
*Lets say a worker has been started by the supervisor*

2016-04-26 16:11:48.716 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x10bac1e4 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
supervisor 477ae22e-1a2b-4ea3-afd5-cb969f25e732 on port 6700 with id 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:11:48.727 [o.a.s.d.supervisor] INFO: Launching worker with 
command: 'C:\LightningDeployment\Java\bin\java' '-cp'  
2016-04-26 16:11:48.910 [o.a.s.config] INFO: SET worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 LIGHTNINGVM14$


*note this bit is is new for storm 1.0.0*


2016-04-26 16:11:49.405 [o.a.s.d.supervisor] INFO: Creating symlinks for 
worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
Lightning-1-1461683473 to its port artifacts directory
2016-04-26 16:11:50.251 [o.a.s.d.supervisor] INFO: Creating symlinks for 
worker-id: a5d51626-6e9f-4614-9ebb-a6263c140ca2 storm-id: 
Lightning-1-1461683473 for files(1): ("resources")



*When a worker dies we correctly see some clean up and a new worker started...*



2016-04-26 16:15:35.520 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 exited with code: 20
2016-04-26 16:15:39.674 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.675 [o.a.s.d.supervisor] INFO: Shutting down and clearing 
state for id a5d51626-6e9f-4614-9ebb-a6263c140ca2. Current supervisor time: 
1461683739. State: :timed-out, Heartbeat: {:time-secs 1461683734, :storm-id 
"Lightning-1-1461683473", :executors [[12 12] [54 54] [42 42] [24 24] [18 18] 
[6 6] [48 48] [30 30] [-1 -1] [36 36]], :port 6700}
2016-04-26 16:15:39.676 [o.a.s.d.supervisor] INFO: Shutting down 
477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:39.676 [o.a.s.config] INFO: GET worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:39.677 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.681 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:39.857 [o.a.s.util] INFO: Error when trying to kill 1352. 
Process is probably already dead.
2016-04-26 16:15:39.955 [o.a.s.util] INFO: Error when trying to kill 2372. 
Process is probably already dead.
2016-04-26 16:15:40.009 [o.a.s.util] INFO: Error when trying to kill 4932. 
Process is probably already dead.
2016-04-26 16:15:40.009 [o.a.s.d.supervisor] INFO: Sleep 10 seconds for 
execution of cleanup threads on worker.
2016-04-26 16:15:49.677 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:49.679 [o.a.s.d.supervisor] INFO: Worker Process 
a5d51626-6e9f-4614-9ebb-a6263c140ca2 has died!
2016-04-26 16:15:50.056 [o.a.s.util] INFO: Error when trying to kill 1352. 
Process is probably already dead.
2016-04-26 16:15:50.119 [o.a.s.util] INFO: Error when trying to kill 2372. 
Process is probably already dead.
2016-04-26 16:15:50.175 [o.a.s.util] INFO: Error when trying to kill 4932. 
Process is probably already dead.
2016-04-26 16:15:50.257 [o.a.s.config] INFO: REMOVE worker-user 
a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Shut down 
477ae22e-1a2b-4ea3-afd5-cb969f25e732:a5d51626-6e9f-4614-9ebb-a6263c140ca2
2016-04-26 16:15:50.257 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x20e1ad4f 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this 
supervisor 477ae22e-1a2b-4ea3-afd5-cb969f25e732 on port 6700 with id 
e413447b-c9ca-417d-8e55-e10dd0edc6a4


*When the worker has been cleaned up, it seems the folders that the symlinks 
are pointing to are also cleaned (this maybe a windows only problem)*

*This is bad as it deletes the contents of the "resources" directory and hence 
any multilang stuff that was in those directories*

*also I think STORM-876 indroduced this problem*

  was:
*Lets say a worker has been started by the supervisor*

2016-04-26 16:11:48.716 [o.a.s.d.supervisor] INFO: Launching worker with 
assignment {:storm-id "Lightning-1-1461683473", :executors [[12 12] [54 54] [42 
42] [24 24] [18 18] [6 6] [48 48] [30 30] [36 36]], :resources 
#object[org.apache.storm.generated.WorkerResources 0x10bac1e4 
"WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, 

[GitHub] storm pull request: STORM-1735: Nimbus should log that replication...

[srdo]

GitHub user srdo opened a pull request:

https://github.com/apache/storm/pull/1369

STORM-1735: Nimbus should log that replication succeeded when min rep…

…licas was reached exactly

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/srdo/storm STORM-1735

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/1369.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1369


commit 4fdb5e1c7e83ccb95feabffbbf05b52f584cfce2
Author: Stig Rohde Døssing 
Date:   2016-04-27T10:03:43Z

STORM-1735: Nimbus should log that replication succeeded when min replicas 
was reached exactly




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-1735) Nimbus logs that replication was not reached when min-replication-count was reached exactly

[JIRA]

Stig Rohde Døssing created STORM-1735:
-

 Summary: Nimbus logs that replication was not reached when 
min-replication-count was reached exactly
 Key: STORM-1735
 URL: https://issues.apache.org/jira/browse/STORM-1735
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 1.0.0, 2.0.0
Reporter: Stig Rohde Døssing
Assignee: Stig Rohde Døssing
Priority: Minor


When waiting for replication during topology submission, Nimbus logs whether 
replication succeeded within the timeout or not. The check for whether to log 
that it timed out or succeeded is off by one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1731) Avoid looking up debug / backpressure enable flags within critical path

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259828#comment-15259828
 ] 

ASF GitHub Bot commented on STORM-1731:
---

Github user satishd commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215020195
  
+1. Nice to remove unnecessary lookups in critical paths.


[worker.clj#L768](https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/daemon/worker.clj#L768)
HashMap instance is received in 
[config.clj#L76](https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/config.clj#L76)
 from `Utils/readStormConfig` and converts that into clojure's 
`PersistentHashMap` while `clojurify-structure` is done. This map is merged 
later with other configs. This is all done when a worker is initialized. 
Clojure's `PersistentHashMap` is based on _Hash Array Mapped Trie_ which can 
have performance issues in critical paths for adding elements but lookups may 
not cost much more than java's `HashMap` which uses an array with RB tree.

 


> Avoid looking up debug / backpressure enable flags within critical path
> ---
>
> Key: STORM-1731
> URL: https://issues.apache.org/jira/browse/STORM-1731
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>Priority: Critical
>
> While profiling the result of STORM-1729, I also found that there're many 
> places in critical path which look up the value of flags which are not 
> updated dynamically.
> ("get from map" is on top 5 from each spout / bolt thread.)
> This should be fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1731 (1.x) Avoid looking up debug / back...

[satishd]

Github user satishd commented on the pull request:

https://github.com/apache/storm/pull/1362#issuecomment-215020195
  
+1. Nice to remove unnecessary lookups in critical paths.


[worker.clj#L768](https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/daemon/worker.clj#L768)
HashMap instance is received in 
[config.clj#L76](https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/config.clj#L76)
 from `Utils/readStormConfig` and converts that into clojure's 
`PersistentHashMap` while `clojurify-structure` is done. This map is merged 
later with other configs. This is all done when a worker is initialized. 
Clojure's `PersistentHashMap` is based on _Hash Array Mapped Trie_ which can 
have performance issues in critical paths for adding elements but lookups may 
not cost much more than java's `HashMap` which uses an array with RB tree.

 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-143) Launching a process throws away standard out; can hang

[Erik Weathers (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259806#comment-15259806
 ] 

Erik Weathers commented on STORM-143:
-

[~revans2] : seems this issue is fixed with the LogWriter that was introduced 
in storm-0.10.0.  I cannot find a ticket for that feature to link this against 
though.

> Launching a process throws away standard out; can hang
> --
>
> Key: STORM-143
> URL: https://issues.apache.org/jira/browse/STORM-143
> Project: Apache Storm
>  Issue Type: Bug
>  Components: storm-core
>Reporter: James Xu
>Priority: Minor
>
> https://github.com/nathanmarz/storm/issues/489
> https://github.com/nathanmarz/storm/blob/master/src/clj/backtype/storm/util.clj#L349
> When we launch a process, standard out is written to a system buffer and does 
> not appear to be read. Also, nothing is redirected to standard in. This can 
> have the following effects:
> A worker can hang when initializing (e.g. UnsatisfiedLinkError looking for 
> jzmq), and it will be unable to communicate the error as standard out is 
> being swallowed.
> A process that writes too much to standard out will block if the buffer fills
> A process that tries to read form standard in for any reason will block.
> Perhaps we can redirect standard out to an .out file, and redirect /dev/null 
> to the standard in stream of the process?
> --
> nathanmarz: Storm redirects stdout to the logging system. It's worked fine 
> for us in our topologies.
> --
> d2r: We see in worker.clj, in mk-worker, where there is a call to 
> redirect-stdio-to-slf4j!. This would not seem to help in cases such as we are 
> seeing when there is a problem launching the worker itself.
> (defn -main [storm-id assignment-id port-str worker-id]
>   (let [conf1 (read-storm-config)
> login_conf_file (System/getProperty "java.security.auth.login.config")
> conf (if login_conf_file (merge conf1 
> {"java.security.auth.login.config" login_conf_file}) conf1)]
> (validate-distributed-mode! conf)
> (mk-worker conf nil (java.net.URLDecoder/decode storm-id) assignment-id 
> (Integer/parseInt port-str) worker-id)))
> If anything were to go wrong (CLASSPATH, jvm opts, misconfiguration...) 
> before -main or before mk-worker, then any output would be lost. The symptom 
> we saw was that the topology sat around apparently doing nothing, yet there 
> was no log indicating that the workers were failing to start.
> Is there other redirection to logs that I'm missing?
> --
> xiaokang: we use bash to launch worker process and redirect its stdout to 
> woker-port.out file. it heleped us find the zeromq jni problem that cause the 
> jvm crash without any log.
> --
> nathanmarz: @d2r Yea, that's all I was referring to. If we redirect stdout, 
> will the code that redirects stdout to the logging system still take effect? 
> This is important because we can control the size of the logfiles (via the 
> logback config) but not the size of the redirected stdout file.
> --
> d2r: My hunch is that it will work as it does now, except that any messages 
> that are getting thrown away before that point would go to a file instead. I 
> can play with it and find out. We wouldn't want to change the redirection, 
> just restore visibility to any output that might occur prior to the 
> redirection. There should be some safety valve to control the size of any new 
> .out in case something goes berserk.
> @xiaokang I see how that would work. We also need to make sure redirection 
> continues to work as it currently does for the above reason.
> --
> xiaokang: @d2r @nathanmarz In out cluster, storm's stdout redirection still 
> works for any System.out output while JNI errors goes to worker-port.out 
> file. I think it will be nice to use the same worker-port.log file for bash 
> stdout redirection since logback can control log file size. But it is a 
> little bit ugly to use bash to launch worker java process.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1734) ClassNotFound error when running storm-starter topologies in local mode

[Pavel Smirnov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Smirnov updated STORM-1734:
-
Issue Type: Bug  (was: Question)

> ClassNotFound error when running storm-starter topologies in local mode
> ---
>
> Key: STORM-1734
> URL: https://issues.apache.org/jira/browse/STORM-1734
> Project: Apache Storm
>  Issue Type: Bug
>  Components: examples, storm-core
>Affects Versions: 1.0.0
>Reporter: Pavel Smirnov
>
> The problem is described here: 
> http://stackoverflow.com/questions/36649346/classnotfound-error-when-running-storm-starter-topologies-in-local-mode
> Please help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1734) ClassNotFound error when running storm-starter topologies in local mode

[Pavel Smirnov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Smirnov updated STORM-1734:
-
Priority: Major  (was: Trivial)

> ClassNotFound error when running storm-starter topologies in local mode
> ---
>
> Key: STORM-1734
> URL: https://issues.apache.org/jira/browse/STORM-1734
> Project: Apache Storm
>  Issue Type: Question
>  Components: examples, storm-core
>Affects Versions: 1.0.0
>Reporter: Pavel Smirnov
>
> The problem is described here: 
> http://stackoverflow.com/questions/36649346/classnotfound-error-when-running-storm-starter-topologies-in-local-mode
> Please help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259734#comment-15259734
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-214998893
  
+1


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 Flush stdout and stderr before call...

[arunmahadevan]

Github user arunmahadevan commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-214998893
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-1734) ClassNotFound error when running storm-starter topologies in local mode

[Pavel Smirnov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Smirnov updated STORM-1734:
-
   Priority: Trivial  (was: Minor)
Component/s: examples

> ClassNotFound error when running storm-starter topologies in local mode
> ---
>
> Key: STORM-1734
> URL: https://issues.apache.org/jira/browse/STORM-1734
> Project: Apache Storm
>  Issue Type: Question
>  Components: examples, storm-core
>Affects Versions: 1.0.0
>Reporter: Pavel Smirnov
>Priority: Trivial
>
> The problem is described here: 
> http://stackoverflow.com/questions/36649346/classnotfound-error-when-running-storm-starter-topologies-in-local-mode
> Please help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (STORM-1734) ClassNotFound error when running storm-starter topologies in local mode

[Pavel Smirnov (JIRA)]

Pavel Smirnov created STORM-1734:


 Summary: ClassNotFound error when running storm-starter topologies 
in local mode
 Key: STORM-1734
 URL: https://issues.apache.org/jira/browse/STORM-1734
 Project: Apache Storm
  Issue Type: Question
  Components: storm-core
Affects Versions: 1.0.0
Reporter: Pavel Smirnov


The problem is described here: 
http://stackoverflow.com/questions/36649346/classnotfound-error-when-running-storm-starter-topologies-in-local-mode

Please help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-1734) ClassNotFound error when running storm-starter topologies in local mode

[Pavel Smirnov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Smirnov updated STORM-1734:
-
Priority: Minor  (was: Major)

> ClassNotFound error when running storm-starter topologies in local mode
> ---
>
> Key: STORM-1734
> URL: https://issues.apache.org/jira/browse/STORM-1734
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Pavel Smirnov
>Priority: Minor
>
> The problem is described here: 
> http://stackoverflow.com/questions/36649346/classnotfound-error-when-running-storm-starter-topologies-in-local-mode
> Please help!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-1733) Logs from bin/storm are lost because stdout and stderr are not flushed

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15259658#comment-15259658
 ] 

ASF GitHub Bot commented on STORM-1733:
---

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-214988139
  
> Can you update the PR title to include the JIRA, i.e prefix it with 
[STORM-1733]
Done.


> Logs from bin/storm are lost because stdout and stderr are not flushed
> --
>
> Key: STORM-1733
> URL: https://issues.apache.org/jira/browse/STORM-1733
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3, 0.10.0, 0.9.4, 0.9.5, 0.9.6
>Reporter: Karthick Duraisamy Soundararaj
>Assignee: Karthick Duraisamy Soundararaj
>
> bin/storm.py emits the following crucial information that is lost because we 
> don't flush the stdout before exec.
> {code}
> 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= 
> -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log 
> -Dlogback.configurationFile=logback/cluster.xml  backtype.storm.ui.core.nimbus
> {code}
> Observed Environment:
> {code}
> OS: CentOS release 6.5 
> Kernel: 2.6.32-431.el6.x86_64
> Python version: Python 2.7.2
> {code}
> For example, I using runit to start storm components like nimbus, ui, etc and 
> the problem is applicable to all the components and in all the cases, I am 
> not seeing logs that are emitted by bin/storm before {{os.execvp}} is called 
> to actually launch the component. 
> Please note that in cases where stdout and stderr is terminal, the stdout and 
> stderr are always flushed and the bug is not applicable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-1733 Flush stdout and stderr before call...

[dsKarthick]

Github user dsKarthick commented on the pull request:

https://github.com/apache/storm/pull/1360#issuecomment-214988139
  
> Can you update the PR title to include the JIRA, i.e prefix it with 
[STORM-1733]
Done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---