[jira] [Commented] (STORM-378) SleepSpoutWaitStrategy.emptyEmit should use the variable "streak"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197697#comment-14197697
 ] 

ASF GitHub Bot commented on STORM-378:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61767078
  
To make it clear, PR points to "increase/decrease" sleep time with streak.
Fixed sleep time is configurable so it doesn't matter how long it is, 
optimal value should be vary for workload.

I think we can make new ISpoutWaitStrategy implementation that play with 
streak if we really need it.
We definitely need wait strategy to always sleep same time (ex. 1ms), so it 
isn't a good idea to change existing class's behavior.


> SleepSpoutWaitStrategy.emptyEmit should use  the variable "streak"
> --
>
> Key: STORM-378
> URL: https://issues.apache.org/jira/browse/STORM-378
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: caofangkun
>Priority: Minor
>
> {code:java}
> Index: src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java
> ===
> --- src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (revision 2868)
> +++ src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (working copy)
> @@ -18,6 +18,8 @@
>  package backtype.storm.spout;
>  
>  import backtype.storm.Config;
> +import backtype.storm.utils.Utils;
> +
>  import java.util.Map;
>  
>  
> @@ -27,13 +29,14 @@
>  
>  @Override
>  public void prepare(Map conf) {
> -sleepMillis = ((Number) 
> conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS)).longValue();
> +sleepMillis = Utils.getLong(
> +conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS), 
> 500);
>  }
>  
>  @Override
>  public void emptyEmit(long streak) {
>  try {
> -Thread.sleep(sleepMillis);
> +Thread.sleep(Math.abs(sleepMillis + streak));
>  } catch (InterruptedException e) {
>  throw new RuntimeException(e);
>  }
> Index: src/jvm/backtype/storm/utils/Utils.java
> ===
> --- src/jvm/backtype/storm/utils/Utils.java   (revision 2888)
> +++ src/jvm/backtype/storm/utils/Utils.java   (working copy)
> @@ -325,6 +325,24 @@
>throw new IllegalArgumentException("Don't know how to convert " + 
> o + " + to int");
>}
>  }
> +
> +public static Long getLong(Object o, long defaultValue) {
> +
> +  if (o == null) {
> +return defaultValue;
> +  }
> +
> +  if (o instanceof String) {
> +return Long.valueOf(String.valueOf(o));
> +  } else if (o instanceof Integer) {
> +Integer value = (Integer) o;
> +return Long.valueOf((Integer) value);
> +  } else if (o instanceof Long) {
> +return (Long) o;
> +  } else {
> +return defaultValue;
> +  }
> +}
>  
>  public static boolean getBoolean(Object o, boolean defaultValue) {
>if (null == o) {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-378,SleepSpoutWaitStrategy.emptyEmit sho...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61767078
  
To make it clear, PR points to "increase/decrease" sleep time with streak.
Fixed sleep time is configurable so it doesn't matter how long it is, 
optimal value should be vary for workload.

I think we can make new ISpoutWaitStrategy implementation that play with 
streak if we really need it.
We definitely need wait strategy to always sleep same time (ex. 1ms), so it 
isn't a good idea to change existing class's behavior.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-378) SleepSpoutWaitStrategy.emptyEmit should use the variable "streak"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197683#comment-14197683
 ] 

ASF GitHub Bot commented on STORM-378:
--

Github user itaifrenkel commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61766182
  
On the one hand - when the spout is a multilang bolt, 1ms drains ~10-5% of 
a CPU core on aws c3.large, so we increase it to 10ms.
On the other hand, increasing the sleep adds jitter to the spout latency 
when maxSpoutPending is enabled. Is there a configurable maximum value for the 
streak?


> SleepSpoutWaitStrategy.emptyEmit should use  the variable "streak"
> --
>
> Key: STORM-378
> URL: https://issues.apache.org/jira/browse/STORM-378
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: caofangkun
>Priority: Minor
>
> {code:java}
> Index: src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java
> ===
> --- src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (revision 2868)
> +++ src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (working copy)
> @@ -18,6 +18,8 @@
>  package backtype.storm.spout;
>  
>  import backtype.storm.Config;
> +import backtype.storm.utils.Utils;
> +
>  import java.util.Map;
>  
>  
> @@ -27,13 +29,14 @@
>  
>  @Override
>  public void prepare(Map conf) {
> -sleepMillis = ((Number) 
> conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS)).longValue();
> +sleepMillis = Utils.getLong(
> +conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS), 
> 500);
>  }
>  
>  @Override
>  public void emptyEmit(long streak) {
>  try {
> -Thread.sleep(sleepMillis);
> +Thread.sleep(Math.abs(sleepMillis + streak));
>  } catch (InterruptedException e) {
>  throw new RuntimeException(e);
>  }
> Index: src/jvm/backtype/storm/utils/Utils.java
> ===
> --- src/jvm/backtype/storm/utils/Utils.java   (revision 2888)
> +++ src/jvm/backtype/storm/utils/Utils.java   (working copy)
> @@ -325,6 +325,24 @@
>throw new IllegalArgumentException("Don't know how to convert " + 
> o + " + to int");
>}
>  }
> +
> +public static Long getLong(Object o, long defaultValue) {
> +
> +  if (o == null) {
> +return defaultValue;
> +  }
> +
> +  if (o instanceof String) {
> +return Long.valueOf(String.valueOf(o));
> +  } else if (o instanceof Integer) {
> +Integer value = (Integer) o;
> +return Long.valueOf((Integer) value);
> +  } else if (o instanceof Long) {
> +return (Long) o;
> +  } else {
> +return defaultValue;
> +  }
> +}
>  
>  public static boolean getBoolean(Object o, boolean defaultValue) {
>if (null == o) {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-378,SleepSpoutWaitStrategy.emptyEmit sho...

[itaifrenkel]

Github user itaifrenkel commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61766182
  
On the one hand - when the spout is a multilang bolt, 1ms drains ~10-5% of 
a CPU core on aws c3.large, so we increase it to 10ms.
On the other hand, increasing the sleep adds jitter to the spout latency 
when maxSpoutPending is enabled. Is there a configurable maximum value for the 
streak?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-537) A worker reconnects infinitely to another dead worker

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197658#comment-14197658
 ] 

ASF GitHub Bot commented on STORM-537:
--

Github user Sergeant007 commented on the pull request:

https://github.com/apache/storm/pull/304#issuecomment-61764038
  
Thanks for the review, @clockfly 

I have added necessary comment and removed the tests. Sorry, I wasn't able 
to simplify them - if they were in simple synchronous mode, the tests would 
hang infinitely if smth. is wrong instead of failures. So I've implemented them 
in quite complicated (since I'm new in Clojure), but robust way. Another issue 
was in complexity of reproducing of the bug: 1. you should be already connected 
and 2. it is reproduced only if you send several messages at once. Anyway, I 
have removed the tests as you asked me. Please, review.


> A worker reconnects infinitely to another dead worker
> -
>
> Key: STORM-537
> URL: https://issues.apache.org/jira/browse/STORM-537
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: Sergey Tryuber
>
> We're using 0.9.3-rc1. Most probably this wrong behavior was introduced as a 
> side efffect for STORM-409. When I kill a worker, another worker starts to 
> print messages like:
> {noformat}
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [0]
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [1]
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [2]
> . so on
> {noformat}
> Then it reaches default 300 max_retries and starts the cycle again:
> {noformat}
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] connection established to a remote 
> host Netty-Client-:4706, [id: 
> 0xec088412, /:39795 :> :4706]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [0]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [1]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [2]
> {noformat}
> And so on infinitely... 
> An issue most probably is in backtype.storm.messaging.netty.Client#connect 
> method in following place which determines that we give up on reconnection:
> {code}
> if (null != channel) {
> LOG.info("connection established to a remote host " + name() + ", " + 
> channel.toString());
> channelRef.set(channel);
> } else {
> close();
> throw new RuntimeException("Remote address is not reachable. We will 
> close this client " + name());
> }
> {code}
> I guess (not tried yet), that _channel_ object is not _null_ if this is a 
> real reconnection. So the method return a _channel_ object and then 
> reconnection starts again and again.
> This might be fixed by adding explicity *current = null;* into following code 
> block of the same method:
> {code}
> if (!future.isSuccess()) {
> if (null != current) {
> current.close();
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-537] A worker reconnects infinitely to ...

[Sergeant007]

Github user Sergeant007 commented on the pull request:

https://github.com/apache/storm/pull/304#issuecomment-61764038
  
Thanks for the review, @clockfly 

I have added necessary comment and removed the tests. Sorry, I wasn't able 
to simplify them - if they were in simple synchronous mode, the tests would 
hang infinitely if smth. is wrong instead of failures. So I've implemented them 
in quite complicated (since I'm new in Clojure), but robust way. Another issue 
was in complexity of reproducing of the bug: 1. you should be already connected 
and 2. it is reproduced only if you send several messages at once. Anyway, I 
have removed the tests as you asked me. Please, review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-329) Add Option to Config Message handling strategy when connection timeout

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197418#comment-14197418
 ] 

ASF GitHub Bot commented on STORM-329:
--

Github user tedxia commented on the pull request:

https://github.com/apache/storm/pull/268#issuecomment-61752674
  
I test this patch on our product cluster, with five machine, each with 6 
workers as max;

The topology based on trident run about 5 hours without fails.


Then I kill one worker called A, then I found the log below on worker 
B.Worker B don't exit as worker A died. 
```
2014-11-04 17:18:08 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [47]
2014-11-04 17:18:12 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [48]
2014-11-04 17:18:16 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [49]
2014-11-04 17:18:20 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [50]
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Closing Netty Client 
Netty-Client-A/xxx.xxx.xxx.xxx:21812
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Waiting for pending batchs to be 
sent with Netty-Client-A/xxx.xxx.xxx.xxx:21812..., timeout: 60ms, pendings: 0
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Client is being closed, and does 
not take requests any more, drop the messages...
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Client is being closed, and does 
not take requests any more, drop the messages...
```

As worker A died, nimbus reschedule a new worker F, then worker B connect 
to worker F.
```
2014-11-04 17:16:53 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [21]
2014-11-04 17:16:54 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-F/xxx.xxx.xxx.xxx:21813... [17]
2014-11-04 17:16:54 b.s.m.n.Client [INFO] connection established to a 
remote host Netty-Client-F/xxx.xxx.xxx.xxx:21813, [id: 0xbf721a18, 
/xxx.xxx.xxx.xxx:63811 => F/xxx.xxx.xxx.xxx:21813]
2014-11-04 17:16:55 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/10.2.201.65:21812... [22]
```
worker B connect to worker F successful before worker B close connection 
with Worker A.

Because this is our product cluster, I rewrite the hostname and ip in the 
log.




> Add Option to Config Message handling strategy when connection timeout
> --
>
> Key: STORM-329
> URL: https://issues.apache.org/jira/browse/STORM-329
> Project: Apache Storm
>  Issue Type: Improvement
>Affects Versions: 0.9.2-incubating
>Reporter: Sean Zhong
>Priority: Minor
>  Labels: Netty
> Fix For: 0.9.2-incubating
>
> Attachments: storm-329.patch, worker-kill-recover3.jpg
>
>
> This is to address a [concern brought 
> up|https://github.com/apache/incubator-storm/pull/103#issuecomment-43632986] 
> during the work at STORM-297:
> {quote}
> [~revans2] wrote: Your logic makes since to me on why these calls are 
> blocking. My biggest concern around the blocking is in the case of a worker 
> crashing. If a single worker crashes this can block the entire topology from 
> executing until that worker comes back up. In some cases I can see that being 
> something that you would want. In other cases I can see speed being the 
> primary concern and some users would like to get partial data fast, rather 
> then accurate data later.
> Could we make it configurable on a follow up JIRA where we can have a max 
> limit to the buffering that is allowed, before we block, or throw data away 
> (which is what zeromq does)?
> {quote}
> If some worker crash suddenly, how to handle the message which was supposed 
> to be delivered to the worker?
> 1. Should we buffer all message infinitely?
> 2. Should we block the message sending until the connection is resumed?
> 3. Should we config a buffer limit, try to buffer the message first, if the 
> limit is met, then block?
> 4. Should we neither block, nor buffer too much, but choose to drop the 
> messages, and use the built-in storm failover mechanism? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-329 : buffer message in client and recon...

[tedxia]

Github user tedxia commented on the pull request:

https://github.com/apache/storm/pull/268#issuecomment-61752674
  
I test this patch on our product cluster, with five machine, each with 6 
workers as max;

The topology based on trident run about 5 hours without fails.


Then I kill one worker called A, then I found the log below on worker 
B.Worker B don't exit as worker A died. 
```
2014-11-04 17:18:08 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [47]
2014-11-04 17:18:12 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [48]
2014-11-04 17:18:16 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [49]
2014-11-04 17:18:20 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [50]
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Closing Netty Client 
Netty-Client-A/xxx.xxx.xxx.xxx:21812
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Waiting for pending batchs to be 
sent with Netty-Client-A/xxx.xxx.xxx.xxx:21812..., timeout: 60ms, pendings: 0
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Client is being closed, and does 
not take requests any more, drop the messages...
2014-11-04 17:18:24 b.s.m.n.Client [INFO] Client is being closed, and does 
not take requests any more, drop the messages...
```

As worker A died, nimbus reschedule a new worker F, then worker B connect 
to worker F.
```
2014-11-04 17:16:53 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/xxx.xxx.xxx.xxx:21812... [21]
2014-11-04 17:16:54 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-F/xxx.xxx.xxx.xxx:21813... [17]
2014-11-04 17:16:54 b.s.m.n.Client [INFO] connection established to a 
remote host Netty-Client-F/xxx.xxx.xxx.xxx:21813, [id: 0xbf721a18, 
/xxx.xxx.xxx.xxx:63811 => F/xxx.xxx.xxx.xxx:21813]
2014-11-04 17:16:55 b.s.m.n.Client [INFO] Reconnect started for 
Netty-Client-A/10.2.201.65:21812... [22]
```
worker B connect to worker F successful before worker B close connection 
with Worker A.

Because this is our product cluster, I rewrite the hostname and ip in the 
log.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-547) Build Problem(s)

[Jungtaek Lim (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197266#comment-14197266
 ] 

Jungtaek Lim commented on STORM-547:


[~sriharsha] 
Previous PR we agreed to go with System.getProperty(), which is 
-DSTORM_TEST_TIMEOUT_MS=3.
Actually I'm still confused about System Environment and System Properties, but 
since Tailor checked it, it would be right.

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.3-rc2
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-547) Build Problem(s)

[Sriharsha Chintalapani (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197218#comment-14197218
 ] 

Sriharsha Chintalapani commented on STORM-547:
--

[~dbl001] I don't think you need that patch. We already have 
https://github.com/apache/storm/commit/d92e974a113bcaba656dd777985c520c8c9a4dd8 
this in the trunk.
do export STORM_TEST_TIMEOUT_MS=3 and try mvn clean package.

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.3-rc2
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-547) Build Problem(s)

[David Laxer (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197191#comment-14197191
 ] 

David Laxer commented on STORM-547:
---

I’m currently on a dual core Mac Book Pro running at 3.0 GHz with 8gb DRAM, 
however, I’m traveling and on a slow DSL connection (e.g. - 2.6 mb/sec).
So, perhaps allowing the timeout to be set in a configuration file would help.

Should I pull the patch?

Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/268.patch 





> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.3-rc2
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-547) Build Problem(s)

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-547:
-
Fix Version/s: (was: 0.9.0.1)

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.3-rc2
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-547) Build Problem(s)

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-547:
-
  Component/s: (was: storm-hdfs)
Affects Version/s: (was: 0.9.0.1)
   0.9.2-incubating
Fix Version/s: 0.9.3-rc2
   Issue Type: Bug  (was: Question)

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.0.1, 0.9.3-rc2
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (STORM-547) Build Problem(s)

[Sean Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197159#comment-14197159
 ] 

Sean Zhong edited comment on STORM-547 at 11/4/14 11:55 PM:


If you are trying to build a package, you can add: "-DskipTests" when using 
maven

About the problem, at first, there was a timeout of 5000ms, will lead to 
supervisor shutting down worker, which then lead to Exception thrown because 
connection is closed.

For the issues mentioned in this jira:
1. timeout 5000ms
   Seems this often happen when the machine is slow. Maybe we need to change it 
to a bigger default value.
2. Connection layer throw Exception when trying to send data to a closed 
connection.
  STORM-329 PR has fixed this. After the fix, the connection layer will drop 
the data directly instead of throw RuntimeException.


was (Author: clockfly):
If you are trying to build a package, you can add: "-DskipTests" when using 
maven

About the problem, at first, there was a timeout of 5000ms, will lead to 
supervisor shutting down worker, which then lead to Exception thrown because 
connection is closed.

For the issues mentioned in this jira:
1. timeout 5000ms
   Seems this often happen when the machine is slow. Maybe we need to change it 
to a bigger default value.
2. Netty throw Exception when trying to send data to a closed connection.
  STORM-329 PR has fixed this. After the fix, the connection layer will drop 
the data directly instead of throw RuntimeException.

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-hdfs
>Affects Versions: 0.9.0.1
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.0.1
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy t

[jira] [Comment Edited] (STORM-547) Build Problem(s)

[Sean Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197159#comment-14197159
 ] 

Sean Zhong edited comment on STORM-547 at 11/4/14 11:55 PM:


If you are trying to build a package, you can add: "-DskipTests" when using 
maven

About the problem, at first, there was a timeout of 5000ms, will lead to 
supervisor shutting down worker, which then lead to Exception thrown because 
connection is closed.

For the issues mentioned in this jira:
1. timeout 5000ms
   Seems this often happen when the machine is slow. Maybe we need to change it 
to a bigger default value.
2. Netty throw Exception when trying to send data to a closed connection.
  STORM-329 PR has fixed this. After the fix, the connection layer will drop 
the data directly instead of throw RuntimeException.


was (Author: clockfly):
If you are trying to build a package, you can add: "-DskipTests" when using 
maven

About the problem, at first, there was a timeout of 5000ms, will lead to 
supervisor shutting down worker, which then lead to Exception thrown because 
connection is closed.

For the issues mentioned in this jira:
1. timeout 5000ms
   Seems this often happen when the machine is slow. Maybe we need to change a 
bigger default value.
2. Netty throw Exception when trying to send data to a closed connection.
  STORM-329 PR has fixed this. After the fix, the connection layer will drop 
the data directly instead of throw RuntimeException.

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-hdfs
>Affects Versions: 0.9.0.1
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.0.1
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> v

[jira] [Commented] (STORM-547) Build Problem(s)

[Sean Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197159#comment-14197159
 ] 

Sean Zhong commented on STORM-547:
--

If you are trying to build a package, you can add: "-DskipTests" when using 
maven

About the problem, at first, there was a timeout of 5000ms, will lead to 
supervisor shutting down worker, which then lead to Exception thrown because 
connection is closed.

For the issues mentioned in this jira:
1. timeout 5000ms
   Seems this often happen when the machine is slow. Maybe we need to change a 
bigger default value.
2. Netty throw Exception when trying to send data to a closed connection.
  STORM-329 PR has fixed this. After the fix, the connection layer will drop 
the data directly instead of throw RuntimeException.

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-hdfs
>Affects Versions: 0.9.0.1
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.0.1
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by

[jira] [Updated] (STORM-547) Build Problem(s)

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani updated STORM-547:
-
Assignee: (was: Sriharsha Chintalapani)

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-hdfs
>Affects Versions: 0.9.0.1
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
> Fix For: 0.9.0.1
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196930#comment-14196930
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/305#discussion_r19839876
  
--- Diff: storm-core/src/jvm/backtype/storm/utils/ShellProcess.java ---
@@ -135,7 +135,14 @@ public void logErrorStream() {
 public String getErrorsString() {
 if (processErrorStream != null) {
 try {
-return IOUtils.toString(processErrorStream);
+StringBuilder sb = new StringBuilder();
+while (processErrorStream.available() > 0) {
+int bufferSize = processErrorStream.available();
--- End diff --

I'm curious to. Is it not safe because of chance of blocking, or 
available() can always return 0 with some InputStream subclasses so we cannot 
read any messages from subprocess?


> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[HeartSaVioR]

Github user HeartSaVioR commented on a diff in the pull request:

https://github.com/apache/storm/pull/305#discussion_r19839876
  
--- Diff: storm-core/src/jvm/backtype/storm/utils/ShellProcess.java ---
@@ -135,7 +135,14 @@ public void logErrorStream() {
 public String getErrorsString() {
 if (processErrorStream != null) {
 try {
-return IOUtils.toString(processErrorStream);
+StringBuilder sb = new StringBuilder();
+while (processErrorStream.available() > 0) {
+int bufferSize = processErrorStream.available();
--- End diff --

I'm curious to. Is it not safe because of chance of blocking, or 
available() can always return 0 with some InputStream subclasses so we cannot 
read any messages from subprocess?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-533) Add metrics collection for IConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196913#comment-14196913
 ] 

ASF GitHub Bot commented on STORM-533:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61722105
  
Btw, I think it's amazing to have connection / transmitted status metric. :)


> Add metrics collection for IConnection
> --
>
> Key: STORM-533
> URL: https://issues.apache.org/jira/browse/STORM-533
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> It would really be great to get some metrics from an IConnection that are 
> then sent to the metrics consumer. 
> We have seen issues in the past where a fire wall rule is mis-configured and 
> one host is unable to talk to another host.  If we had some metrics about how 
> many reconnection attempts are being made by the client to a given host we 
> could easily diagnose this.
> There are other metrics that would be nice to know too, like how many 
> bytes/tuples are being sent between different hosts. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-533] Added in client and server IConnec...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61722105
  
Btw, I think it's amazing to have connection / transmitted status metric. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-533) Add metrics collection for IConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196910#comment-14196910
 ] 

ASF GitHub Bot commented on STORM-533:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61721876
  
I agreed to @clockfly. I also think it would be better to make metrics 
easier to add. 
Actually I was spending a time to contribute STORM-533, but I cannot 
understand current storm metric system so I cannot even try it.


> Add metrics collection for IConnection
> --
>
> Key: STORM-533
> URL: https://issues.apache.org/jira/browse/STORM-533
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> It would really be great to get some metrics from an IConnection that are 
> then sent to the metrics consumer. 
> We have seen issues in the past where a fire wall rule is mis-configured and 
> one host is unable to talk to another host.  If we had some metrics about how 
> many reconnection attempts are being made by the client to a given host we 
> could easily diagnose this.
> There are other metrics that would be nice to know too, like how many 
> bytes/tuples are being sent between different hosts. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-533] Added in client and server IConnec...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61721876
  
I agreed to @clockfly. I also think it would be better to make metrics 
easier to add. 
Actually I was spending a time to contribute STORM-533, but I cannot 
understand current storm metric system so I cannot even try it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (STORM-547) Build Problem(s)

[Sriharsha Chintalapani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sriharsha Chintalapani reassigned STORM-547:


Assignee: Sriharsha Chintalapani

> Build Problem(s)
> 
>
> Key: STORM-547
> URL: https://issues.apache.org/jira/browse/STORM-547
> Project: Apache Storm
>  Issue Type: Question
>  Components: storm-hdfs
>Affects Versions: 0.9.0.1
> Environment: Mac OS X 10.10 Yosemite
>Reporter: David Laxer
>Assignee: Sriharsha Chintalapani
> Fix For: 0.9.0.1
>
>
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
> Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
> clojars (https://clojars.org/repo/)
> This could be due to a typo in :dependencies or network issues.
> If you are behind a proxy, try setting the 'http_proxy' environment variable.
> David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
> David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
> /Users/davidlaxer/bin/lein
> David-Laxers-MacBook-Pro:~ davidlaxer$ lein
> Leiningen is a tool for working with Clojure projects.
> Several tasks are available:
> bluuugh Dummy task for tests.
> change  Rewrite project.clj by applying a function.
> check   Check syntax and warn on reflection.
> classpath   Print the classpath of the current project.
> clean   Remove all files from project's target-path.
> compile Compile Clojure source into .class files.
> deploy  Build and deploy jar to remote repository.
> depsDownload all dependencies.
> do  Higher-order task to perform other tasks in succession.
> downloads   Calculate download statistics from logs.
> echoTask: 'echo' not found
> helpDisplay a list of tasks or help for a given task.
> install Install the current project to the local repository.
> jar Package up all the project's files into a jar file.
> javac   Compile Java source files.
> new Generate project scaffolding based on a template.
> one-or-two  Dummy task for tests
> plugin  DEPRECATED. Please use the :user profile instead.
> pom Write a pom.xml file to disk for Maven interoperability.
> pprint  Task: 'pprint' not found
> leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
> resolve symbol: defproject in this context, 
> compiling:(leiningen/project.clj:4:1)
> release Perform :release-tasks.
> replStart a repl session either with the current project or 
> standalone.
> retest  Run only the test namespaces which failed last time 
> around.
> run Run a -main function with optional command-line arguments.
> search  Search remote maven repositories for matching jars.
> show-profiles   List all available profiles or display one if given an 
> argument.
> sirius  Task: 'sirius' not found
> testRun the project's tests.
> trampoline  Run a task without nesting the project's JVM inside 
> Leiningen's.
> uberjar Package up the project files and dependencies into a jar 
> file.
> update-in   Perform arbitrary transformations on your project map.
> upgrade Upgrade Leiningen to specified version or latest stable.
> var-argsDummy task for tests.
> vcs Interact with the version control system.
> version Print version for Leiningen and the current JVM.
> with-profileApply the given task with the profile(s) specified.
> zeroDummy task for tests.
> Run `lein help $TASK` for details.
> Global Options:
>   -o Run a task offline.
>   -U Run a task after forcing update of snapshots.
>   -h, --help Print this help or help for a specific task.
>   -v, --version  Print Leiningen's version.
> See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
> mixed-source, templates, and copying.
> David-Laxers-MacBook-Pro:~ davidlaxer$ 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-513) ShellBolt keeps sending heartbeats even when child process is hung

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196893#comment-14196893
 ] 

ASF GitHub Bot commented on STORM-513:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61720636
  
OK, I've upmerged.
Btw, I found py files are diverged too so I need to copy and paste one file 
to another.


> ShellBolt keeps sending heartbeats even when child process is hung
> --
>
> Key: STORM-513
> URL: https://issues.apache.org/jira/browse/STORM-513
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Linux: 2.6.32-431.11.2.el6.x86_64 (RHEL 6.5)
>Reporter: Dan Blanchard
>Priority: Blocker
> Fix For: 0.9.3-rc2
>
>
> If I'm understanding everything correctly with how ShellBolts work, the Java 
> ShellBolt executor is the part of the topology that sends heartbeats back to 
> Nimbus to let it know that a particular multilang bolt is still alive.  The 
> problem with this is that if the multilang subprocess/bolt severely hangs 
> (i.e., it will not even respond to {{SIGALRM}} and the like), the Java 
> ShellBolt does not seem to notice or care. Simply having the tuple get 
> replayed when it times out will not suffice either, because the subprocess 
> will still be stuck.
> The most obvious way to handle this seem to be to add heartbeating to the 
> multilang protocol itself, so that the ShellBolt expects a message of some 
> kind every {{timeout}} seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-513 check heartbeat from multilang subpr...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61720636
  
OK, I've upmerged.
Btw, I found py files are diverged too so I need to copy and paste one file 
to another.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-513) ShellBolt keeps sending heartbeats even when child process is hung

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196872#comment-14196872
 ] 

ASF GitHub Bot commented on STORM-513:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61719213
  
I've got a change to discuss about this PR with @clockfly , and he also 
stated if subprocess is too busy, subprocess cannot send heartbeat in time, 
which I've stated first of this PR.

Actually it's better to let subprocess have heartbeat thread and send 
heartbeat periodically.
But there're two things to consider.
1. ShellSpout runs with PING-PONG communication, and ShellSpout must wait 
"sync" from nextTuple(). So if we change ShellSpout to have reader thread, we 
should implement nextTuple() to wait for reading "sync" from reader thread, 
which is a little complex than current.
2. We should ensure that main thread and heartbeat thread don't write 
stdout (maybe Pipe) at the same time. GIL could let us feel free, but there 
will be other languages that support real (?) thread. Writing operation should 
be with lock.

Since I'm not a Javascript (nodejs) guy, and I'm a beginner to Ruby, I 
cannot cover two things with .js. 
So I wish to implement it to other PR when we think we can't stand its 
limitation, or I have some more time.

Btw, Nimbus / Supervisor can find dead process due to subprocess hang up to 
SUPERVISOR_WORKER_TIMEOUT_SECS * 2 + a (maybe), cause there're two heartbeat 
check, ShellProcess checks subprocess (and suicide if subprocess cannot 
respond), Nimbus / Supervisor checks ShellProcess.
(Just for @clockfly )


> ShellBolt keeps sending heartbeats even when child process is hung
> --
>
> Key: STORM-513
> URL: https://issues.apache.org/jira/browse/STORM-513
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Linux: 2.6.32-431.11.2.el6.x86_64 (RHEL 6.5)
>Reporter: Dan Blanchard
>Priority: Blocker
> Fix For: 0.9.3-rc2
>
>
> If I'm understanding everything correctly with how ShellBolts work, the Java 
> ShellBolt executor is the part of the topology that sends heartbeats back to 
> Nimbus to let it know that a particular multilang bolt is still alive.  The 
> problem with this is that if the multilang subprocess/bolt severely hangs 
> (i.e., it will not even respond to {{SIGALRM}} and the like), the Java 
> ShellBolt does not seem to notice or care. Simply having the tuple get 
> replayed when it times out will not suffice either, because the subprocess 
> will still be stuck.
> The most obvious way to handle this seem to be to add heartbeating to the 
> multilang protocol itself, so that the ShellBolt expects a message of some 
> kind every {{timeout}} seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-513 check heartbeat from multilang subpr...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61719213
  
I've got a change to discuss about this PR with @clockfly , and he also 
stated if subprocess is too busy, subprocess cannot send heartbeat in time, 
which I've stated first of this PR.

Actually it's better to let subprocess have heartbeat thread and send 
heartbeat periodically.
But there're two things to consider.
1. ShellSpout runs with PING-PONG communication, and ShellSpout must wait 
"sync" from nextTuple(). So if we change ShellSpout to have reader thread, we 
should implement nextTuple() to wait for reading "sync" from reader thread, 
which is a little complex than current.
2. We should ensure that main thread and heartbeat thread don't write 
stdout (maybe Pipe) at the same time. GIL could let us feel free, but there 
will be other languages that support real (?) thread. Writing operation should 
be with lock.

Since I'm not a Javascript (nodejs) guy, and I'm a beginner to Ruby, I 
cannot cover two things with .js. 
So I wish to implement it to other PR when we think we can't stand its 
limitation, or I have some more time.

Btw, Nimbus / Supervisor can find dead process due to subprocess hang up to 
SUPERVISOR_WORKER_TIMEOUT_SECS * 2 + a (maybe), cause there're two heartbeat 
check, ShellProcess checks subprocess (and suicide if subprocess cannot 
respond), Nimbus / Supervisor checks ShellProcess.
(Just for @clockfly )


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-547) Build Problem(s)

[David Laxer (JIRA)]

David Laxer created STORM-547:
-

 Summary: Build Problem(s)
 Key: STORM-547
 URL: https://issues.apache.org/jira/browse/STORM-547
 Project: Apache Storm
  Issue Type: Question
  Components: storm-hdfs
Affects Versions: 0.9.0.1
 Environment: Mac OS X 10.10 Yosemite
Reporter: David Laxer
 Fix For: 0.9.0.1


David-Laxers-MacBook-Pro:leiningen davidlaxer$ which lein
/Users/davidlaxer/bin/lein
David-Laxers-MacBook-Pro:leiningen davidlaxer$ lein
Could not find artifact leiningen-core:leiningen-core:jar:2.5.1-SNAPSHOT in 
clojars (https://clojars.org/repo/)
This could be due to a typo in :dependencies or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
David-Laxers-MacBook-Pro:leiningen davidlaxer$ cd
David-Laxers-MacBook-Pro:~ davidlaxer$ which lein
/Users/davidlaxer/bin/lein
David-Laxers-MacBook-Pro:~ davidlaxer$ lein
Leiningen is a tool for working with Clojure projects.

Several tasks are available:
bluuugh Dummy task for tests.
change  Rewrite project.clj by applying a function.
check   Check syntax and warn on reflection.
classpath   Print the classpath of the current project.
clean   Remove all files from project's target-path.
compile Compile Clojure source into .class files.
deploy  Build and deploy jar to remote repository.
depsDownload all dependencies.
do  Higher-order task to perform other tasks in succession.
downloads   Calculate download statistics from logs.
echoTask: 'echo' not found
helpDisplay a list of tasks or help for a given task.
install Install the current project to the local repository.
jar Package up all the project's files into a jar file.
javac   Compile Java source files.
new Generate project scaffolding based on a template.
one-or-two  Dummy task for tests
plugin  DEPRECATED. Please use the :user profile instead.
pom Write a pom.xml file to disk for Maven interoperability.
pprint  Task: 'pprint' not found
leiningen.project  Problem loading: java.lang.RuntimeException: Unable to 
resolve symbol: defproject in this context, 
compiling:(leiningen/project.clj:4:1)
release Perform :release-tasks.
replStart a repl session either with the current project or 
standalone.
retest  Run only the test namespaces which failed last time around.
run Run a -main function with optional command-line arguments.
search  Search remote maven repositories for matching jars.
show-profiles   List all available profiles or display one if given an 
argument.
sirius  Task: 'sirius' not found
testRun the project's tests.
trampoline  Run a task without nesting the project's JVM inside 
Leiningen's.
uberjar Package up the project files and dependencies into a jar 
file.
update-in   Perform arbitrary transformations on your project map.
upgrade Upgrade Leiningen to specified version or latest stable.
var-argsDummy task for tests.
vcs Interact with the version control system.
version Print version for Leiningen and the current JVM.
with-profileApply the given task with the profile(s) specified.
zeroDummy task for tests.

Run `lein help $TASK` for details.

Global Options:
  -o Run a task offline.
  -U Run a task after forcing update of snapshots.
  -h, --help Print this help or help for a specific task.
  -v, --version  Print Leiningen's version.

See also: readme, faq, tutorial, news, sample, profiles, deploying, gpg,
mixed-source, templates, and copying.
David-Laxers-MacBook-Pro:~ davidlaxer$ 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-513) ShellBolt keeps sending heartbeats even when child process is hung

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196828#comment-14196828
 ] 

ASF GitHub Bot commented on STORM-513:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61715752
  
@harshach Sure. I'll upmerge it.


> ShellBolt keeps sending heartbeats even when child process is hung
> --
>
> Key: STORM-513
> URL: https://issues.apache.org/jira/browse/STORM-513
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Linux: 2.6.32-431.11.2.el6.x86_64 (RHEL 6.5)
>Reporter: Dan Blanchard
>Priority: Blocker
> Fix For: 0.9.3-rc2
>
>
> If I'm understanding everything correctly with how ShellBolts work, the Java 
> ShellBolt executor is the part of the topology that sends heartbeats back to 
> Nimbus to let it know that a particular multilang bolt is still alive.  The 
> problem with this is that if the multilang subprocess/bolt severely hangs 
> (i.e., it will not even respond to {{SIGALRM}} and the like), the Java 
> ShellBolt does not seem to notice or care. Simply having the tuple get 
> replayed when it times out will not suffice either, because the subprocess 
> will still be stuck.
> The most obvious way to handle this seem to be to add heartbeating to the 
> multilang protocol itself, so that the ShellBolt expects a message of some 
> kind every {{timeout}} seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-513 check heartbeat from multilang subpr...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61715752
  
@harshach Sure. I'll upmerge it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-535) setup 'java.library.path' for native-storm code if necessary

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196799#comment-14196799
 ] 

ASF GitHub Bot commented on STORM-535:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61713139
  
Just 2 cents.
extrajars consists of jar, ~/.storm, /bin, which may not fit for 
linking dll or so (for JNI) which is already installed for other places.
(Btw, is ~/.storm used for storing native library?)
AFAIK we could link it without modifying storm, LD_LIBRARY_PATH (Linux) and 
PATH (Windows) so we already have option.
Since I agree it's not sufficient to support both platforms (though it can 
fixed easily), I also think new path shouldn't in storm distribute directory. 
If we really need this, path should be fully flexible, not relative to storm 
dir. Because it can be harder for users to deploy their own library with Storm.
(@caofangkun Did you think how you deploy your so file to Storm?)
Regarding to PLATFORM distribution, it could be up to users to deploy their 
library which fit machine's architecture and OS.

tl;dr. It seems to not convenient after applying PR, though current may be 
not convenient, but we can use OS environment to cover this.


> setup 'java.library.path' for native-storm code if necessary
> 
>
> Key: STORM-535
> URL: https://issues.apache.org/jira/browse/STORM-535
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: caofangkun
>Priority: Minor
>
>  JAVA_LIBRARY_PATH=${STORM_HOME}/lib/native/${JAVA_PLATFORM}
> eg:
> If run storm on amd64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-amd64-64/libsigar-amd64-linux.so
> If run storm on x86_64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-x86_64-64/libsigar-x86-linux.so



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-535:setup 'java.library.path' for native...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61713139
  
Just 2 cents.
extrajars consists of jar, ~/.storm, /bin, which may not fit for 
linking dll or so (for JNI) which is already installed for other places.
(Btw, is ~/.storm used for storing native library?)
AFAIK we could link it without modifying storm, LD_LIBRARY_PATH (Linux) and 
PATH (Windows) so we already have option.
Since I agree it's not sufficient to support both platforms (though it can 
fixed easily), I also think new path shouldn't in storm distribute directory. 
If we really need this, path should be fully flexible, not relative to storm 
dir. Because it can be harder for users to deploy their own library with Storm.
(@caofangkun Did you think how you deploy your so file to Storm?)
Regarding to PLATFORM distribution, it could be up to users to deploy their 
library which fit machine's architecture and OS.

tl;dr. It seems to not convenient after applying PR, though current may be 
not convenient, but we can use OS environment to cover this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-535) setup 'java.library.path' for native-storm code if necessary

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196471#comment-14196471
 ] 

ASF GitHub Bot commented on STORM-535:
--

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61685448
  
-1 on this. I don't see why the config option is not sufficient. The code 
seem to make an assumption that STORM_DIR\lib\nativ exists which is not true 
for windows.


> setup 'java.library.path' for native-storm code if necessary
> 
>
> Key: STORM-535
> URL: https://issues.apache.org/jira/browse/STORM-535
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: caofangkun
>Priority: Minor
>
>  JAVA_LIBRARY_PATH=${STORM_HOME}/lib/native/${JAVA_PLATFORM}
> eg:
> If run storm on amd64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-amd64-64/libsigar-amd64-linux.so
> If run storm on x86_64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-x86_64-64/libsigar-x86-linux.so



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-535:setup 'java.library.path' for native...

[Parth-Brahmbhatt]

Github user Parth-Brahmbhatt commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61685448
  
-1 on this. I don't see why the config option is not sufficient. The code 
seem to make an assumption that STORM_DIR\lib\nativ exists which is not true 
for windows.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-456) Storm UI: cannot navigate to topology page when name contains spaces

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196339#comment-14196339
 ] 

ASF GitHub Bot commented on STORM-456:
--

Github user NareshKosgi commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61671428
  
@harshach Working on exactly those changes.  Will have a updated PR in a 
few days.


> Storm UI: cannot navigate to topology page when name contains spaces
> 
>
> Key: STORM-456
> URL: https://issues.apache.org/jira/browse/STORM-456
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: storm version: 0.9.1.2.1.2.0-402
> Firefox ESR 17.0.9 on RHEL
>Reporter: Steven Magana-Zook
>Priority: Trivial
>
> 1. Create a Java class that makes your topology
> 2. Submit the topology with a name that contains spaces: 
> StormSubmitter.submitTopology("I Dont Want to Use Underscores", conf, 
> builder.createTopology());
> 3. Submit the jar to storm: storm jar yourUberJar your.topology.class.name
> 4. Open Storm UI in your browser
> 5. Click the link for the submitted topology under "Topology Summary"
> Result: Page refreshes but does not show the topology page
> Expected Result: clicking the link should take you to the topology screen 
> like it does for topologies whose names do not contain spaces OR an error 
> should be returned if a user submits a topology whose names contains spaces 
> and this is not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...

[NareshKosgi]

Github user NareshKosgi commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61671428
  
@harshach Working on exactly those changes.  Will have a updated PR in a 
few days.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-513) ShellBolt keeps sending heartbeats even when child process is hung

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196324#comment-14196324
 ] 

ASF GitHub Bot commented on STORM-513:
--

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61669206
  
@HeartSaVioR  Thanks for the patch. can you upmerge the changes.


> ShellBolt keeps sending heartbeats even when child process is hung
> --
>
> Key: STORM-513
> URL: https://issues.apache.org/jira/browse/STORM-513
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Linux: 2.6.32-431.11.2.el6.x86_64 (RHEL 6.5)
>Reporter: Dan Blanchard
>Priority: Blocker
> Fix For: 0.9.3-rc2
>
>
> If I'm understanding everything correctly with how ShellBolts work, the Java 
> ShellBolt executor is the part of the topology that sends heartbeats back to 
> Nimbus to let it know that a particular multilang bolt is still alive.  The 
> problem with this is that if the multilang subprocess/bolt severely hangs 
> (i.e., it will not even respond to {{SIGALRM}} and the like), the Java 
> ShellBolt does not seem to notice or care. Simply having the tuple get 
> replayed when it times out will not suffice either, because the subprocess 
> will still be stuck.
> The most obvious way to handle this seem to be to add heartbeating to the 
> multilang protocol itself, so that the ShellBolt expects a message of some 
> kind every {{timeout}} seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-513 check heartbeat from multilang subpr...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61669206
  
@HeartSaVioR  Thanks for the patch. can you upmerge the changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-546) Local hostname configuration ignored by executor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196322#comment-14196322
 ] 

ASF GitHub Bot commented on STORM-546:
--

GitHub user strongh opened a pull request:

https://github.com/apache/storm/pull/306

use configured local hostname for reporting metrics and errors

This is a proposed fix for

https://issues.apache.org/jira/browse/STORM-546

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yieldbot/storm local-hostname-conf

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/306.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #306


commit a370c16780d143eb756c428c4c6a26dac05de5ab
Author: Homer Strong 
Date:   2014-11-04T16:30:20Z

use configured local hostname for reporting metrics and errors




> Local hostname configuration ignored by executor
> 
>
> Key: STORM-546
> URL: https://issues.apache.org/jira/browse/STORM-546
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3-rc2
>Reporter: Homer Strong
>Priority: Minor
>
> The executor reports hostname using `util/memoized-local-hostname`, but 
> doesn't check the configuration to see whether `STORM_LOCAL_HOSTNAME` has 
> been set. Under this change the executor checks the configuration before 
> falling back to guessing the local hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: use configured local hostname for reporting me...

[strongh]

GitHub user strongh opened a pull request:

https://github.com/apache/storm/pull/306

use configured local hostname for reporting metrics and errors

This is a proposed fix for

https://issues.apache.org/jira/browse/STORM-546

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yieldbot/storm local-hostname-conf

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/storm/pull/306.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #306


commit a370c16780d143eb756c428c4c6a26dac05de5ab
Author: Homer Strong 
Date:   2014-11-04T16:30:20Z

use configured local hostname for reporting metrics and errors




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (STORM-546) Local hostname configuration ignored by executor

[Homer Strong (JIRA)]

Homer Strong created STORM-546:
--

 Summary: Local hostname configuration ignored by executor
 Key: STORM-546
 URL: https://issues.apache.org/jira/browse/STORM-546
 Project: Apache Storm
  Issue Type: Bug
Affects Versions: 0.9.3-rc2
Reporter: Homer Strong
Priority: Minor


The executor reports hostname using `util/memoized-local-hostname`, but doesn't 
check the configuration to see whether `STORM_LOCAL_HOSTNAME` has been set. 
Under this change the executor checks the configuration before falling back to 
guessing the local hostname.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-456) Storm UI: cannot navigate to topology page when name contains spaces

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196321#comment-14196321
 ] 

ASF GitHub Bot commented on STORM-456:
--

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61668317
  
@NareshKosgi  I am -1 on current PR. Please look at the @Parth-Brahmbhatt  
suggested approach.
Also lets not modify the existing examples to test it. It will be great if 
you can add unit tests to check if the different topology names are valid or 
not. Thanks.

@clockfly special characters seems to have no issue. Please check the 
attached image. I was able to submit ./bin/storm jar 
examples/storm-starter/storm-starter-topologies-0.9.3-rc2-SNAPSHOT.jar 
storm.starter.WordCountTopology "word#&count)(" 

![image](https://cloud.githubusercontent.com/assets/38649/4903389/0a1d1cfa-6440-11e4-972b-1618dc1d1193.png)



> Storm UI: cannot navigate to topology page when name contains spaces
> 
>
> Key: STORM-456
> URL: https://issues.apache.org/jira/browse/STORM-456
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: storm version: 0.9.1.2.1.2.0-402
> Firefox ESR 17.0.9 on RHEL
>Reporter: Steven Magana-Zook
>Priority: Trivial
>
> 1. Create a Java class that makes your topology
> 2. Submit the topology with a name that contains spaces: 
> StormSubmitter.submitTopology("I Dont Want to Use Underscores", conf, 
> builder.createTopology());
> 3. Submit the jar to storm: storm jar yourUberJar your.topology.class.name
> 4. Open Storm UI in your browser
> 5. Click the link for the submitted topology under "Topology Summary"
> Result: Page refreshes but does not show the topology page
> Expected Result: clicking the link should take you to the topology screen 
> like it does for topologies whose names do not contain spaces OR an error 
> should be returned if a user submits a topology whose names contains spaces 
> and this is not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...

[harshach]

Github user harshach commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61668317
  
@NareshKosgi  I am -1 on current PR. Please look at the @Parth-Brahmbhatt  
suggested approach.
Also lets not modify the existing examples to test it. It will be great if 
you can add unit tests to check if the different topology names are valid or 
not. Thanks.

@clockfly special characters seems to have no issue. Please check the 
attached image. I was able to submit ./bin/storm jar 
examples/storm-starter/storm-starter-topologies-0.9.3-rc2-SNAPSHOT.jar 
storm.starter.WordCountTopology "word#&count)(" 

![image](https://cloud.githubusercontent.com/assets/38649/4903389/0a1d1cfa-6440-11e4-972b-1618dc1d1193.png)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-513 check heartbeat from multilang subpr...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61657845
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-533) Add metrics collection for IConnection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196242#comment-14196242
 ] 

ASF GitHub Bot commented on STORM-533:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61658959
  
+1 for the ideas. We should have more metrics about the connection status.

From other side, current storm metric system code is too hard to read and 
too hard to use. 

 Maybe we should use mature library like codahale to ease this process, 
instead of maintaining our own metric facility.




> Add metrics collection for IConnection
> --
>
> Key: STORM-533
> URL: https://issues.apache.org/jira/browse/STORM-533
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> It would really be great to get some metrics from an IConnection that are 
> then sent to the metrics consumer. 
> We have seen issues in the past where a fire wall rule is mis-configured and 
> one host is unable to talk to another host.  If we had some metrics about how 
> many reconnection attempts are being made by the client to a given host we 
> could easily diagnose this.
> There are other metrics that would be nice to know too, like how many 
> bytes/tuples are being sent between different hosts. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-533] Added in client and server IConnec...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/302#issuecomment-61658959
  
+1 for the ideas. We should have more metrics about the connection status.

From other side, current storm metric system code is too hard to read and 
too hard to use. 

 Maybe we should use mature library like codahale to ease this process, 
instead of maintaining our own metric facility.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-513) ShellBolt keeps sending heartbeats even when child process is hung

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196234#comment-14196234
 ] 

ASF GitHub Bot commented on STORM-513:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/286#issuecomment-61657845
  
+1


> ShellBolt keeps sending heartbeats even when child process is hung
> --
>
> Key: STORM-513
> URL: https://issues.apache.org/jira/browse/STORM-513
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
> Environment: Linux: 2.6.32-431.11.2.el6.x86_64 (RHEL 6.5)
>Reporter: Dan Blanchard
>Priority: Blocker
> Fix For: 0.9.3-rc2
>
>
> If I'm understanding everything correctly with how ShellBolts work, the Java 
> ShellBolt executor is the part of the topology that sends heartbeats back to 
> Nimbus to let it know that a particular multilang bolt is still alive.  The 
> problem with this is that if the multilang subprocess/bolt severely hangs 
> (i.e., it will not even respond to {{SIGALRM}} and the like), the Java 
> ShellBolt does not seem to notice or care. Simply having the tuple get 
> replayed when it times out will not suffice either, because the subprocess 
> will still be stuck.
> The most obvious way to handle this seem to be to add heartbeating to the 
> multilang protocol itself, so that the ShellBolt expects a message of some 
> kind every {{timeout}} seconds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: Remove 'provided' from storm-kafka pom.xml exa...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/290


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Update README.md

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/292


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Update README.md

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/292#issuecomment-61644279
  
committed to trunk.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: Remove 'provided' from storm-kafka pom.xml exa...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/290#issuecomment-61644034
  
committed into trunk


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-525) UI Component Page Executor Uptimes are not sorted correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196119#comment-14196119
 ] 

ASF GitHub Bot commented on STORM-525:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/289#issuecomment-61643705
  
+1


> UI Component Page Executor Uptimes are not sorted correctly
> ---
>
> Key: STORM-525
> URL: https://issues.apache.org/jira/browse/STORM-525
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: Derek Dagit
>Assignee: Derek Dagit
>Priority: Minor
>
> When manually sorting the Uptime column on a bolt component page, the times 
> are incorrectly sorted (by ASCII).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: Remove 'provided' from storm-kafka pom.xml exa...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/290#issuecomment-61643778
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-525: Add time sorting function to the 2n...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/289#issuecomment-61643705
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-492) Test timeout should be configurable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196113#comment-14196113
 ] 

ASF GitHub Bot commented on STORM-492:
--

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/279#issuecomment-61642901
  
@harshach @ptgoetz @clockfly Thanks for reviewing and merging!


> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>Assignee: Jungtaek Lim
>  Labels: newbie
> Fix For: 0.9.3-rc2
>
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-492: Fixed bug from tracked-wait / Added...

[HeartSaVioR]

Github user HeartSaVioR commented on the pull request:

https://github.com/apache/storm/pull/279#issuecomment-61642901
  
@harshach @ptgoetz @clockfly Thanks for reviewing and merging!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-492) Test timeout should be configurable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196112#comment-14196112
 ] 

ASF GitHub Bot commented on STORM-492:
--

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/279


> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>Assignee: Jungtaek Lim
>  Labels: newbie
> Fix For: 0.9.3-rc2
>
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-492: Fixed bug from tracked-wait / Added...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/279


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (STORM-492) Test timeout should be configurable

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-492:
-
Assignee: Jungtaek Lim  (was: Sean Zhong)

> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>Assignee: Jungtaek Lim
>  Labels: newbie
> Fix For: 0.9.3-rc2
>
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-492) Test timeout should be configurable

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong resolved STORM-492.
--
   Resolution: Fixed
Fix Version/s: 0.9.3-rc2

committed.

Thanks, Jungtaek.

> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>Assignee: Jungtaek Lim
>  Labels: newbie
> Fix For: 0.9.3-rc2
>
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (STORM-492) Test timeout should be configurable

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong reassigned STORM-492:


Assignee: Sean Zhong

> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>Assignee: Sean Zhong
>  Labels: newbie
> Fix For: 0.9.3-rc2
>
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-492) Test timeout should be configurable

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196108#comment-14196108
 ] 

ASF GitHub Bot commented on STORM-492:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/279#issuecomment-61641290
  
+1


> Test timeout should be configurable
> ---
>
> Key: STORM-492
> URL: https://issues.apache.org/jira/browse/STORM-492
> Project: Apache Storm
>  Issue Type: Bug
>Reporter: Devika Nair
>  Labels: newbie
>
> Test timeout is hard coded to 5000ms in 
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj.
>  Provide a way to override this value in order to handle longer running tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-492: Fixed bug from tracked-wait / Added...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/279#issuecomment-61641290
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-493: Workers inherit storm.conf.file/sto...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/252


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-493) Workers don't inherit storm.conf.file/storm.options properties of the supervisor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196095#comment-14196095
 ] 

ASF GitHub Bot commented on STORM-493:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/252#issuecomment-61639240
  
merged.


> Workers don't inherit storm.conf.file/storm.options properties of the 
> supervisor
> 
>
> Key: STORM-493
> URL: https://issues.apache.org/jira/browse/STORM-493
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating, 0.9.3
>Reporter: Christophe Carré
>Assignee: Christophe Carré
>Priority: Minor
> Fix For: 0.9.3
>
>
> If we override some configuration parameters on the command line (using storm 
> -c "param=value") when we launch the supervisor, workers don't inherit them.
> {noformat}
> > cat conf/storm.yaml
> storm.zookeeper.servers:
>  - "127.0.0.1"
> nimbus.host: "127.0.0.1"
> storm.zookeeper.root: "/stormtest"
> storm.local.dir: "storm-local-main"
> > python bin/storm -c "storm.local.dir=\"storm-local-custom\"" supervisor
> > less logs/worker-6701.log
> [...]
> 2014-09-10 09:35:00 o.a.s.z.s.ZooKeeperServer [INFO] Server 
> environment:user.dir=/optc/2014-09-05-5aae7686
> 2014-09-10 09:35:01 b.s.d.worker [INFO] Launching worker for 
> mytopo-1-1410334488 on 96f32da2-2043-4371-988b-ec9ca107ce69:6701 with id 
> b9178c80-922b-4b8d-9984-7cfaf06f3c86 and conf {"dev.zookeeper.path" 
> "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, 
> "topology.builtin.metrics.bucket.size.secs" 60, 
> "topology.fall.back.on.java.serialization" true, 
> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, 
> "topology.skip.missing.kryo.registrations" false, 
> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", 
> "storm.zookeeper.session.timeout" 2, "nimbus.reassign" true, 
> "topology.trident.batch.emit.interval.millis" 500, 
> "storm.messaging.netty.flush.check.interval.ms" 10, 
> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", 
> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", 
> "topology.executor.send.buffer.size" 1024, "storm.local.dir" 
> "storm-local-main", "storm.messaging.netty.buffer_size" 5242880, 
> "supervisor.worker.start.timeout.secs" 120, 
> "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 
> 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, 
> "storm.meta.serialization.delegate" 
> "backtype.storm.serialization.DefaultSerializationDelegate", 
> "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "127.0.0.1", 
> "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181, 
> "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 
> 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" 
> "/stormtest", "storm.zookeeper.retry.intervalceiling.millis" 3, 
> "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, 
> "storm.zookeeper.servers" ["127.0.0.1"], "transactional.zookeeper.root" 
> "/transactional", "topology.acker.executors" nil, 
> "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, 
> "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", 
> "supervisor.heartbeat.frequency.secs" 5, 
> "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, 
> "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", 
> "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, 
> "topology.tasks" nil, "storm.messaging.netty.max_retries" 300, 
> "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", 
> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, 
> "storm.zookeeper.retry.interval" 1000, 
> "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" 
> "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" 
> [6700 6701 6702 6703], "topology.environment" nil, "topology.debug" false, 
> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, 
> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, 
> "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 
> 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, 
> "topology.tuple.serializer" 
> "backtype.storm.serialization.types.ListDelegateSerializer", 
> "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", 
> "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", 
> "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, 
> "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFact

[jira] [Commented] (STORM-493) Workers don't inherit storm.conf.file/storm.options properties of the supervisor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196096#comment-14196096
 ] 

ASF GitHub Bot commented on STORM-493:
--

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/252


> Workers don't inherit storm.conf.file/storm.options properties of the 
> supervisor
> 
>
> Key: STORM-493
> URL: https://issues.apache.org/jira/browse/STORM-493
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating, 0.9.3
>Reporter: Christophe Carré
>Assignee: Christophe Carré
>Priority: Minor
> Fix For: 0.9.3
>
>
> If we override some configuration parameters on the command line (using storm 
> -c "param=value") when we launch the supervisor, workers don't inherit them.
> {noformat}
> > cat conf/storm.yaml
> storm.zookeeper.servers:
>  - "127.0.0.1"
> nimbus.host: "127.0.0.1"
> storm.zookeeper.root: "/stormtest"
> storm.local.dir: "storm-local-main"
> > python bin/storm -c "storm.local.dir=\"storm-local-custom\"" supervisor
> > less logs/worker-6701.log
> [...]
> 2014-09-10 09:35:00 o.a.s.z.s.ZooKeeperServer [INFO] Server 
> environment:user.dir=/optc/2014-09-05-5aae7686
> 2014-09-10 09:35:01 b.s.d.worker [INFO] Launching worker for 
> mytopo-1-1410334488 on 96f32da2-2043-4371-988b-ec9ca107ce69:6701 with id 
> b9178c80-922b-4b8d-9984-7cfaf06f3c86 and conf {"dev.zookeeper.path" 
> "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, 
> "topology.builtin.metrics.bucket.size.secs" 60, 
> "topology.fall.back.on.java.serialization" true, 
> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, 
> "topology.skip.missing.kryo.registrations" false, 
> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", 
> "storm.zookeeper.session.timeout" 2, "nimbus.reassign" true, 
> "topology.trident.batch.emit.interval.millis" 500, 
> "storm.messaging.netty.flush.check.interval.ms" 10, 
> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", 
> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", 
> "topology.executor.send.buffer.size" 1024, "storm.local.dir" 
> "storm-local-main", "storm.messaging.netty.buffer_size" 5242880, 
> "supervisor.worker.start.timeout.secs" 120, 
> "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 
> 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, 
> "storm.meta.serialization.delegate" 
> "backtype.storm.serialization.DefaultSerializationDelegate", 
> "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "127.0.0.1", 
> "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181, 
> "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 
> 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" 
> "/stormtest", "storm.zookeeper.retry.intervalceiling.millis" 3, 
> "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, 
> "storm.zookeeper.servers" ["127.0.0.1"], "transactional.zookeeper.root" 
> "/transactional", "topology.acker.executors" nil, 
> "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, 
> "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", 
> "supervisor.heartbeat.frequency.secs" 5, 
> "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, 
> "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", 
> "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, 
> "topology.tasks" nil, "storm.messaging.netty.max_retries" 300, 
> "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", 
> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, 
> "storm.zookeeper.retry.interval" 1000, 
> "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" 
> "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" 
> [6700 6701 6702 6703], "topology.environment" nil, "topology.debug" false, 
> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, 
> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, 
> "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 
> 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, 
> "topology.tuple.serializer" 
> "backtype.storm.serialization.types.ListDelegateSerializer", 
> "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", 
> "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", 
> "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, 
> "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", 
> "drpc.invocations.port" 3773, "lo

[GitHub] storm pull request: STORM-493: Workers inherit storm.conf.file/sto...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/252#issuecomment-61639240
  
merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-493) Workers don't inherit storm.conf.file/storm.options properties of the supervisor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196094#comment-14196094
 ] 

ASF GitHub Bot commented on STORM-493:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/252#issuecomment-61639159
  
+1


> Workers don't inherit storm.conf.file/storm.options properties of the 
> supervisor
> 
>
> Key: STORM-493
> URL: https://issues.apache.org/jira/browse/STORM-493
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating, 0.9.3
>Reporter: Christophe Carré
>Assignee: Christophe Carré
>Priority: Minor
> Fix For: 0.9.3
>
>
> If we override some configuration parameters on the command line (using storm 
> -c "param=value") when we launch the supervisor, workers don't inherit them.
> {noformat}
> > cat conf/storm.yaml
> storm.zookeeper.servers:
>  - "127.0.0.1"
> nimbus.host: "127.0.0.1"
> storm.zookeeper.root: "/stormtest"
> storm.local.dir: "storm-local-main"
> > python bin/storm -c "storm.local.dir=\"storm-local-custom\"" supervisor
> > less logs/worker-6701.log
> [...]
> 2014-09-10 09:35:00 o.a.s.z.s.ZooKeeperServer [INFO] Server 
> environment:user.dir=/optc/2014-09-05-5aae7686
> 2014-09-10 09:35:01 b.s.d.worker [INFO] Launching worker for 
> mytopo-1-1410334488 on 96f32da2-2043-4371-988b-ec9ca107ce69:6701 with id 
> b9178c80-922b-4b8d-9984-7cfaf06f3c86 and conf {"dev.zookeeper.path" 
> "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, 
> "topology.builtin.metrics.bucket.size.secs" 60, 
> "topology.fall.back.on.java.serialization" true, 
> "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 5000, 
> "topology.skip.missing.kryo.registrations" false, 
> "storm.messaging.netty.client_worker_threads" 1, "ui.childopts" "-Xmx768m", 
> "storm.zookeeper.session.timeout" 2, "nimbus.reassign" true, 
> "topology.trident.batch.emit.interval.millis" 500, 
> "storm.messaging.netty.flush.check.interval.ms" 10, 
> "nimbus.monitor.freq.secs" 10, "logviewer.childopts" "-Xmx128m", 
> "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", 
> "topology.executor.send.buffer.size" 1024, "storm.local.dir" 
> "storm-local-main", "storm.messaging.netty.buffer_size" 5242880, 
> "supervisor.worker.start.timeout.secs" 120, 
> "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 
> 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, 
> "storm.meta.serialization.delegate" 
> "backtype.storm.serialization.DefaultSerializationDelegate", 
> "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "127.0.0.1", 
> "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2181, 
> "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 
> 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" 
> "/stormtest", "storm.zookeeper.retry.intervalceiling.millis" 3, 
> "supervisor.enable" true, "storm.messaging.netty.server_worker_threads" 1, 
> "storm.zookeeper.servers" ["127.0.0.1"], "transactional.zookeeper.root" 
> "/transactional", "topology.acker.executors" nil, 
> "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, 
> "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", 
> "supervisor.heartbeat.frequency.secs" 5, 
> "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, 
> "supervisor.monitor.frequency.secs" 3, "drpc.childopts" "-Xmx768m", 
> "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, 
> "topology.tasks" nil, "storm.messaging.netty.max_retries" 300, 
> "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", 
> "nimbus.thrift.max_buffer_size" 1048576, "topology.max.spout.pending" nil, 
> "storm.zookeeper.retry.interval" 1000, 
> "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" 
> "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" 
> [6700 6701 6702 6703], "topology.environment" nil, "topology.debug" false, 
> "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, 
> "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, 
> "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 
> 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, 
> "topology.tuple.serializer" 
> "backtype.storm.serialization.types.ListDelegateSerializer", 
> "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", 
> "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", 
> "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, 
> "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory",

[GitHub] storm pull request: STORM-493: Workers inherit storm.conf.file/sto...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/252#issuecomment-61639159
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (STORM-307) After host crash, supervisor is unable to restart itself

[Sean Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196077#comment-14196077
 ] 

Sean Zhong edited comment on STORM-307 at 11/4/14 1:13 PM:
---

committed to trunk.
Thanks, wurstmeister and Damien Raude-Morvan


was (Author: clockfly):
committed to trunk.
Thanks, wurstmeister

> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
> Fix For: 0.9.3-rc2
>
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-307) After host crash, supervisor is unable to restart itself

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-307:
-
Assignee: (was: Jiahong Li)

> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
> Fix For: 0.9.3-rc2
>
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (STORM-307) After host crash, supervisor is unable to restart itself

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong resolved STORM-307.
--
   Resolution: Fixed
Fix Version/s: 0.9.3-rc2

committed to trunk.
Thanks, wurstmeister

> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
> Fix For: 0.9.3-rc2
>
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-307) After host crash, supervisor is unable to restart itself

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-307:
-
Assignee: Jiahong Li

> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
>Assignee: Jiahong Li
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-307) After host crash, supervisor is unable to restart itself

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196076#comment-14196076
 ] 

ASF GitHub Bot commented on STORM-307:
--

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/282


> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-307: reset LocalState if files are corru...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/282


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-307) After host crash, supervisor is unable to restart itself

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196071#comment-14196071
 ] 

ASF GitHub Bot commented on STORM-307:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/282#issuecomment-61636291
  
+1


> After host crash, supervisor is unable to restart itself
> 
>
> Key: STORM-307
> URL: https://issues.apache.org/jira/browse/STORM-307
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: Debian Linux Wheezy
> Zookeeper 3.3.3
> Java 1.7.0_25
>Reporter: Damien Raude-Morvan
> Attachments: supeof.tar.bz2
>
>
> Hi,
> I've observed [multiple times|#links] that supervisor state de-serialisation 
> after host crash or reboot can fail. Supervisor is then unable to come up 
> without manual intervention. AFAICT, it seems that serialized supervisor 
> state if invalid and coun't be read at next start.
> Observed error in supervisor log :
> {noformat}
> 2014-04-29 19:38:35 c.n.c.f.i.CuratorFrameworkImpl [INFO] Starting
> 2014-04-29 19:38:35 o.a.z.ZooKeeper [INFO] Initiating client connection, 
> connectString=127.0.0.1:2181/storm sessionTimeout=2 
> watcher=com.netflix.curator.ConnectionState@18d055e0
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Opening socket connection to 
> server /127.0.0.1:2181
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Socket connection established to 
> localhost/127.0.0.1:2181, initiating session
> 2014-04-29 19:38:35 o.a.z.ClientCnxn [INFO] Session establishment complete on 
> server localhost/127.0.0.1:2181, sessionid = 0x145a7cc1c7e48b1, negotiated 
> timeout = 2
> 2014-04-29 19:38:35 b.s.d.supervisor [INFO] Starting supervisor with id 
> 71b01216-9d00-4fb6-8538-6673058ab5ef at host storm
> 2014-04-29 19:38:36 b.s.event [ERROR] Error when processing event
> java.lang.RuntimeException: java.io.EOFException
> at backtype.storm.utils.Utils.deserialize(Utils.java:86) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.snapshot(LocalState.java:45) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at backtype.storm.utils.LocalState.get(LocalState.java:56) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at 
> backtype.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:207) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> at clojure.lang.AFn.applyToHelper(AFn.java:161) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.AFn.applyTo(AFn.java:151) ~[clojure-1.4.0.jar:na]
> at clojure.core$apply.invoke(core.clj:603) ~[clojure-1.4.0.jar:na]
> at clojure.core$partial$fn__4070.doInvoke(core.clj:2343) 
> ~[clojure-1.4.0.jar:na]
> at clojure.lang.RestFn.invoke(RestFn.java:397) ~[clojure-1.4.0.jar:na]
> at backtype.storm.event$event_manager$fn__2593.invoke(event.clj:39) 
> ~[na:na]
> at clojure.lang.AFn.run(AFn.java:24) ~[clojure-1.4.0.jar:na]
> at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_25]
> Caused by: java.io.EOFException: null
> at 
> java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2323)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2792)
>  ~[na:1.7.0_25]
> at 
> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:799) 
> ~[na:1.7.0_25]
> at java.io.ObjectInputStream.(ObjectInputStream.java:299) 
> ~[na:1.7.0_25]
> at backtype.storm.utils.Utils.deserialize(Utils.java:81) 
> ~[storm-core-0.9.1-incubating.jar:0.9.1-incubating]
> ... 11 common frames omitted
> 2014-04-29 19:38:36 b.s.util [INFO] Halting process: ("Error when processing 
> an event")
> {noformat}
> Current workaround : full stop supervisor daemon and delete all Storm's 
> data/supervisor directory helped, and after restarting Supervisor is now 
> running smoothly. 
> {anchor:links} Here is some references of very similar issues :
> * 
> http://mail-archives.apache.org/mod_mbox/storm-user/201402.mbox/%3c23100d14e7ac4cef947f7236ef896...@by2pr08mb144.namprd08.prod.outlook.com%3E
> * https://groups.google.com/forum/#!topic/storm-user/SL9FK9XeoI8
> * https://groups.google.com/forum/#!topic/storm-user/2gapTYTRrX8
> Regards,



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-307: reset LocalState if files are corru...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/282#issuecomment-61636291
  
+1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (STORM-529) Python BasicBolt does not behave as the Java BasicBolt

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong resolved STORM-529.
--
   Resolution: Fixed
Fix Version/s: 0.9.3-rc2

committed, thanks, Itai.

> Python BasicBolt does not behave as the Java BasicBolt
> --
>
> Key: STORM-529
> URL: https://issues.apache.org/jira/browse/STORM-529
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: Itai Frenkel
>Assignee: Itai Frenkel
> Fix For: 0.9.3-rc2
>
>
> We expect the basicbolt on exception to report an error and fail the tuple. 
> Instead it just reports an error and stops processing any more tuples.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-529) Python BasicBolt does not behave as the Java BasicBolt

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196064#comment-14196064
 ] 

ASF GitHub Bot commented on STORM-529:
--

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/291


> Python BasicBolt does not behave as the Java BasicBolt
> --
>
> Key: STORM-529
> URL: https://issues.apache.org/jira/browse/STORM-529
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: Itai Frenkel
>Assignee: Itai Frenkel
> Fix For: 0.9.3-rc2
>
>
> We expect the basicbolt on exception to report an error and fail the tuple. 
> Instead it just reports an error and stops processing any more tuples.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (STORM-529) Python BasicBolt does not behave as the Java BasicBolt

[Sean Zhong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/STORM-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Zhong updated STORM-529:
-
Assignee: Itai Frenkel

> Python BasicBolt does not behave as the Java BasicBolt
> --
>
> Key: STORM-529
> URL: https://issues.apache.org/jira/browse/STORM-529
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: Itai Frenkel
>Assignee: Itai Frenkel
> Fix For: 0.9.3-rc2
>
>
> We expect the basicbolt on exception to report an error and fail the tuple. 
> Instead it just reports an error and stops processing any more tuples.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-529] Send fail(tup) when BasicBolt proc...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/storm/pull/291


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-378,SleepSpoutWaitStrategy.emptyEmit sho...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61634029
  
To make the whole topology responsive, the spout need to stay active to 
pull data frequently from acker or system tick.

When setting "topology.sleep.spout.wait.strategy.time.ms" to 1 ms,  it 
should be good enough, the system load is relatively small.

What is the motivation to make it increasing?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-378) SleepSpoutWaitStrategy.emptyEmit should use the variable "streak"

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196059#comment-14196059
 ] 

ASF GitHub Bot commented on STORM-378:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/295#issuecomment-61634029
  
To make the whole topology responsive, the spout need to stay active to 
pull data frequently from acker or system tick.

When setting "topology.sleep.spout.wait.strategy.time.ms" to 1 ms,  it 
should be good enough, the system load is relatively small.

What is the motivation to make it increasing?


> SleepSpoutWaitStrategy.emptyEmit should use  the variable "streak"
> --
>
> Key: STORM-378
> URL: https://issues.apache.org/jira/browse/STORM-378
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.2-incubating
>Reporter: caofangkun
>Priority: Minor
>
> {code:java}
> Index: src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java
> ===
> --- src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (revision 2868)
> +++ src/jvm/backtype/storm/spout/SleepSpoutWaitStrategy.java  (working copy)
> @@ -18,6 +18,8 @@
>  package backtype.storm.spout;
>  
>  import backtype.storm.Config;
> +import backtype.storm.utils.Utils;
> +
>  import java.util.Map;
>  
>  
> @@ -27,13 +29,14 @@
>  
>  @Override
>  public void prepare(Map conf) {
> -sleepMillis = ((Number) 
> conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS)).longValue();
> +sleepMillis = Utils.getLong(
> +conf.get(Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS), 
> 500);
>  }
>  
>  @Override
>  public void emptyEmit(long streak) {
>  try {
> -Thread.sleep(sleepMillis);
> +Thread.sleep(Math.abs(sleepMillis + streak));
>  } catch (InterruptedException e) {
>  throw new RuntimeException(e);
>  }
> Index: src/jvm/backtype/storm/utils/Utils.java
> ===
> --- src/jvm/backtype/storm/utils/Utils.java   (revision 2888)
> +++ src/jvm/backtype/storm/utils/Utils.java   (working copy)
> @@ -325,6 +325,24 @@
>throw new IllegalArgumentException("Don't know how to convert " + 
> o + " + to int");
>}
>  }
> +
> +public static Long getLong(Object o, long defaultValue) {
> +
> +  if (o == null) {
> +return defaultValue;
> +  }
> +
> +  if (o instanceof String) {
> +return Long.valueOf(String.valueOf(o));
> +  } else if (o instanceof Integer) {
> +Integer value = (Integer) o;
> +return Long.valueOf((Integer) value);
> +  } else if (o instanceof Long) {
> +return (Long) o;
> +  } else {
> +return defaultValue;
> +  }
> +}
>  
>  public static boolean getBoolean(Object o, boolean defaultValue) {
>if (null == o) {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196050#comment-14196050
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61632248
  
dashengju is describling a use scenario that parent process blocks while 
trying to read error stream data from sub-process (parent java space throw an 
exception, which trigger the error handling).

Becasue sub-process is still running, there is no EOF of errorStream, so 
errorStream.read() will blocks.

If in this scenario, we can make sure the child process is dead by called 
child.destroy(), then there will be an EOF signalled to its error Stream, and 
parent process will not block.


> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61632248
  
dashengju is describling a use scenario that parent process blocks while 
trying to read error stream data from sub-process (parent java space throw an 
exception, which trigger the error handling).

Becasue sub-process is still running, there is no EOF of errorStream, so 
errorStream.read() will blocks.

If in this scenario, we can make sure the child process is dead by called 
child.destroy(), then there will be an EOF signalled to its error Stream, and 
parent process will not block.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196047#comment-14196047
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631943
  
@itaifrenkel,

```
 When you say "the parent process fails" do you mean that the worker 
process is no longer running?
```
No, it means the case described by dashengju, that an exception is thrown 
in the java space(not originated from sub process)


> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631943
  
@itaifrenkel,

```
 When you say "the parent process fails" do you mean that the worker 
process is no longer running?
```
No, it means the case described by dashengju, that an exception is thrown 
in the java space(not originated from sub process)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196044#comment-14196044
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user itaifrenkel commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631559
  
@clockfly - Please clarify. When you say "the parent process fails" do you 
mean that the worker process is no longer running? If this is the case that 
process cannot call cleanup() since it is not running. We are using os specific 
hooks in the child process to receive signal when parent process dies. Another 
cross platform alternative is using file lock checks. Either way, are you sure 
this is related to this pull request ?


> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (STORM-456) Storm UI: cannot navigate to topology page when name contains spaces

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196043#comment-14196043
 ] 

ASF GitHub Bot commented on STORM-456:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61631456
  
@NareshKosgi,

Have you considered other special character that may used as topology name?
Like the table, #, &, (, and etc..

Is it possble that this fix only works for "space", but fails for "table"?

Another approach is to add strict name checking when submitting topology. 
This may make life easier?


> Storm UI: cannot navigate to topology page when name contains spaces
> 
>
> Key: STORM-456
> URL: https://issues.apache.org/jira/browse/STORM-456
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.1-incubating
> Environment: storm version: 0.9.1.2.1.2.0-402
> Firefox ESR 17.0.9 on RHEL
>Reporter: Steven Magana-Zook
>Priority: Trivial
>
> 1. Create a Java class that makes your topology
> 2. Submit the topology with a name that contains spaces: 
> StormSubmitter.submitTopology("I Dont Want to Use Underscores", conf, 
> builder.createTopology());
> 3. Submit the jar to storm: storm jar yourUberJar your.topology.class.name
> 4. Open Storm UI in your browser
> 5. Click the link for the submitted topology under "Topology Summary"
> Result: Page refreshes but does not show the topology page
> Expected Result: clicking the link should take you to the topology screen 
> like it does for topologies whose names do not contain spaces OR an error 
> should be returned if a user submits a topology whose names contains spaces 
> and this is not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[itaifrenkel]

Github user itaifrenkel commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631559
  
@clockfly - Please clarify. When you say "the parent process fails" do you 
mean that the worker process is no longer running? If this is the case that 
process cannot call cleanup() since it is not running. We are using os specific 
hooks in the child process to receive signal when parent process dies. Another 
cross platform alternative is using file lock checks. Either way, are you sure 
this is related to this pull request ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] storm pull request: STORM-456:Storm UI cannot navigate to topology...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/303#issuecomment-61631456
  
@NareshKosgi,

Have you considered other special character that may used as topology name?
Like the table, #, &, (, and etc..

Is it possble that this fix only works for "space", but fails for "table"?

Another approach is to add strict name checking when submitting topology. 
This may make life easier?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196040#comment-14196040
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631128
  

@dashengju 

When parent fails while child shell process is running correctly, should we 
call cleanup() to kill the suprocess first?

This has two benefits:
1. avoid zoombie sub process to exist
2. Avoid blocking at sub process errorstream reading.


> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Who is the contact point for the Storm Topology Visualization ?

[Itai Frenkel]

Hi,


We are dedicating resources (half time intern) to write our own storm topology 
visualization (graph) needed for internal use.

I would like to reach out to the code owner of the existing ui interactive 
graph, to see if we can align our efforts with other Storm community efforts 
(or at least align JS technologies).

Who is the contact person I'm looking for?


Thanks,

Itai

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[clockfly]

Github user clockfly commented on the pull request:

https://github.com/apache/storm/pull/305#issuecomment-61631128
  

@dashengju 

When parent fails while child shell process is running correctly, should we 
call cleanup() to kill the suprocess first?

This has two benefits:
1. avoid zoombie sub process to exist
2. Avoid blocking at sub process errorstream reading.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-442) multilang ShellBolt/ShellSpout die() can be hang when Exception happened

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196038#comment-14196038
 ] 

ASF GitHub Bot commented on STORM-442:
--

Github user clockfly commented on a diff in the pull request:

https://github.com/apache/storm/pull/305#discussion_r19798787
  
--- Diff: storm-core/src/jvm/backtype/storm/utils/ShellProcess.java ---
@@ -135,7 +135,14 @@ public void logErrorStream() {
 public String getErrorsString() {
 if (processErrorStream != null) {
 try {
-return IOUtils.toString(processErrorStream);
+StringBuilder sb = new StringBuilder();
+while (processErrorStream.available() > 0) {
+int bufferSize = processErrorStream.available();
--- End diff --

It is not safe to use available(). Check 
http://stackoverflow.com/questions/804951/is-it-possible-to-read-from-a-inputstream-with-a-timeout



> multilang ShellBolt/ShellSpout die() can be hang when Exception happened
> 
>
> Key: STORM-442
> URL: https://issues.apache.org/jira/browse/STORM-442
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: DashengJu
>
> In ShellBolt,  the _readerThread read command from python/shell process, and 
> handle like this:
>  try {
> ShellMsg shellMsg = _process.readShellMsg();
> ...
>  } catch (InterruptedException e) {
>  } catch (Throwable t) {
> die(t);
>  }
> And in the die function, getProcessTerminationInfoString will read 
> getErrorsString() from processErrorStream.
>  private void die(Throwable exception) {
>  
>  String processInfo = _process.getProcessInfoString() + 
> _process.getProcessTerminationInfoString();
>  
>  _exception = new RuntimeException(processInfo, exception);
>  
>  }
> so when ShellBolt got exception(for example, readShellMsg() throw NPE ) ,  
> but it is not an error from sub process,  then 
> getProcessTerminationInfoString will be hang because processErrorStream have 
> no data to read.
> On the other hand, as [~xiaokang] says ShellBolt should fail fast on 
> exception ( https://github.com/apache/incubator-storm/pull/46 ) , I think it 
> is not a good idea to read error info from stream.
> Because [~xiaokang] 's PR is based old version, so I will move his code to 
> this PR, and modify some other place in ShellSpout.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: [STORM-442] multilang ShellBolt/ShellSpout die...

[clockfly]

Github user clockfly commented on a diff in the pull request:

https://github.com/apache/storm/pull/305#discussion_r19798787
  
--- Diff: storm-core/src/jvm/backtype/storm/utils/ShellProcess.java ---
@@ -135,7 +135,14 @@ public void logErrorStream() {
 public String getErrorsString() {
 if (processErrorStream != null) {
 try {
-return IOUtils.toString(processErrorStream);
+StringBuilder sb = new StringBuilder();
+while (processErrorStream.available() > 0) {
+int bufferSize = processErrorStream.available();
--- End diff --

It is not safe to use available(). Check 
http://stackoverflow.com/questions/804951/is-it-possible-to-read-from-a-inputstream-with-a-timeout



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-535) setup 'java.library.path' for native-storm code if necessary

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195930#comment-14195930
 ] 

ASF GitHub Bot commented on STORM-535:
--

Github user caofangkun commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61609994
  
@clockfly 
the storm platform itself may need some native lib 
or common native lib may be provided by storm server side 


> setup 'java.library.path' for native-storm code if necessary
> 
>
> Key: STORM-535
> URL: https://issues.apache.org/jira/browse/STORM-535
> Project: Apache Storm
>  Issue Type: Improvement
>Reporter: caofangkun
>Priority: Minor
>
>  JAVA_LIBRARY_PATH=${STORM_HOME}/lib/native/${JAVA_PLATFORM}
> eg:
> If run storm on amd64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-amd64-64/libsigar-amd64-linux.so
> If run storm on x86_64 , then will add following into  JAVA_LIBRARY_PATH
> ${STORM_HOME}/lib/native/Linux-x86_64-64/libsigar-x86-linux.so



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] storm pull request: STORM-535:setup 'java.library.path' for native...

[caofangkun]

Github user caofangkun commented on the pull request:

https://github.com/apache/storm/pull/298#issuecomment-61609994
  
@clockfly 
the storm platform itself may need some native lib 
or common native lib may be provided by storm server side 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (STORM-537) A worker reconnects infinitely to another dead worker

[Sean Zhong (JIRA)]

[ 
https://issues.apache.org/jira/browse/STORM-537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195862#comment-14195862
 ] 

Sean Zhong commented on STORM-537:
--

Oh, with [] it can work, sorry.

> A worker reconnects infinitely to another dead worker
> -
>
> Key: STORM-537
> URL: https://issues.apache.org/jira/browse/STORM-537
> Project: Apache Storm
>  Issue Type: Bug
>Affects Versions: 0.9.3
>Reporter: Sergey Tryuber
>
> We're using 0.9.3-rc1. Most probably this wrong behavior was introduced as a 
> side efffect for STORM-409. When I kill a worker, another worker starts to 
> print messages like:
> {noformat}
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [0]
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [1]
> 2014-10-20 11:45:03 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [2]
> . so on
> {noformat}
> Then it reaches default 300 max_retries and starts the cycle again:
> {noformat}
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] connection established to a remote 
> host Netty-Client-:4706, [id: 
> 0xec088412, /:39795 :> :4706]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [0]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [1]
> 2014-10-20 11:54:38 b.s.m.n.Client [INFO] Reconnect started for 
> Netty-Client-:4706... [2]
> {noformat}
> And so on infinitely... 
> An issue most probably is in backtype.storm.messaging.netty.Client#connect 
> method in following place which determines that we give up on reconnection:
> {code}
> if (null != channel) {
> LOG.info("connection established to a remote host " + name() + ", " + 
> channel.toString());
> channelRef.set(channel);
> } else {
> close();
> throw new RuntimeException("Remote address is not reachable. We will 
> close this client " + name());
> }
> {code}
> I guess (not tried yet), that _channel_ object is not _null_ if this is a 
> real reconnection. So the method return a _channel_ object and then 
> reconnection starts again and again.
> This might be fixed by adding explicity *current = null;* into following code 
> block of the same method:
> {code}
> if (!future.isSuccess()) {
> if (null != current) {
> current.close();
> }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)