[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103777141 I'm also seeing a performance regression. IMO resolving issue could introduce performance drop slightly, but shouldn't too much. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14551893#comment-14551893 ] ASF GitHub Bot commented on STORM-737: -- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103777141 I'm also seeing a performance regression. IMO resolving issue could introduce performance drop slightly, but shouldn't too much. Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103843197 Since current approach do more works within read lock, it may be necessary to test performance with previous approach, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-710) bin/storm command list out all the classes in the output for a command
[ https://issues.apache.org/jira/browse/STORM-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552163#comment-14552163 ] Oleg Ostashchuk commented on STORM-710: --- Hello, I've modified the output of the bin/storm commands according to the description. Now the original long outputs can be displayed by running commands with -v flag. Can somebody check this and give me please some feedback. Thanks! https://github.com/oleg03/storm/tree/test/ bin/storm command list out all the classes in the output for a command -- Key: STORM-710 URL: https://issues.apache.org/jira/browse/STORM-710 Project: Apache Storm Issue Type: Bug Reporter: Sriharsha Chintalapani Assignee: Oleg Ostashchuk Priority: Minor Labels: newbie when running bin/storm list command or any other command it outputs Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT -Dstorm.log.dir=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp
[jira] [Commented] (STORM-827) storm-hdfs requires you to ship a keytab to access secure HDFS
[ https://issues.apache.org/jira/browse/STORM-827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552353#comment-14552353 ] ASF GitHub Bot commented on STORM-827: -- Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/558#issuecomment-103900579 @Parth-Brahmbhatt I did run with this on one of our secure clusters. It was not exactly the same code, as our version is older and does not have the AutoHDFS in it. Which is why I originally filed the JIRA incorrectly and said it should support that too. storm-hdfs requires you to ship a keytab to access secure HDFS -- Key: STORM-827 URL: https://issues.apache.org/jira/browse/STORM-827 Project: Apache Storm Issue Type: Bug Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Priority: Critical storm-hdfs to access secure hdfs assumes that you have to use a keytab to access HDFS. It should be able to work with either AutoTGT or AutoHDFS too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: STORM-827: Allow AutoTGT to work with storm-hd...
Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/558#issuecomment-103900579 @Parth-Brahmbhatt I did run with this on one of our secure clusters. It was not exactly the same code, as our version is older and does not have the AutoHDFS in it. Which is why I originally filed the JIRA incorrectly and said it should support that too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552373#comment-14552373 ] ASF GitHub Bot commented on STORM-737: -- Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103904248 I'm definitely seeing a performance regression. In a core storm topology I'm seeing a drop in throughput and slight increase in latency. In a trident topology I'm seeing a significant increase in latency (2x). I'll try testing with #521. Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552495#comment-14552495 ] ASF GitHub Bot commented on STORM-737: -- Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103928314 Testing with #521 applied to 0.10.x-branch I'm actually seeing a performance ***improvement***. With core storm topologies there's an increase in throughput and a small reduction in latency. With trident, throughput is the same and latency is slightly lower. I'll run my fault tolerance tests next to see if there's any change in behavior Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103928314 Testing with #521 applied to 0.10.x-branch I'm actually seeing a performance ***improvement***. With core storm topologies there's an increase in throughput and a small reduction in latency. With trident, throughput is the same and latency is slightly lower. I'll run my fault tolerance tests next to see if there's any change in behavior --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Issue Comment Deleted] (STORM-710) bin/storm command list out all the classes in the output for a command
[ https://issues.apache.org/jira/browse/STORM-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Ostashchuk updated STORM-710: -- Comment: was deleted (was: Hello, I've modified the output of the bin/storm commands according to the description. Now the original long outputs can be displayed by running commands with -v flag. Can somebody check this and give me please some feedback. Thanks! https://github.com/oleg03/storm/tree/test/) bin/storm command list out all the classes in the output for a command -- Key: STORM-710 URL: https://issues.apache.org/jira/browse/STORM-710 Project: Apache Storm Issue Type: Bug Reporter: Sriharsha Chintalapani Assignee: Oleg Ostashchuk Priority: Minor Labels: newbie when running bin/storm list command or any other command it outputs Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT -Dstorm.log.dir=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/521#issuecomment-104045430 @d2r @ptgoetz Applied @d2r's comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553164#comment-14553164 ] ASF GitHub Bot commented on STORM-737: -- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/521#issuecomment-104045430 @d2r @ptgoetz Applied @d2r's comments. Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553109#comment-14553109 ] ASF GitHub Bot commented on STORM-737: -- Github user HeartSaVioR commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-104039348 @ptgoetz Thanks for testing! I'll move on to #521. Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[
https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553138#comment-14553138
]
ASF GitHub Bot commented on STORM-737:
--
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/521#discussion_r30750154
--- Diff: storm-core/src/jvm/backtype/storm/utils/TransferDrainer.java ---
@@ -23,40 +23,62 @@
import backtype.storm.messaging.IConnection;
import backtype.storm.messaging.TaskMessage;
+import com.google.common.collect.Maps;
public class TransferDrainer {
- private HashMapString, ArrayListArrayListTaskMessage bundles = new
HashMap();
+ private HashMapInteger, ArrayListArrayListTaskMessage bundles =
new HashMap();
- public void add(HashMapString, ArrayListTaskMessage
workerTupleSetMap) {
-for (String key : workerTupleSetMap.keySet()) {
-
- ArrayListArrayListTaskMessage bundle = bundles.get(key);
- if (null == bundle) {
-bundle = new ArrayListArrayListTaskMessage();
-bundles.put(key, bundle);
- }
-
- ArrayList tupleSet = workerTupleSetMap.get(key);
- if (null != tupleSet tupleSet.size() 0) {
-bundle.add(tupleSet);
- }
-}
+ public void add(HashMapInteger, ArrayListTaskMessage
taskTupleSetMap) {
+for (Integer task : taskTupleSetMap.keySet()) {
+ addListRefToMap(this.bundles, task, taskTupleSetMap.get(task));
+}
}
- public void send(HashMapString, IConnection connections) {
-for (String hostPort : bundles.keySet()) {
- IConnection connection = connections.get(hostPort);
- if (null != connection) {
-ArrayListArrayListTaskMessage bundle = bundles.get(hostPort);
-IteratorTaskMessage iter = getBundleIterator(bundle);
-if (null != iter iter.hasNext()) {
- connection.send(iter);
+ public void send(HashMapInteger, String taskToNode, HashMapString,
IConnection connections) {
+HashMapString, ArrayListArrayListTaskMessage
bundleMapByDestination = groupBundleByDestination(taskToNode);
+
+for (String hostPort : bundleMapByDestination.keySet()) {
+ if (hostPort != null) {
--- End diff --
@d2r
No, we removed it into groupBundleByDestination(), so we don't need to
check again. Thanks!
Workers may try to send to closed connections
-
Key: STORM-737
URL: https://issues.apache.org/jira/browse/STORM-737
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.2-incubating
Reporter: Derek Dagit
There is a race condition in the worker code that can allow for a send() to
be called on a closed connection.
[Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672]
The assignment mapping from task - node+port needs to be read and used in
the read lock when sending, so that an accurate mapping is used that does not
include any connections that are closed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[
https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14553139#comment-14553139
]
ASF GitHub Bot commented on STORM-737:
--
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/521#discussion_r30750199
--- Diff: storm-core/src/jvm/backtype/storm/utils/TransferDrainer.java ---
@@ -23,40 +23,62 @@
import backtype.storm.messaging.IConnection;
import backtype.storm.messaging.TaskMessage;
+import com.google.common.collect.Maps;
public class TransferDrainer {
- private HashMapString, ArrayListArrayListTaskMessage bundles = new
HashMap();
+ private HashMapInteger, ArrayListArrayListTaskMessage bundles =
new HashMap();
- public void add(HashMapString, ArrayListTaskMessage
workerTupleSetMap) {
-for (String key : workerTupleSetMap.keySet()) {
-
- ArrayListArrayListTaskMessage bundle = bundles.get(key);
- if (null == bundle) {
-bundle = new ArrayListArrayListTaskMessage();
-bundles.put(key, bundle);
- }
-
- ArrayList tupleSet = workerTupleSetMap.get(key);
- if (null != tupleSet tupleSet.size() 0) {
-bundle.add(tupleSet);
- }
-}
+ public void add(HashMapInteger, ArrayListTaskMessage
taskTupleSetMap) {
+for (Integer task : taskTupleSetMap.keySet()) {
+ addListRefToMap(this.bundles, task, taskTupleSetMap.get(task));
+}
}
- public void send(HashMapString, IConnection connections) {
-for (String hostPort : bundles.keySet()) {
- IConnection connection = connections.get(hostPort);
- if (null != connection) {
-ArrayListArrayListTaskMessage bundle = bundles.get(hostPort);
-IteratorTaskMessage iter = getBundleIterator(bundle);
-if (null != iter iter.hasNext()) {
- connection.send(iter);
+ public void send(HashMapInteger, String taskToNode, HashMapString,
IConnection connections) {
+HashMapString, ArrayListArrayListTaskMessage
bundleMapByDestination = groupBundleByDestination(taskToNode);
+
+for (String hostPort : bundleMapByDestination.keySet()) {
+ if (hostPort != null) {
+IConnection connection = connections.get(hostPort);
+if (null != connection) {
+ ArrayListArrayListTaskMessage bundle =
bundleMapByDestination.get(hostPort);
+ IteratorTaskMessage iter = getBundleIterator(bundle);
+ if (null != iter iter.hasNext()) {
+connection.send(iter);
+ }
}
}
-}
+}
}
-
+
+ private HashMapString, ArrayListArrayListTaskMessage
groupBundleByDestination(HashMapInteger, String taskToNode) {
+HashMapString, ArrayListArrayListTaskMessage bundleMap =
Maps.newHashMap();
+for (Integer task : this.bundles.keySet()) {
+ String hostPort = taskToNode.get(task);
+ if (hostPort != null) {
+for (ArrayListTaskMessage chunk : this.bundles.get(task)) {
+ addListRefToMap(bundleMap, hostPort, chunk);
+}
+ }
+}
+return bundleMap;
+ }
+
+ private T void addListRefToMap(HashMapT,
ArrayListArrayListTaskMessage bundles,
--- End diff --
@d2r OK, it's better than shadowing fields. Thanks!
Workers may try to send to closed connections
-
Key: STORM-737
URL: https://issues.apache.org/jira/browse/STORM-737
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.9.2-incubating
Reporter: Derek Dagit
There is a race condition in the worker code that can allow for a send() to
be called on a closed connection.
[Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672]
The assignment mapping from task - node+port needs to be read and used in
the read lock when sending, so that an accurate mapping is used that does not
include any connections that are closed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/521#discussion_r30750199
--- Diff: storm-core/src/jvm/backtype/storm/utils/TransferDrainer.java ---
@@ -23,40 +23,62 @@
import backtype.storm.messaging.IConnection;
import backtype.storm.messaging.TaskMessage;
+import com.google.common.collect.Maps;
public class TransferDrainer {
- private HashMapString, ArrayListArrayListTaskMessage bundles = new
HashMap();
+ private HashMapInteger, ArrayListArrayListTaskMessage bundles =
new HashMap();
- public void add(HashMapString, ArrayListTaskMessage
workerTupleSetMap) {
-for (String key : workerTupleSetMap.keySet()) {
-
- ArrayListArrayListTaskMessage bundle = bundles.get(key);
- if (null == bundle) {
-bundle = new ArrayListArrayListTaskMessage();
-bundles.put(key, bundle);
- }
-
- ArrayList tupleSet = workerTupleSetMap.get(key);
- if (null != tupleSet tupleSet.size() 0) {
-bundle.add(tupleSet);
- }
-}
+ public void add(HashMapInteger, ArrayListTaskMessage
taskTupleSetMap) {
+for (Integer task : taskTupleSetMap.keySet()) {
+ addListRefToMap(this.bundles, task, taskTupleSetMap.get(task));
+}
}
- public void send(HashMapString, IConnection connections) {
-for (String hostPort : bundles.keySet()) {
- IConnection connection = connections.get(hostPort);
- if (null != connection) {
-ArrayListArrayListTaskMessage bundle = bundles.get(hostPort);
-IteratorTaskMessage iter = getBundleIterator(bundle);
-if (null != iter iter.hasNext()) {
- connection.send(iter);
+ public void send(HashMapInteger, String taskToNode, HashMapString,
IConnection connections) {
+HashMapString, ArrayListArrayListTaskMessage
bundleMapByDestination = groupBundleByDestination(taskToNode);
+
+for (String hostPort : bundleMapByDestination.keySet()) {
+ if (hostPort != null) {
+IConnection connection = connections.get(hostPort);
+if (null != connection) {
+ ArrayListArrayListTaskMessage bundle =
bundleMapByDestination.get(hostPort);
+ IteratorTaskMessage iter = getBundleIterator(bundle);
+ if (null != iter iter.hasNext()) {
+connection.send(iter);
+ }
}
}
-}
+}
}
-
+
+ private HashMapString, ArrayListArrayListTaskMessage
groupBundleByDestination(HashMapInteger, String taskToNode) {
+HashMapString, ArrayListArrayListTaskMessage bundleMap =
Maps.newHashMap();
+for (Integer task : this.bundles.keySet()) {
+ String hostPort = taskToNode.get(task);
+ if (hostPort != null) {
+for (ArrayListTaskMessage chunk : this.bundles.get(task)) {
+ addListRefToMap(bundleMap, hostPort, chunk);
+}
+ }
+}
+return bundleMap;
+ }
+
+ private T void addListRefToMap(HashMapT,
ArrayListArrayListTaskMessage bundles,
--- End diff --
@d2r OK, it's better than shadowing fields. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/521#discussion_r30750154
--- Diff: storm-core/src/jvm/backtype/storm/utils/TransferDrainer.java ---
@@ -23,40 +23,62 @@
import backtype.storm.messaging.IConnection;
import backtype.storm.messaging.TaskMessage;
+import com.google.common.collect.Maps;
public class TransferDrainer {
- private HashMapString, ArrayListArrayListTaskMessage bundles = new
HashMap();
+ private HashMapInteger, ArrayListArrayListTaskMessage bundles =
new HashMap();
- public void add(HashMapString, ArrayListTaskMessage
workerTupleSetMap) {
-for (String key : workerTupleSetMap.keySet()) {
-
- ArrayListArrayListTaskMessage bundle = bundles.get(key);
- if (null == bundle) {
-bundle = new ArrayListArrayListTaskMessage();
-bundles.put(key, bundle);
- }
-
- ArrayList tupleSet = workerTupleSetMap.get(key);
- if (null != tupleSet tupleSet.size() 0) {
-bundle.add(tupleSet);
- }
-}
+ public void add(HashMapInteger, ArrayListTaskMessage
taskTupleSetMap) {
+for (Integer task : taskTupleSetMap.keySet()) {
+ addListRefToMap(this.bundles, task, taskTupleSetMap.get(task));
+}
}
- public void send(HashMapString, IConnection connections) {
-for (String hostPort : bundles.keySet()) {
- IConnection connection = connections.get(hostPort);
- if (null != connection) {
-ArrayListArrayListTaskMessage bundle = bundles.get(hostPort);
-IteratorTaskMessage iter = getBundleIterator(bundle);
-if (null != iter iter.hasNext()) {
- connection.send(iter);
+ public void send(HashMapInteger, String taskToNode, HashMapString,
IConnection connections) {
+HashMapString, ArrayListArrayListTaskMessage
bundleMapByDestination = groupBundleByDestination(taskToNode);
+
+for (String hostPort : bundleMapByDestination.keySet()) {
+ if (hostPort != null) {
--- End diff --
@d2r
No, we removed it into groupBundleByDestination(), so we don't need to
check again. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-737] Check task-node+port with read lo...
Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103992667 @HeartSaVioR, #521 passed my fault tolerance test (which randomly kills workers and tests for data loss). I'd suggest closing this pull request, and reopening #521 to address @d2r's comments there. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-710) bin/storm command list out all the classes in the output for a command
[ https://issues.apache.org/jira/browse/STORM-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552919#comment-14552919 ] Oleg Ostashchuk commented on STORM-710: --- Hello, [~sriharsha] I've modified the output of the bin/storm commands according to the description. Now the original long outputs can be displayed by running commands with -v flag. Can somebody check this please and give me some feedback. Thanks! https://github.com/oleg03/storm/tree/test/ bin/storm command list out all the classes in the output for a command -- Key: STORM-710 URL: https://issues.apache.org/jira/browse/STORM-710 Project: Apache Storm Issue Type: Bug Reporter: Sriharsha Chintalapani Assignee: Oleg Ostashchuk Priority: Minor Labels: newbie when running bin/storm list command or any other command it outputs Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT -Dstorm.log.dir=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp
[jira] [Commented] (STORM-737) Workers may try to send to closed connections
[ https://issues.apache.org/jira/browse/STORM-737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552858#comment-14552858 ] ASF GitHub Bot commented on STORM-737: -- Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/557#issuecomment-103992667 @HeartSaVioR, #521 passed my fault tolerance test (which randomly kills workers and tests for data loss). I'd suggest closing this pull request, and reopening #521 to address @d2r's comments there. Workers may try to send to closed connections - Key: STORM-737 URL: https://issues.apache.org/jira/browse/STORM-737 Project: Apache Storm Issue Type: Bug Affects Versions: 0.9.2-incubating Reporter: Derek Dagit There is a race condition in the worker code that can allow for a send() to be called on a closed connection. [Discussion|https://github.com/apache/storm/pull/349#issuecomment-87778672] The assignment mapping from task - node+port needs to be read and used in the read lock when sending, so that an accurate mapping is used that does not include any connections that are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-831) Add Jira and Central Logging URL to UI
Kishor Patil created STORM-831: -- Summary: Add Jira and Central Logging URL to UI Key: STORM-831 URL: https://issues.apache.org/jira/browse/STORM-831 Project: Apache Storm Issue Type: Improvement Reporter: Kishor Patil Assignee: Kishor Patil Priority: Trivial As a user, I would like to see a link to take me to JIRA for reporting bug. Also, optionally if link to splunk/logstash/kibana from UI would be helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-831) Add Jira and Central Logging URL to UI
[ https://issues.apache.org/jira/browse/STORM-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14552882#comment-14552882 ] ASF GitHub Bot commented on STORM-831: -- GitHub user kishorvpatil opened a pull request: https://github.com/apache/storm/pull/559 [STORM-831] Adding jira and central logging links to UI To main UI page - Add link to create JIRA page - Optional link to central logging  Both links can be configured to custom service desk link. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kishorvpatil/incubator-storm storm831 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/559.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #559 commit 5ba0e62a51bb73ed99c041c336f3877970513dae Author: Kishor Patil kpa...@yahoo-inc.com Date: 2015-05-20T18:52:30Z Adding jira and central logging links to UI Add Jira and Central Logging URL to UI -- Key: STORM-831 URL: https://issues.apache.org/jira/browse/STORM-831 Project: Apache Storm Issue Type: Improvement Reporter: Kishor Patil Assignee: Kishor Patil Priority: Trivial As a user, I would like to see a link to take me to JIRA for reporting bug. Also, optionally if link to splunk/logstash/kibana from UI would be helpful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: Storm-818: storm-eventhubs configuration impro...
Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/552#issuecomment-104067426 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
