[GitHub] storm issue #2853: STORM-2963 Adding notes on GC to Performance.md

2018-09-30 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2853 i had already updated the PR yesterday before above comment ... so its ready. ---

[GitHub] storm issue #2853: STORM-2963 Adding notes on GC to Performance.md

2018-09-30 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2853 @HeartSaVioR ... sorry .. needed to add one more section to it. ---

[GitHub] storm pull request #2853: STORM-2963 Adding notes on GC to Performance.md

2018-09-28 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2853 STORM-2963 Adding notes on GC to Performance.md You can merge this pull request into a Git repository by running: $ git pull https://github.com/roshannaik/storm STORM-2963 Alternatively

[GitHub] storm issue #2829: STORM-3222: Fix KafkaSpout internals to use LinkedList in...

2018-09-16 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2829 @arunmahadevan .. need to be careful when that we are not doing too many emits in a single nextTuple()... it can cause a OOM situation by flooding the pendingQ. Worth checking if

[GitHub] storm issue #2829: STORM-3222: Fix KafkaSpout internals to use LinkedList in...

2018-09-16 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2829 @HeartSaVioR 1) pendingEmitsQ prevents nextTuple() from blocking when downstream queue is full. It holds the overflow emits (one or more) that occurred within a **single** nextTuple

[GitHub] storm issue #2832: STORM-3205 Optimization in TuplImpl

2018-09-15 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2832 travis failure seems unrelated. ---

[GitHub] storm pull request #2832: STORM-3205 Optimization in TuplImpl

2018-09-14 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2832 STORM-3205 Optimization in TuplImpl See [Jira](https://issues.apache.org/jira/browse/STORM-3205) description for measurements and details You can merge this pull request into a Git repository

[GitHub] storm issue #2711: STORM-3100 : Introducing CustomIndexArray to speedup Hash...

2018-08-02 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2711 Spent sometime trying to triage this failure but unable to figure out the cause. i know the issue has to do with this patch as it passes w/o this patch. I think it will take me some more time to

[GitHub] storm issue #2711: STORM-3100 : Introducing CustomIndexArray to speedup Hash...

2018-07-02 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2711 Sorry for not being able to address the comments sooner. Have been traveling and should be able to get back to this soon after jul 12 th. ---

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-23 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2711#discussion_r197605635 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/Task.java --- @@ -112,15 +112,15 @@ public Task(Executor executor, Integer taskId) throws

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-23 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2711#discussion_r197605410 --- Diff: examples/storm-elasticsearch-examples/src/main/java/org/apache/storm/elasticsearch/common/EsTestUtil.java --- @@ -56,7 +57,7 @@ public static

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-23 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2711#discussion_r197605381 --- Diff: examples/storm-elasticsearch-examples/src/main/java/org/apache/storm/elasticsearch/common/EsTestUtil.java --- @@ -56,7 +57,7 @@ public static

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-23 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2711#discussion_r197605298 --- Diff: storm-client/src/jvm/org/apache/storm/utils/CustomIndexArray.java --- @@ -0,0 +1,174 @@ +/* + * Licensed to the Apache Software

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-23 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2711#discussion_r197605303 --- Diff: storm-client/src/jvm/org/apache/storm/utils/CustomIndexArray.java --- @@ -0,0 +1,174 @@ +/* + * Licensed to the Apache Software

[GitHub] storm issue #2711: STORM-3100 : Introducing CustomIndexArray to speedup Hash...

2018-06-15 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2711 Looks like tester_bolt_metrics.py under storm-core is failing. I see that it can be run via mvn clojure:test under storm-core. But unable to figure out what really going wrong. Any help would be

[GitHub] storm pull request #2711: STORM-3100 : Introducing CustomIndexArray to speed...

2018-06-10 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2711 STORM-3100 : Introducing CustomIndexArray to speedup HashMap lookups - Added Unit Tests You can merge this pull request into a Git repository by running: $ git pull https://github.com

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-20 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r183200719 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -305,8 +305,9 @@ private void setupFlushTupleTimer(final Map

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-19 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182909146 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -305,8 +305,9 @@ private void setupFlushTupleTimer(final Map

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-18 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182628537 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -305,8 +305,9 @@ private void setupFlushTupleTimer(final Map

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-18 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182584203 --- Diff: storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java --- @@ -3032,9 +3019,19 @@ public void submitTopologyWithOpts(String

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-18 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182552073 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -305,8 +305,9 @@ private void setupFlushTupleTimer(final Map

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-18 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182549691 --- Diff: storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java --- @@ -3032,9 +3019,19 @@ public void submitTopologyWithOpts(String

[GitHub] storm pull request #2634: [STORM-3021] Fix wrong usages of Config.TOPOLOGY_W...

2018-04-18 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2634#discussion_r182545551 --- Diff: docs/Resource_Aware_Scheduler_overview.md --- @@ -184,6 +184,10 @@ The user can set some default configurations for the Resource Aware

[GitHub] storm issue #2000: STORM-2411 Setting topology.eventlogger.executors=0 in de...

2018-04-13 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2000 thanks. ---

[GitHub] storm issue #2000: STORM-2411 Setting topology.eventlogger.executors=0 in de...

2018-04-12 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2000 Good point. Will provide a fix. ---

[GitHub] storm issue #459: [STORM-2999] Add generics to static type of method paramet...

2018-03-19 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/459 @krichter722 thanks for this! ---

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-27 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 @Ethanlm can you please open a jira for this issue ? ---

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-26 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 Not very familiar with RAS and how it works . it seems to be default enabled ... Can you elaborate what setup was needed for RAS (if any) in your case ? ---

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-26 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 @Ethanlm should be able to take a look in a couple days. can you share any settings you are using outside in config file other than the one you noted on the cmd line (toplogy.debug=true) ? esp

[GitHub] storm pull request #2572: BUG 95921 - Support or HDFS 3.0

2018-02-22 Thread roshannaik
Github user roshannaik closed the pull request at: https://github.com/apache/storm/pull/2572 ---

[GitHub] storm pull request #2572: BUG 95921 - Support or HDFS 3.0

2018-02-22 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2572 BUG 95921 - Support or HDFS 3.0 fixes build and test issues You can merge this pull request into a Git repository by running: $ git pull https://github.com/roshannaik/storm BUG-95921

[GitHub] storm issue #2571: [STORM-2969] remove @isDerivedFrom validation for topolog...

2018-02-22 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2571 +1 Thank you. ---

[GitHub] storm issue #2571: [STORM-2969] remove @isDerivedFrom validation for topolog...

2018-02-22 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2571 Not clear why `@isDerivedFrom` wont be able to help validate inheritance of user defined classes from IWaitStrategy interface. Can you elaborate ? ---

[GitHub] storm issue #2561: STORM-2958 - Use new wait strategy model for Spout as wel...

2018-02-19 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2561 thanks for catching that. fixed it. ---

[GitHub] storm pull request #2561: STORM-2958 - Use new wait strategy model for Spout...

2018-02-18 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2561 STORM-2958 - Use new wait strategy model for Spout as well You can merge this pull request into a Git repository by running: $ git pull https://github.com/roshannaik/storm STORM-2958

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-16 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 I suspect this issue won’t show up on the setups I am using and will need to be triaged on your setup itself. Will work with you offline on that. BTW what version of Java are you using ? ---

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-16 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 @HeartSaVioR I am not seeing the reported excessive CPU usage behavior for 4 workers. First I tried taking some runs on my mid 2015 macbook pro (single cpu socket). Since no issue was seen

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-14 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 Nice feeling for me to see this merged in and I will look into the CPU usage issue reported by @HeartSaVioR. Just Wanted to specifically call out and thank the following folks who have been very

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-14 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 Commits Quashed. Thanks. Will use follow up jiras to address any issues discovered. ---

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-14 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 - Thanks for giving it a spin and posting the numbers. The higher CPU in multi-worker mode is not something I have seen before. Will take a look. Can you please share the cmd line that you used

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r168046999 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -22,19 +22,23 @@ import

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r168037754 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -22,19 +22,23 @@ import

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r168033940 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -22,19 +22,23 @@ import

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r168026607 --- Diff: storm-client/src/jvm/org/apache/storm/messaging/netty/BackPressureStatus.java --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-13 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 I think I have addressed all the major and minor issues as well. ---

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167814459 --- Diff: storm-client/src/jvm/org/apache/storm/messaging/netty/BackPressureStatus.java --- @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167807269 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -44,14 +48,15 @@ private final Boolean

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-13 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167800249 --- Diff: storm-client/src/jvm/org/apache/storm/trident/spout/RichSpoutBatchExecutor.java --- @@ -194,7 +194,12 @@ public void reportError(Throwable t

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167388018 --- Diff: storm-client/src/jvm/org/apache/storm/messaging/netty/BackPressureStatus.java --- @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167385890 --- Diff: storm-client/src/jvm/org/apache/storm/messaging/netty/BackPressureStatus.java --- @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167385590 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -44,14 +48,15 @@ private final Boolean

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167383283 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/Acker.java --- @@ -99,17 +100,19 @@ public void execute(Tuple input) { pending.put

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167381302 --- Diff: storm-client/src/jvm/org/apache/storm/cluster/IStormClusterState.java --- @@ -40,109 +40,117 @@ import org.apache.storm.nimbus.NimbusInfo

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-02-09 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r167379385 --- Diff: pom.xml --- @@ -259,6 +259,7 @@ 1.11 4.3.3 0.2.4 +2.0.1 --- End diff -- I dont

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-08 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 @revans2 and @HeartSaVioR Just rebased this PR and also included the fix for the making ExecutorTransfer.tryTransfer() thread safe to allow concurrent emits from background threads spun by

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-02-01 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 **Update:** - The code has been rebased to latest master. - Getting clean test runs on my local machine - Travis runs are indicating these test failures. 1. **storm

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-30 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r164956555 --- Diff: storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java --- @@ -17,72 +17,124 @@ */ package org.apache.storm.executor

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-30 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r164955129 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/BackPressureTracker.java --- @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-25 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r163967849 --- Diff: storm-client/src/jvm/org/apache/storm/cluster/ClusterUtils.java --- @@ -47,7 +47,6 @@ public static final String STORMS_ROOT = "s

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-21 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 **With Acking, Workers=1** TVL (rate=500k)| lat | throughput -- | -- | -- 1.x | 88.5ms | 52,913 v2-metrics | 91ms | 58,623 Const->Id->Null  | Throughput

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-21 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 **ACKers=0, Workers=1** Const->Id->Null | 1.x | v2-metrics | v2-no-cm -- | -- | -- | -- elapsed (sec) | Throughput | Throughput | Throughput 60 | 553,881 | 5

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-19 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 Just wanted to unblock things here ... will post summarized numbers from my runs by tomorrow here is a very brief summary. I ran both TVL and ConstSpoutIdBoltNullBoltTopo (with and

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-18 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 So far it seems only throughput impact has been measured. Impact on latency (if any) appears to be unknown. ---

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-18 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 Without getting some perf numbers for TVL and ConstSpoutIDBoltNullBoltTopo with ACK=0 and workers=1 we wont know for sure if there are perf regressions. ---

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161361250 --- Diff: storm-client/src/jvm/org/apache/storm/utils/ObjectReader.java --- @@ -76,6 +76,32 @@ public static Integer getInt(Object o, Integer

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161361184 --- Diff: storm-client/src/jvm/org/apache/storm/tuple/TupleImpl.java --- @@ -24,50 +24,46 @@ import org.apache.storm.task.GeneralTopologyContext

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161359459 --- Diff: storm-client/src/jvm/org/apache/storm/serialization/SerializationFactory.java --- @@ -20,6 +20,7 @@ import org.apache.storm.Config

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161359446 --- Diff: storm-client/src/jvm/org/apache/storm/executor/TupleInfo.java --- @@ -23,7 +23,7 @@ import java.io.Serializable; import java.util.List

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161358991 --- Diff: storm-client/src/jvm/org/apache/storm/executor/Executor.java --- @@ -225,51 +228,62 @@ private static String getExecutorType

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161358833 --- Diff: storm-client/src/jvm/org/apache/storm/executor/Executor.java --- @@ -225,51 +228,62 @@ private static String getExecutorType

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161358582 --- Diff: storm-client/src/jvm/org/apache/storm/executor/Executor.java --- @@ -196,19 +197,21 @@ public static Executor mkExecutor(WorkerState

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161358493 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -155,134 +150,159 @@ public void start() throws Exception

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161358387 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/metrics/SpoutThrottlingMetrics.java --- @@ -22,24 +22,25 @@ public class

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161357014 --- Diff: storm-client/src/jvm/org/apache/storm/StormTimer.java --- @@ -193,6 +210,24 @@ public void run

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-12 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r161356532 --- Diff: docs/Performance.md --- @@ -0,0 +1,132 @@ +--- --- End diff -- I am putting a link to this doc from Concepts.md. If you have

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-11 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 @HeartSaVioR to see the perf impact on throughput, you want to see what is its peak ability by setting a high rate... then see what is the highest rate each binary achieves. Here both seem to be

[GitHub] storm issue #2203: STORM-2153: New Metrics Reporting API

2018-01-11 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2203 @ptgoetz, I suggest measuring the perf impact with ACKers=0 as well (with workers=1). Having ACKing enabled or interworker-communication going on hides many perf issues. In addition

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-05 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r160011014 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/Task.java --- @@ -177,6 +195,35 @@ public BuiltinMetrics getBuiltInMetrics() { return

[GitHub] storm pull request #2502: new PR for STORM-2306 : Messaging subsystem redesi...

2018-01-05 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2502#discussion_r160010894 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutOutputCollectorImpl.java --- @@ -105,19 +134,16 @@ public void reportError(Throwable

[GitHub] storm issue #2502: new PR for STORM-2306 : Messaging subsystem redesign

2018-01-04 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2502 @revans2 , UTs for the storm-client module were fixed (including JCQueueTest). They run cleanly for me. Are you seeing them fail in your local setup ? **Rebasing:** Due to the length

[GitHub] storm issue #2241: STORM-2306 : Messaging subsystem redesign.

2018-01-04 Thread roshannaik
Github user roshannaik commented on the issue: https://github.com/apache/storm/pull/2241 This PR page has become unusable. Most of the time it doesn't load. If it does load, it is too slow to use...even for just typing a comment. Let stop using this and switch to the [ne

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2018-01-04 Thread roshannaik
Github user roshannaik closed the pull request at: https://github.com/apache/storm/pull/2241 ---

[GitHub] storm pull request #2502: new PR for STORM-2306

2018-01-04 Thread roshannaik
GitHub user roshannaik opened a pull request: https://github.com/apache/storm/pull/2502 new PR for STORM-2306 Since the [old PR page](https://github.com/apache/storm/pull/2241) had become unusable (due to lots of old comments). Creating a new PR for the same. You can merge this

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158942600 --- Diff: storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java --- @@ -79,6 +79,7 @@ protected final long

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158942500 --- Diff: storm-client/test/jvm/org/apache/storm/utils/JCQueueBackpressureTest.java --- @@ -0,0 +1,94 @@ +/** + * Licensed to the Apache Software

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158942187 --- Diff: storm-client/src/jvm/org/apache/storm/utils/JCQueue.java --- @@ -0,0 +1,457 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158941280 --- Diff: storm-client/src/jvm/org/apache/storm/utils/JCQueue.java --- @@ -0,0 +1,457 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158940938 --- Diff: storm-client/src/jvm/org/apache/storm/utils/JCQueue.java --- @@ -0,0 +1,457 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158940542 --- Diff: storm-client/src/jvm/org/apache/storm/utils/Utils.java --- @@ -1505,4 +1507,20 @@ public static StormTopology addVersions(StormTopology

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158940303 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutExecutor.java --- @@ -127,74 +141,130 @@ public void expire(Long key, TupleInfo

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158939356 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutExecutor.java --- @@ -127,74 +141,130 @@ public void expire(Long key, TupleInfo

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158939120 --- Diff: storm-client/src/jvm/org/apache/storm/executor/spout/SpoutExecutor.java --- @@ -127,74 +141,130 @@ public void expire(Long key, TupleInfo

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158937490 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/WorkerState.java --- @@ -439,29 +436,40 @@ public void refreshStormActive(Runnable callback

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158937298 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -442,32 +406,29 @@ private WorkerBackpressureCallback

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158937185 --- Diff: storm-client/src/jvm/org/apache/storm/daemon/worker/Worker.java --- @@ -155,134 +150,159 @@ public void start() throws Exception

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158936048 --- Diff: storm-client/test/jvm/org/apache/storm/utils/JCQueueTest.java --- @@ -109,58 +125,61 @@ public void onEvent(Object obj, long sequence, boolean

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158934969 --- Diff: storm-client/src/jvm/org/apache/storm/tuple/TupleImpl.java --- @@ -24,50 +24,56 @@ import org.apache.storm.task.GeneralTopologyContext

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158931899 --- Diff: examples/storm-perf/src/main/java/org/apache/storm/perf/JCToolsPerfTest.java --- @@ -0,0 +1,221 @@ +/* + * Licensed to the Apache

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158932246 --- Diff: storm-client/src/jvm/org/apache/storm/executor/ExecutorTransfer.java --- @@ -17,72 +17,123 @@ */ package org.apache.storm.executor

[GitHub] storm pull request #2241: STORM-2306 : Messaging subsystem redesign.

2017-12-28 Thread roshannaik
Github user roshannaik commented on a diff in the pull request: https://github.com/apache/storm/pull/2241#discussion_r158928104 --- Diff: storm-client/src/jvm/org/apache/storm/executor/bolt/BoltExecutor.java --- @@ -18,105 +18,188 @@ package org.apache.storm.executor.bolt

  1   2   3   4   >