[GitHub] storm pull request: Flush stdout and stderr before calling "os.exe...
GitHub user dsKarthick opened a pull request: https://github.com/apache/storm/pull/1360 Flush stdout and stderr before calling "os.execvp" to prevent log loss. bin/storm.py emits the following crucial information that is lost because we don't flush the stdout before exec. ``` 2016-04-25T08:23:43.17141 Running: java -server -Dstorm.options= -Dstorm.home= -Xmx1024m -Dlogfile.name=nimbus.log -Dlogback.configurationFile=logback/cluster.xml backtype.storm.ui.core.nimbus ``` Observed Environment: OS: `CentOS release 6.5` Kernel: `2.6.32-431.el6.x86_64` Python version: `Python 2.7.2` You can merge this pull request into a Git repository by running: $ git pull https://github.com/dsKarthick/storm-1 flush_launch_command_to_stderr Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1360.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1360 commit 17fee522dbcbd4f0e00f886cf17094c761f9831c Author: Karthick Duraisamy Soundararaj Date: 2016-04-25T08:54:39Z Flush stdout and stderr before calling "os.execvp" to prevent log loss. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Flush stdout and stderr before calling "os.exe...
Github user dsKarthick commented on the pull request: https://github.com/apache/storm/pull/1360#issuecomment-214236815 @HeartSaVioR @arunmahadevan @revans2 @ptgoetz @vesense Could one of you guys please take a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: Flush stdout and stderr before calling "os.exe...
Github user arunmahadevan commented on the pull request: https://github.com/apache/storm/pull/1360#issuecomment-214272514 @dsKarthick the change looks good. Can you also raise a jira (at https://issues.apache.org/jira/browse/STORM) and update the PR title ? This would likely be an issue only when you redirect the stdout of "storm jar" command to a file. When stdout is a terminal its unbufferred and stderr is always unbufferred. You might want to update your commit message accordingly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-1723) Introduce ClusterMetricsConsumer
[ https://issues.apache.org/jira/browse/STORM-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256402#comment-15256402 ] ASF GitHub Bot commented on STORM-1723: --- Github user unsleepy22 commented on the pull request: https://github.com/apache/storm/pull/1352#issuecomment-214360458 +1 > Introduce ClusterMetricsConsumer > > > Key: STORM-1723 > URL: https://issues.apache.org/jira/browse/STORM-1723 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Affects Versions: 2.0.0, 1.0.1 >Reporter: Jungtaek Lim >Assignee: Jungtaek Lim > > NOTE: This issue is already discussed shortly. Please refer > [here|http://mail-archives.apache.org/mod_mbox/storm-dev/201604.mbox/%3CCAF5108hDCcMKxLXKUYLReOoKkNNdgW2YudweR+mKr=1hlsl...@mail.gmail.com%3E] > for details. > This issue focuses to introduce ClusterMetricsConsumer and provide interface > to let users plugin their consumers. > ClusterMetricsConsumers will be attached to Nimbus, and leader of Nimbus will > push cluster related metrics to ClusterMetricsConsumer. > Requirements of ClusterMetricsConsumer are here: > - Only leader of Nimbus should publish cluster metrics to consumer. > - Nimbus shouldn't be affected by crashing or heavy latency on consumer. > - Consumer should have resilient when crashing or Nimbus should take care of. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: STORM-1723 (1.x) Introduce ClusterMetricsConsu...
Github user unsleepy22 commented on the pull request: https://github.com/apache/storm/pull/1352#issuecomment-214360458 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: STORM-1725: Kafka Spout New Consumer API - Kaf...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1357 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-1725) Kafka Spout New Consumer API - KafkaSpoutRetryExponentialBackoff method should use HashMap instead of TreeMap not to throw Exception
[ https://issues.apache.org/jira/browse/STORM-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256806#comment-15256806 ] ASF GitHub Bot commented on STORM-1725: --- Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1357 > Kafka Spout New Consumer API - KafkaSpoutRetryExponentialBackoff method > should use HashMap instead of TreeMap not to throw Exception > > > Key: STORM-1725 > URL: https://issues.apache.org/jira/browse/STORM-1725 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka >Affects Versions: 1.0.0 >Reporter: Hugo Louro >Assignee: Hugo Louro >Priority: Blocker > Fix For: 1.0.1 > > > The method > org.apache.storm.kafka.spout.KafkaSpoutRetryExponentialBackoff#retriableTopicPartitions > currently uses TreeMap but TopicPartition does not implement Comparable. > Since ordering is not important, HashMap is the most appropriate data > structure here, and the one I meant to use to begin with. > The fix will replace TreeMap with HashMap. > This issue was found by @jianbzhou and credit should be given to him. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1725) Kafka Spout New Consumer API - KafkaSpoutRetryExponentialBackoff method should use HashMap instead of TreeMap not to throw Exception
[ https://issues.apache.org/jira/browse/STORM-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256808#comment-15256808 ] P. Taylor Goetz commented on STORM-1725: Thanks [~hmclouro]. Merged. > Kafka Spout New Consumer API - KafkaSpoutRetryExponentialBackoff method > should use HashMap instead of TreeMap not to throw Exception > > > Key: STORM-1725 > URL: https://issues.apache.org/jira/browse/STORM-1725 > Project: Apache Storm > Issue Type: Bug > Components: storm-kafka >Affects Versions: 1.0.0 >Reporter: Hugo Louro >Assignee: Hugo Louro >Priority: Blocker > Fix For: 1.0.1 > > > The method > org.apache.storm.kafka.spout.KafkaSpoutRetryExponentialBackoff#retriableTopicPartitions > currently uses TreeMap but TopicPartition does not implement Comparable. > Since ordering is not important, HashMap is the most appropriate data > structure here, and the one I meant to use to begin with. > The fix will replace TreeMap with HashMap. > This issue was found by @jianbzhou and credit should be given to him. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1535) Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple bolts/executors
[ https://issues.apache.org/jira/browse/STORM-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256819#comment-15256819 ] ASF GitHub Bot commented on STORM-1535: --- Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1351 > Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple > bolts/executors > --- > > Key: STORM-1535 > URL: https://issues.apache.org/jira/browse/STORM-1535 > Project: Apache Storm > Issue Type: Bug > Components: storm-hdfs >Affects Versions: 0.10.0, 0.9.5 >Reporter: Dan Bahir >Assignee: Priyank Shah > > When logging in with a keytab, if the topology has more than one instance of > an HDFS bolt then the ticket might not be automatically renewed. > Bug has been fixed for HBase bolt, code is slightly different for HDFS bolt > and therefore requires further investigation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] storm pull request: STORM-1535: Make sure hdfs key tab login happe...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1351 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: STORM-1535: Make sure hdfs key tab login happe...
Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/1351#issuecomment-214497033 Thanks @priyank5485. Merged to master, 1.x-branch and 0.10.x-branch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-1535) Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple bolts/executors
[ https://issues.apache.org/jira/browse/STORM-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256890#comment-15256890 ] ASF GitHub Bot commented on STORM-1535: --- Github user ptgoetz commented on the pull request: https://github.com/apache/storm/pull/1351#issuecomment-214497033 Thanks @priyank5485. Merged to master, 1.x-branch and 0.10.x-branch. > Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple > bolts/executors > --- > > Key: STORM-1535 > URL: https://issues.apache.org/jira/browse/STORM-1535 > Project: Apache Storm > Issue Type: Bug > Components: storm-hdfs >Affects Versions: 0.10.0, 0.9.5 >Reporter: Dan Bahir >Assignee: Priyank Shah > > When logging in with a keytab, if the topology has more than one instance of > an HDFS bolt then the ticket might not be automatically renewed. > Bug has been fixed for HBase bolt, code is slightly different for HDFS bolt > and therefore requires further investigation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1535) Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple bolts/executors
[ https://issues.apache.org/jira/browse/STORM-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] P. Taylor Goetz resolved STORM-1535. Resolution: Fixed Fix Version/s: 1.0.1 2.0.0 0.10.1 > Storm-HDFS - When using keytab Kerberos ticket is not renewed with multiple > bolts/executors > --- > > Key: STORM-1535 > URL: https://issues.apache.org/jira/browse/STORM-1535 > Project: Apache Storm > Issue Type: Bug > Components: storm-hdfs >Affects Versions: 0.10.0, 0.9.5 >Reporter: Dan Bahir >Assignee: Priyank Shah > Fix For: 0.10.1, 2.0.0, 1.0.1 > > > When logging in with a keytab, if the topology has more than one instance of > an HDFS bolt then the ticket might not be automatically renewed. > Bug has been fixed for HBase bolt, code is slightly different for HDFS bolt > and therefore requires further investigation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-834) Class cast exception Object to Iterable
[ https://issues.apache.org/jira/browse/STORM-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15256978#comment-15256978 ] Bryan Baugher commented on STORM-834: - I've actually seen this issue when I had slf4j-log4j12 in my topology which seems to conflict with log4j-slf4j-impl from storm. Excluding it fixed my issue. > Class cast exception Object to Iterable > --- > > Key: STORM-834 > URL: https://issues.apache.org/jira/browse/STORM-834 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 0.9.3, 0.10.0, 0.9.6 > Environment: Debian linux >Reporter: Jason Kania > > This exception is happening after some hours of running. Unfortunately, > nothing in the trace suggests the source of the error for further > investigation. > 2015-05-23T13:24:22.343-0400 b.s.util [ERROR] Async loop died! > java.lang.RuntimeException: java.lang.ClassCastException: java.lang.Object > cannot be cast to java.lang.Iterable > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748) > ~[storm-core-0.9.3.jar:0.9.3] > at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) > ~[storm-core-0.9.3.jar:0.9.3] > at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] > Caused by: java.lang.ClassCastException: java.lang.Object cannot be cast to > java.lang.Iterable > at backtype.storm.util$get_iterator.invoke(util.clj:867) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:397) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:120) > ~[storm-core-0.9.3.jar:0.9.3] > ... 6 common frames omitted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-834) Class cast exception Object to Iterable
[ https://issues.apache.org/jira/browse/STORM-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257359#comment-15257359 ] John Fang commented on STORM-834: - [~lonerzzz] In fact I come into the same problem before, but it haven't happened after I rebuild the topology and rebuild the latest storm. > Class cast exception Object to Iterable > --- > > Key: STORM-834 > URL: https://issues.apache.org/jira/browse/STORM-834 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 0.9.3, 0.10.0, 0.9.6 > Environment: Debian linux >Reporter: Jason Kania > > This exception is happening after some hours of running. Unfortunately, > nothing in the trace suggests the source of the error for further > investigation. > 2015-05-23T13:24:22.343-0400 b.s.util [ERROR] Async loop died! > java.lang.RuntimeException: java.lang.ClassCastException: java.lang.Object > cannot be cast to java.lang.Iterable > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.daemon.executor$fn__3441$fn__3453$fn__3500.invoke(executor.clj:748) > ~[storm-core-0.9.3.jar:0.9.3] > at backtype.storm.util$async_loop$fn__464.invoke(util.clj:463) > ~[storm-core-0.9.3.jar:0.9.3] > at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75] > Caused by: java.lang.ClassCastException: java.lang.Object cannot be cast to > java.lang.Iterable > at backtype.storm.util$get_iterator.invoke(util.clj:867) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.daemon.executor$mk_task_receiver$fn__3364.invoke(executor.clj:397) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.disruptor$clojure_handler$reify__1447.onEvent(disruptor.clj:58) > ~[storm-core-0.9.3.jar:0.9.3] > at > backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:120) > ~[storm-core-0.9.3.jar:0.9.3] > ... 6 common frames omitted -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1729) Get rid of reflections while recording stats
Jungtaek Lim created STORM-1729:
---
Summary: Get rid of reflections while recording stats
Key: STORM-1729
URL: https://issues.apache.org/jira/browse/STORM-1729
Project: Apache Storm
Issue Type: Bug
Components: storm-core
Affects Versions: 1.0.1
Reporter: Jungtaek Lim
Assignee: Jungtaek Lim
I don't add this to 2.0.0 since it only occurs on Clojure.
{code}
(set! *warn-on-reflection* true)
(load-file "src/clj/org/apache/storm/stats.clj")
{code}
{quote}
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
- call to method record can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
- call to method incBy can't be resolved (target class is unknown).
{quote}
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
We expect there's no reflection since we give a type hint while calling, but it
doesn't work.
(I don't know why it doesn't work. Does generic type matter?)
Anyway, defining them into let makes them avoid reflection.
Even though we sample while recording stats, it does hurt performance with fast
& high-traffic components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (STORM-1729) Get rid of reflections while recording stats
[
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated STORM-1729:
Description:
I don't set affects version to 2.0.0 since it only occurs on Clojure.
{code}
(set! *warn-on-reflection* true)
(load-file "src/clj/org/apache/storm/stats.clj")
{code}
{quote}
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
- call to method record can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
- call to method incBy can't be resolved (target class is unknown).
{quote}
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
We expect there's no reflection since we give a type hint while calling, but it
doesn't work.
(I don't know why it doesn't work. Does generic type matter?)
Anyway, defining them into let makes them avoid reflection.
Even though we sample while recording stats, it does hurt performance with fast
& high-traffic components.
was:
I don't add this to 2.0.0 since it only occurs on Clojure.
{code}
(set! *warn-on-reflection* true)
(load-file "src/clj/org/apache/storm/stats.clj")
{code}
{quote}
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
- call to method incBy can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
- call to method record can't be resolved (target class is unknown).
Reflection warning,
/Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
- call to method incBy can't be resolved (target class is unknown).
{quote}
https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
We expect there's no reflection since we give a type hint while calling, but it
doesn't work.
(I don't know why it doesn't work. Does generic type matter?)
Anyway, defining them into let makes them avoid reflection.
Even though we sample while recording stats, it does hurt performance with fast
& high-traffic components.
> Get rid of reflections while recording stats
>
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 1.0.1
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
> - call to method record can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
> - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but
> it doesn't work.
> (I
[jira] [Updated] (STORM-1729) Get rid of reflections while recording stats
[
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated STORM-1729:
Affects Version/s: (was: 1.0.1)
1.0.0
> Get rid of reflections while recording stats
>
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
> - call to method record can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
> - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but
> it doesn't work.
> (I don't know why it doesn't work. Does generic type matter?)
> Anyway, defining them into let makes them avoid reflection.
> Even though we sample while recording stats, it does hurt performance with
> fast & high-traffic components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] storm pull request: STORM-1729 Get rid of reflections while record...
GitHub user HeartSaVioR opened a pull request: https://github.com/apache/storm/pull/1361 STORM-1729 Get rid of reflections while recording stats * define MultiCount/LatencyStatAndMetric in let statement and set type hint to there Before applying I got around 860,000 emitted/sec from BasicTopology, and after applying I got around 940,000~960,000 emitted/sec. And it removes reflection warnings from lines which record stats. Btw, though after applying this, `invokeMatchingMethod` is still on top 10. I'll track where it occurs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HeartSaVioR/storm STORM-1729-1.x-branch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/1361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1361 commit ae78815e6ad27e2301d80c258f4ad535fa1193ff Author: Jungtaek Lim Date: 2016-04-26T03:53:38Z STORM-1729 Get rid of reflections while recording stats * define MultiCount/LatencyStatAndMetric in let statement and set type hint to there --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (STORM-1729) Get rid of reflections while recording stats
[
https://issues.apache.org/jira/browse/STORM-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15257526#comment-15257526
]
ASF GitHub Bot commented on STORM-1729:
---
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1361
STORM-1729 Get rid of reflections while recording stats
* define MultiCount/LatencyStatAndMetric in let statement and set type hint
to there
Before applying I got around 860,000 emitted/sec from BasicTopology, and
after applying I got around 940,000~960,000 emitted/sec.
And it removes reflection warnings from lines which record stats.
Btw, though after applying this, `invokeMatchingMethod` is still on top 10.
I'll track where it occurs.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-1729-1.x-branch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1361.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1361
commit ae78815e6ad27e2301d80c258f4ad535fa1193ff
Author: Jungtaek Lim
Date: 2016-04-26T03:53:38Z
STORM-1729 Get rid of reflections while recording stats
* define MultiCount/LatencyStatAndMetric in let statement and set type hint
to there
> Get rid of reflections while recording stats
>
>
> Key: STORM-1729
> URL: https://issues.apache.org/jira/browse/STORM-1729
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-core
>Affects Versions: 1.0.0
>Reporter: Jungtaek Lim
>Assignee: Jungtaek Lim
>
> I don't set affects version to 2.0.0 since it only occurs on Clojure.
> {code}
> (set! *warn-on-reflection* true)
> (load-file "src/clj/org/apache/storm/stats.clj")
> {code}
> {quote}
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:119:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:123:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:149:3
> - call to method incBy can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:150:3
> - call to method record can't be resolved (target class is unknown).
> Reflection warning,
> /Users/jlim/WorkArea/JavaProjects/storm/storm-core/src/clj/org/apache/storm/stats.clj:154:3
> - call to method incBy can't be resolved (target class is unknown).
> {quote}
> https://github.com/apache/storm/blob/1.x-branch/storm-core/src/clj/org/apache/storm/stats.clj#L119
> We expect there's no reflection since we give a type hint while calling, but
> it doesn't work.
> (I don't know why it doesn't work. Does generic type matter?)
> Anyway, defining them into let makes them avoid reflection.
> Even though we sample while recording stats, it does hurt performance with
> fast & high-traffic components.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
