Re: Are storm metrics reported through JMX too?

Response on this important issue is pretty good. I am happily surprised :) I want to mention our strategy for extracting metrics from other products. We use jolokia_proxy (https://jolokia.org/features/proxy.html) to get JMX beans from several softwares and feed them to telegraf. That way, we

Re: Are storm metrics reported through JMX too?

blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Yeap that's a requirement from our perspective (working through this list). Sure I think as usual we can start with master with

Re: Are storm metrics reported through JMX too?

No worries Taylor. I'm in favor of quality over quantity, and both you and me were having valid but different aspects for that thing. And like I said, I'm all for redesign metrics so no interest on current metrics if we plan to move on. Btw, I'm not sure how metrics will be changed, but requiring

Re: Are storm metrics reported through JMX too?

I hope I didn't come across as overly critical. You did the best with what you had to work with. Which isn't pretty. We could potentially do a parallel metrics API in 1.1, 1.2, or master and still stay close to semantic versioning...? -Taylor > On Oct 11, 2016, at 9:28 PM, Jungtaek Lim

Re: Are storm metrics reported through JMX too?

That sounds like an awesome idea, Alessandro! I would encourage you to teach the students the Apache Way, not just how it applies to the Apache Storm project, but in other projects as well. And remember the ASF tenet "if it didn't happen on a mailing list, it didn't happen." ;) If there's

Re: Are storm metrics reported through JMX too?

Yeah I admit that configuration flag was bad for me also, but I have no alternatives. Only way to avoid struggling with design limitation is revamp / redesign. Thanks S G for exposing willingness of volunteer and great news Alessandro for that project. Alessandro, could you forward the upcoming

Re: Are storm metrics reported through JMX too?

I was thinking on a smaller scale in terms of effort, but the more I think about it, the more supportive I would be of a full revamp (new API) for metrics based on Coda Hale's metrics library. It's proven and stable. I've used it many times. I think either approach would be roughly the same

Re: Are storm metrics reported through JMX too?

blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px #715FFA solid !important; padding-left:1ex !important; background-color:white !important; } Sounds great S G. Thanks for pointers Jungtaek.  We were approached by a group of students from the University of Illinois who

Re: Are storm metrics reported through JMX too?

> On Oct 11, 2016, at 8:34 PM, S G wrote: > > I would like to volunteer for this (Have contributed to Solr, Avro and Hive > previously). > But I would need a little guidance initially to get started because I > haven't dug too deep in storm's code-base. > > -SG > >

[GitHub] storm pull request #1443: Log.warn if found a message in kafka topic larger ...

Github user erikdw commented on a diff in the pull request: https://github.com/apache/storm/pull/1443#discussion_r82918924 --- Diff: external/storm-kafka/src/jvm/org/apache/storm/kafka/KafkaUtils.java --- @@ -218,6 +219,11 @@ public static ByteBufferMessageSet

Re: Are storm metrics reported through JMX too?

I would like to volunteer for this (Have contributed to Solr, Avro and Hive previously). But I would need a little guidance initially to get started because I haven't dug too deep in storm's code-base. -SG On Tue, Oct 11, 2016 at 3:52 PM, Jungtaek Lim wrote: > Alessandro,

[GitHub] storm pull request #1720: STORM-1546: Adding Read and Write Aggregations for...

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1720 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #1720: STORM-1546: Adding Read and Write Aggregations for Pacema...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1720 Thanks for the change. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

Re: Are storm metrics reported through JMX too?

Alessandro, that was before Verisign introduced storm-graphite. In result he followed Storm's metric system. I initiated the discussion around metrics several times (mostly first half of this year), and from many times the results were that all the metrics interfaces are not good and we need to

[GitHub] storm issue #1697: STORM-2018: Supervisor V2

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1697 @revans2 It would be better to address STORM-2131 here as well. Please pull #1724 here. And could you update the pull request according to the review comments? Supervisor V2 is the one

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1714 Calcite 1.10.0 RC1 vote passed and artifact is available to Maven. Applied HeartSaVioR@b2bcf9b with changing Calcite version from 1.10.0-SNAPSHOT to 1.10.0. Also rebased and squashed the

[GitHub] storm pull request #1681: STORM-1444 Support EXPLAIN statement in StormSQL

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1681 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm pull request #1689: STORM-2099 Introduce new sql external module: stor...

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1689 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #1720: STORM-1546: Adding Read and Write Aggregations for Pacema...

Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/1720 Also updated the docs. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] storm issue #1720: STORM-1546: Adding Read and Write Aggregations for Pacema...

Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/1720 @HeartSaVioR I've synced the java files between the branches so there are almost no differences. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] storm pull request #1679: STORM-2087: storm-kafka-client - tuples not always...

Github user jfenc91 commented on a diff in the pull request: https://github.com/apache/storm/pull/1679#discussion_r82872195 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java --- @@ -266,26 +266,31 @@ private void

[GitHub] storm issue #1733: [STORM-2134] - improving the current scheduling strategy ...

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1733 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: Are storm metrics reported through JMX too?

sorry, hopefully the link goes through now: http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite Sending Metrics from Storm to Graphite - Michael G. Noll By Michael G. Noll Sending application-level metrics from Storm topologies to the Graphite monitoring

Re: Are storm metrics reported through JMX too?

I think what Bobby is referring to is that the metrics consumer is another bolt, so stats are flowing through storm. What does changing the model to polling buy us? I could see cases were we'd need more error handling for instance slow/busy workers. If we think that writing a new system is the

Re: Are storm metrics reported through JMX too?

"Dropwizard has solved all of these problems already and I don't see a reason to reinvent the wheel" - I love dropwizard too and many of the other tools have switched to using the same too. "I don't personally see a lot of value in trying to send all of the metrics through storm itself" - How

[GitHub] storm pull request #1685: STORM-2097: Improve logging in trident core and ex...

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1685 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm pull request #1724: STORM-2131: Add blob command to worker-launcher, m...

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1724 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm pull request #1679: STORM-2087: storm-kafka-client - tuples not always...

Github user harshach commented on a diff in the pull request: https://github.com/apache/storm/pull/1679#discussion_r82825994 --- Diff: external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/KafkaSpout.java --- @@ -266,26 +266,31 @@ private void

[GitHub] storm issue #1689: STORM-2099 Introduce new sql external module: storm-sql-r...

Github user harshach commented on the issue: https://github.com/apache/storm/pull/1689 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] storm issue #1681: STORM-1444 Support EXPLAIN statement in StormSQL

Github user harshach commented on the issue: https://github.com/apache/storm/pull/1681 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] storm issue #1733: [STORM-2134] - improving the current scheduling strategy ...

Github user jerrypeng commented on the issue: https://github.com/apache/storm/pull/1733 @revans2 thanks for the reviews. I have addressed your comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] storm issue #1724: STORM-2131: Add blob command to worker-launcher, make sto...

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1724 +1 looks good --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] storm pull request #1724: STORM-2131: Add blob command to worker-launcher, m...

Github user revans2 commented on a diff in the pull request: https://github.com/apache/storm/pull/1724#discussion_r82815297 --- Diff: storm-core/src/native/worker-launcher/impl/worker-launcher.c --- @@ -485,14 +482,14 @@ int setup_stormdist_dir(const char* local_dir) {

Re: Are storm metrics reported through JMX too?

I agree that IMetricsConsumer is not good, but the reality is that all of the metrics system needs to be redone.  The problem is that we ship an object as a metric.  If I get an object I have no idea what it is hand hence no idea how to report it or what to do with it.  What is more the common

[GitHub] storm issue #1702: Update Eventhub-Client jar dependency to 1.0.1

Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1702 I am not really familiar with EventHub all that much so I don't really feel all that confident in reviewing it. I mostly jumped on to answer a few generic questions. --- If your project is set up

[GitHub] storm issue #1729: [STORM-2144] Fix Storm-sql group-by behavior in standalon...

Github user satishd commented on the issue: https://github.com/apache/storm/pull/1729 Pushed to master branch also. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] storm issue #1729: [STORM-2144] Fix Storm-sql group-by behavior in standalon...

Github user satishd commented on the issue: https://github.com/apache/storm/pull/1729 Thanks @arunmahadevan for the fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] storm pull request #1729: [STORM-2144] Fix Storm-sql group-by behavior in st...

Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] storm issue #1729: [STORM-2144] Fix Storm-sql group-by behavior in standalon...

Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1729 Spend some time to look into it but it looks like breaking changes are needed. Interpreter expects static input table before initializing Interpreter, which is valid for most of case, but

[GitHub] storm issue #1702: Update Eventhub-Client jar dependency to 1.0.1

Github user raviperi commented on the issue: https://github.com/apache/storm/pull/1702 @revans2 Please let me know if you have further comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not