First of all there really are two different sets of metrics. One set is the
topology metrics and the other set is the daemon metrics (metrics for things
like the ui and nimbus). The JmxPreparableReporter plugin only exposes daemon
metrics not the topology metrics through JMX. Exposing topology metrics
through JMX is a non trivial task. The current metrics feature was not
designed for this. We are in the process of trying to redesign the metrics
system to allow for features like this, but it is still a ways off. - Bobby
On Saturday, October 8, 2016 11:39 AM, S G <[email protected]>
wrote:
Thanks Bobby,
We will need some kind of IMetricsConsumer to talk to telegraf.Many other
softwares like Solr, Elastic-Search, Cassandra etc. provide metrics through a
URL making it very easy to consume by tools like telegraf.How about a
IMetricsConsumer that will run on storm-ui and provide the metrics through a
URL such as <storm-ui-host>/metrics ?
Also, I see the following option in defaults.yaml:#default storm daemon metrics
reporter pluginsstorm.daemon.metrics.reporter.plugins: -
"org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"
Is this a good option to use for converting metrics into JMX ?
ThanksSG
On Fri, Oct 7, 2016 at 8:11 AM, Bobby Evans <[email protected]> wrote:
HttpForwardingMetricsServer is a real hack intended really for tests. I know I
wrote it :). Please don't use it in production. You can write your own
IMetricesConsumer to do whatever you want to with the metrics.
https://github.com/apache/ storm/blob/master/storm-core/
src/jvm/org/apache/storm/ metric/api/IMetricsConsumer. java
That is the correct way to get the data out. If you want to write a bridge to
JMX for this that might work, but going directly to telegraph would probably be
better. - Bobby
On Thursday, October 6, 2016 1:43 PM, S G <[email protected]> wrote:
Hi,
We want to use Telegraf (
https://github.com/influxdata/ telegraf/tree/master/plugins) for getting
storm's metrics.
But we do not want to add a HttpForwardingMetricsServer just to get the
metrics and send them to telegraf.
Other option is to use Jolokia (https://jolokia.org/) that can read JMX and
write into telegraf.
Does storm report all its metrics (including those of custom spouts/bolts)
into JMX?
Or spawning a HttpForwardingMetricsServer is the only option?
Thanks
SG
