Re: Java API to read metrics via JMX

[Diego Parra]

Hi,
you can try with: https://github.com/Segence/kamon-jmx-collector that uses
Kamon  underlying and allow
report metrics to several backends like : Prometheus, InfluxDB, Datadog.

Cheers,
Diego.

On Sun, Aug 12, 2018 at 10:16 AM Matt Farmer  wrote:

> I will also +1 the JMX Prometheus exporter. It's capable of running as a
> -javaagent so it's really easy to get up and running. And we happen to use
> Prometheus anyways so it's pretty convenient for us.
>
> Link: https://github.com/prometheus/jmx_exporter
>
> On Fri, Aug 10, 2018 at 1:55 AM Ishwor Gurung  >
> wrote:
>
> > I don’t know of Java-based solution but I have successfully used:
> >
> > Kafka JMX Exporter for Prometheus <———> Kafka
> >
> > to collect JMX metrics from Kafka.
> >
> > > On 10 Aug 2018, at 11:10 am, Raghav  wrote:
> > >
> > > Hi
> > >
> > > I found
> > >
> >
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java
> > > code
> > > written by Neha to pull JMX metrics via MBean.
> > >
> > > In here:
> > >
> >
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java#L37
> > > there
> > > is a mention of object name (kafka:type=kafka.SocketServerStats)  and
> > > subsequently SocketServerStatsMBean.class.
> > >
> > > My question is how to get these two for the latest Kafka 1.1.
> > >
> > > I want to write code in Java to get Kafka Stats exposed via JMX and
> then
> > > want to write to DB that our UI can read.
> > >
> > > Thanks.
> > >
> > > R
> > >
> > >> On Wed, Aug 8, 2018 at 6:46 PM, Raghav  wrote:
> > >>
> > >> Hi
> > >>
> > >> Is there any Java API available so that I can enable our Kafka
> cluster's
> > >> JMX port, and consume metrics via JMX api, and dump to a time series
> > >> database.
> > >>
> > >> I checked out jmxtrans, but currently it does not dump to TSDB (time
> > >> series database).
> > >>
> > >> Thanks.
> > >>
> > >> R
> > >>
> > >
> > >
> > >
> > > --
> > > Raghav
> >
> >
>

Re: Java API to read metrics via JMX

[Matt Farmer]

I will also +1 the JMX Prometheus exporter. It's capable of running as a
-javaagent so it's really easy to get up and running. And we happen to use
Prometheus anyways so it's pretty convenient for us.

Link: https://github.com/prometheus/jmx_exporter

On Fri, Aug 10, 2018 at 1:55 AM Ishwor Gurung 
wrote:

> I don’t know of Java-based solution but I have successfully used:
>
> Kafka JMX Exporter for Prometheus <———> Kafka
>
> to collect JMX metrics from Kafka.
>
> > On 10 Aug 2018, at 11:10 am, Raghav  wrote:
> >
> > Hi
> >
> > I found
> >
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java
> > code
> > written by Neha to pull JMX metrics via MBean.
> >
> > In here:
> >
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java#L37
> > there
> > is a mention of object name (kafka:type=kafka.SocketServerStats)  and
> > subsequently SocketServerStatsMBean.class.
> >
> > My question is how to get these two for the latest Kafka 1.1.
> >
> > I want to write code in Java to get Kafka Stats exposed via JMX and then
> > want to write to DB that our UI can read.
> >
> > Thanks.
> >
> > R
> >
> >> On Wed, Aug 8, 2018 at 6:46 PM, Raghav  wrote:
> >>
> >> Hi
> >>
> >> Is there any Java API available so that I can enable our Kafka cluster's
> >> JMX port, and consume metrics via JMX api, and dump to a time series
> >> database.
> >>
> >> I checked out jmxtrans, but currently it does not dump to TSDB (time
> >> series database).
> >>
> >> Thanks.
> >>
> >> R
> >>
> >
> >
> >
> > --
> > Raghav
>
>

Re: Java API to read metrics via JMX

[Ishwor Gurung]

I don’t know of Java-based solution but I have successfully used:

Kafka JMX Exporter for Prometheus <———> Kafka

to collect JMX metrics from Kafka.

> On 10 Aug 2018, at 11:10 am, Raghav  wrote:
> 
> Hi
> 
> I found
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java
> code
> written by Neha to pull JMX metrics via MBean.
> 
> In here:
> https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java#L37
> there
> is a mention of object name (kafka:type=kafka.SocketServerStats)  and
> subsequently SocketServerStatsMBean.class.
> 
> My question is how to get these two for the latest Kafka 1.1.
> 
> I want to write code in Java to get Kafka Stats exposed via JMX and then
> want to write to DB that our UI can read.
> 
> Thanks.
> 
> R
> 
>> On Wed, Aug 8, 2018 at 6:46 PM, Raghav  wrote:
>> 
>> Hi
>> 
>> Is there any Java API available so that I can enable our Kafka cluster's
>> JMX port, and consume metrics via JMX api, and dump to a time series
>> database.
>> 
>> I checked out jmxtrans, but currently it does not dump to TSDB (time
>> series database).
>> 
>> Thanks.
>> 
>> R
>> 
> 
> 
> 
> -- 
> Raghav

Re: Java API to read metrics via JMX

[Raghav]

Hi

I found
https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java
code
written by Neha to pull JMX metrics via MBean.

In here:
https://github.com/kafka-dev/kafka/blob/master/perf/src/main/java/kafka/perf/jmx/BrokerJmxClient.java#L37
there
is a mention of object name (kafka:type=kafka.SocketServerStats)  and
subsequently SocketServerStatsMBean.class.

My question is how to get these two for the latest Kafka 1.1.

I want to write code in Java to get Kafka Stats exposed via JMX and then
want to write to DB that our UI can read.

Thanks.

R

On Wed, Aug 8, 2018 at 6:46 PM, Raghav  wrote:

> Hi
>
> Is there any Java API available so that I can enable our Kafka cluster's
> JMX port, and consume metrics via JMX api, and dump to a time series
> database.
>
> I checked out jmxtrans, but currently it does not dump to TSDB (time
> series database).
>
> Thanks.
>
> R
>



-- 
Raghav

RE: [External] Re: Java API to read metrics via JMX

[Tauzell, Dave]

We use Jolokia (which has a java agent you can load with kafka to expose 
metrics via HTTP) and Influx/Telegraf which has support for Jolokia.   There is 
a fair bit of configuration but it can be done without any coding.

-Dave

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Wednesday, August 8, 2018 9:18 PM
To: users@kafka.apache.org
Subject: [External] Re: Java API to read metrics via JMX

Boris:
BrokerWithJMX is referenced but I didn't find the class source after a brief 
search.

FYI

On Wed, Aug 8, 2018 at 7:10 PM Boris Lublinsky < boris.lublin...@lightbend.com> 
wrote:

> Its actually quite simple, unfortunately you have to read, and then
> write to TSDB.
> Enclosed is an example doing this and dumping to InfluxDB
>
>
> Boris Lublinsky
> FDP Architect
> boris.lublin...@lightbend.com
> https://www.lightbend.com/
>
> On Aug 8, 2018, at 8:46 PM, Raghav  wrote:
>
> Hi
>
> Is there any Java API available so that I can enable our Kafka
> cluster's JMX port, and consume metrics via JMX api, and dump to a
> time series database.
>
> I checked out jmxtrans, but currently it does not dump to TSDB (time
> series database).
>
> Thanks.
>
> R
>
>
>
This e-mail and any files transmitted with it are confidential, may contain 
sensitive information, and are intended solely for the use of the individual or 
entity to whom they are addressed. If you have received this e-mail in error, 
please notify the sender by reply e-mail immediately and destroy all copies of 
the e-mail and any attachments.

Re: Java API to read metrics via JMX

[Raghav]

Thanks Boris. Is there a sample implementation in Java ?

thanks.

R

On Wed, Aug 8, 2018 at 7:10 PM, Boris Lublinsky <
boris.lublin...@lightbend.com> wrote:

> Its actually quite simple, unfortunately you have to read, and then write
> to TSDB.
> Enclosed is an example doing this and dumping to InfluxDB
>
>
>
> Boris Lublinsky
> FDP Architect
> boris.lublin...@lightbend.com
> https://www.lightbend.com/
>
> On Aug 8, 2018, at 8:46 PM, Raghav  wrote:
>
> Hi
>
> Is there any Java API available so that I can enable our Kafka cluster's
> JMX port, and consume metrics via JMX api, and dump to a time series
> database.
>
> I checked out jmxtrans, but currently it does not dump to TSDB (time series
> database).
>
> Thanks.
>
> R
>
>
>
>


-- 
Raghav

Re: Java API to read metrics via JMX

[Boris Lublinsky]

Yea, this is just a simple snippet of how to do this.Boker with JMX is how to get broker from zk

BrokerWithJMX.scala
Description: Binary data

Boris LublinskyFDP Architectboris.lublin...@lightbend.comhttps://www.lightbend.com/


On Aug 8, 2018, at 9:18 PM, Ted Yu  wrote:Boris:BrokerWithJMX is referenced but I didn't find the class source after abrief search.FYIOn Wed, Aug 8, 2018 at 7:10 PM Boris Lublinsky  wrote:Its actually quite simple, unfortunately you have to read, and then writeto TSDB.Enclosed is an example doing this and dumping to InfluxDBBoris LublinskyFDP Architectboris.lublin...@lightbend.comhttps://www.lightbend.com/On Aug 8, 2018, at 8:46 PM, Raghav  wrote:HiIs there any Java API available so that I can enable our Kafka cluster'sJMX port, and consume metrics via JMX api, and dump to a time seriesdatabase.I checked out jmxtrans, but currently it does not dump to TSDB (time seriesdatabase).Thanks.R

Re: Java API to read metrics via JMX

[Ted Yu]

Boris:
BrokerWithJMX is referenced but I didn't find the class source after a
brief search.

FYI

On Wed, Aug 8, 2018 at 7:10 PM Boris Lublinsky <
boris.lublin...@lightbend.com> wrote:

> Its actually quite simple, unfortunately you have to read, and then write
> to TSDB.
> Enclosed is an example doing this and dumping to InfluxDB
>
>
> Boris Lublinsky
> FDP Architect
> boris.lublin...@lightbend.com
> https://www.lightbend.com/
>
> On Aug 8, 2018, at 8:46 PM, Raghav  wrote:
>
> Hi
>
> Is there any Java API available so that I can enable our Kafka cluster's
> JMX port, and consume metrics via JMX api, and dump to a time series
> database.
>
> I checked out jmxtrans, but currently it does not dump to TSDB (time series
> database).
>
> Thanks.
>
> R
>
>
>

Re: Java API to read metrics via JMX

[Boris Lublinsky]

Its actually quite simple, unfortunately you have to read, and then write to TSDB.Enclosed is an example doing this and dumping to InfluxDB

JMXCollector.scala
Description: Binary data

Boris LublinskyFDP Architectboris.lublin...@lightbend.comhttps://www.lightbend.com/


On Aug 8, 2018, at 8:46 PM, Raghav  wrote:HiIs there any Java API available so that I can enable our Kafka cluster'sJMX port, and consume metrics via JMX api, and dump to a time seriesdatabase.I checked out jmxtrans, but currently it does not dump to TSDB (time seriesdatabase).Thanks.R

Java API to read metrics via JMX

[Raghav]

Hi

Is there any Java API available so that I can enable our Kafka cluster's
JMX port, and consume metrics via JMX api, and dump to a time series
database.

I checked out jmxtrans, but currently it does not dump to TSDB (time series
database).

Thanks.

R