Re: [DISCUSS] HBASE-20904 Prometheus /metrics http endpoint for monitoring integration

[Sean Busbey]

Oh! that's a great reference. Are there docs for those over in Hadoop or
Ozone?

On Mon, Jun 8, 2020 at 10:46 AM Wei-Chiu Chuang  wrote:

> Might be useful to look at the Hadoop Prometheus end point and Ozone
> Prometheus end point as well.
>
>1. HADOOP-16398 
>
>
>1. HDDS-919 
>
>
> A known issue with the Hadoop Prometheus implementation is that it lacks
> SPENGO support, so there is a problem integrating it with the Kerberized
> environment. The solution is to whitelist the Prometheus end point so it
> doesn't require SPNEGO.
>
>
> On Sat, Jun 6, 2020 at 10:00 PM ckai2480  wrote:
>
> > Hi,
> >
> > I am working on HBASE-20904 (Prometheus /metrics http endpoint for
> > monitoring integration) and have created a patch (
> > https://github.com/apache/hbase/pull/1814). @busbey and @saintstack
> > suggested some good changes which I have incorporated in my local
> > branch and wanted to know other's suggestions on it before I commit to
> > the remote.
> >
> > Prometheus
> > 1. Prometheus is a monitoring software which uses HTTP to pull the
> > metrics from the monitored processes.
> > 2. The collected data can be used to anamoly detection, altering etc.
> >
> > Problem HBASE-20904 solves.
> > 1. Implementing a servlet to expose these metrics.
> >
> > Currently I have implemented this as follows
> > 1. Expose two end points
> >   /prometheus:
> >   exposes the metrics captured in native hbase metrics
> >   /prometheus-hadoop2:
> >   exposes the metrics captured using hadoop2 metrics.
> >
> > latter is planned to be removed once all the metric sources start
> > using the native metrics API.
> >
> > **do the endpoints names look ok?**
> >
> > 2. Make the /jmx, /prometheus-hadoop2, /prometheus, /metrics servlets
> > optional by providing a multivalued config key with the first two
> > servlets as default (values will be classnames or aliases **which one
> > do you think is good idea**).
> >
> > Out of curiosity.. Why the hadoop2 metrics still exists in HBase, I see
> > HBASE-14282 is open and task issues linked are closed. Are there any
> > unlinked issues here?
> >
> > Thanks,
> > Madhusoodan
> >
> >
>

Re: [DISCUSS] HBASE-20904 Prometheus /metrics http endpoint for monitoring integration

[Wei-Chiu Chuang]

Might be useful to look at the Hadoop Prometheus end point and Ozone
Prometheus end point as well.

   1. HADOOP-16398 


   1. HDDS-919 


A known issue with the Hadoop Prometheus implementation is that it lacks
SPENGO support, so there is a problem integrating it with the Kerberized
environment. The solution is to whitelist the Prometheus end point so it
doesn't require SPNEGO.


On Sat, Jun 6, 2020 at 10:00 PM ckai2480  wrote:

> Hi,
>
> I am working on HBASE-20904 (Prometheus /metrics http endpoint for
> monitoring integration) and have created a patch (
> https://github.com/apache/hbase/pull/1814). @busbey and @saintstack
> suggested some good changes which I have incorporated in my local
> branch and wanted to know other's suggestions on it before I commit to
> the remote.
>
> Prometheus
> 1. Prometheus is a monitoring software which uses HTTP to pull the
> metrics from the monitored processes.
> 2. The collected data can be used to anamoly detection, altering etc.
>
> Problem HBASE-20904 solves.
> 1. Implementing a servlet to expose these metrics.
>
> Currently I have implemented this as follows
> 1. Expose two end points
>   /prometheus:
>   exposes the metrics captured in native hbase metrics
>   /prometheus-hadoop2:
>   exposes the metrics captured using hadoop2 metrics.
>
> latter is planned to be removed once all the metric sources start
> using the native metrics API.
>
> **do the endpoints names look ok?**
>
> 2. Make the /jmx, /prometheus-hadoop2, /prometheus, /metrics servlets
> optional by providing a multivalued config key with the first two
> servlets as default (values will be classnames or aliases **which one
> do you think is good idea**).
>
> Out of curiosity.. Why the hadoop2 metrics still exists in HBase, I see
> HBASE-14282 is open and task issues linked are closed. Are there any
> unlinked issues here?
>
> Thanks,
> Madhusoodan
>
>

Re: [DISCUSS] HBASE-20904 Prometheus /metrics http endpoint for monitoring integration

[ckai2480]

Also please let me know if you have any other suggestions.

Thanks,
Madhusoodan

On Sat, 2020-06-06 at 16:21 +0530, ckai2480 wrote:
> Hi,
> 
> I am working on HBASE-20904 (Prometheus /metrics http endpoint for
> monitoring integration) and have created a patch (
> https://github.com/apache/hbase/pull/1814). @busbey and @saintstack
> suggested some good changes which I have incorporated in my local
> branch and wanted to know other's suggestions on it before I commit
> to
> the remote.
> 
> Prometheus
> 1. Prometheus is a monitoring software which uses HTTP to pull the
> metrics from the monitored processes.
> 2. The collected data can be used to anamoly detection, altering etc.
> 
> Problem HBASE-20904 solves.
> 1. Implementing a servlet to expose these metrics.
> 
> Currently I have implemented this as follows
> 1. Expose two end points
>   /prometheus:
>   exposes the metrics captured in native hbase metrics
>   /prometheus-hadoop2:
>   exposes the metrics captured using hadoop2 metrics.
> 
> latter is planned to be removed once all the metric sources start
> using the native metrics API.
> 
> **do the endpoints names look ok?**
> 
> 2. Make the /jmx, /prometheus-hadoop2, /prometheus, /metrics servlets
> optional by providing a multivalued config key with the first two
> servlets as default (values will be classnames or aliases **which one
> do you think is good idea**).
> 
> Out of curiosity.. Why the hadoop2 metrics still exists in HBase, I
> see
> HBASE-14282 is open and task issues linked are closed. Are there any
> unlinked issues here?
> 
> Thanks,
> Madhusoodan
> 

[DISCUSS] HBASE-20904 Prometheus /metrics http endpoint for monitoring integration

[ckai2480]

Hi,

I am working on HBASE-20904 (Prometheus /metrics http endpoint for
monitoring integration) and have created a patch (
https://github.com/apache/hbase/pull/1814). @busbey and @saintstack
suggested some good changes which I have incorporated in my local
branch and wanted to know other's suggestions on it before I commit to
the remote.

Prometheus
1. Prometheus is a monitoring software which uses HTTP to pull the
metrics from the monitored processes.
2. The collected data can be used to anamoly detection, altering etc.

Problem HBASE-20904 solves.
1. Implementing a servlet to expose these metrics.

Currently I have implemented this as follows
1. Expose two end points
  /prometheus:
  exposes the metrics captured in native hbase metrics
  /prometheus-hadoop2:
  exposes the metrics captured using hadoop2 metrics.

latter is planned to be removed once all the metric sources start
using the native metrics API.

**do the endpoints names look ok?**

2. Make the /jmx, /prometheus-hadoop2, /prometheus, /metrics servlets
optional by providing a multivalued config key with the first two
servlets as default (values will be classnames or aliases **which one
do you think is good idea**).

Out of curiosity.. Why the hadoop2 metrics still exists in HBase, I see
HBASE-14282 is open and task issues linked are closed. Are there any
unlinked issues here?

Thanks,
Madhusoodan