I am not sure it is a good idea to expose the stats via a RPC endpoint. Because most of the stats collection systems (e.g. prometheus) are collecting stats via an http endpoints. I don't think we should duplicate the work that a stats library is good at.
Other thoughts inline. On Sat, Nov 30, 2019 at 4:31 AM Enrico Olivelli <eolive...@gmail.com> wrote: > Hi all, > In my company we are developing a visual tool for Bookkeeper Management and > we would like to have more information about the status of a Bookie just by > using RPC calls. > We already have getBookieInfo that is only reporting about disk space > usage. > > Examples of useful info: > - link to metrics endpoint > - last minor/major gc execution timestamp > - last jvm start timestamp (uptime) > - configuration file > > We already have the http endpoint but it comes with several disadvantages: > - it is a secondary channel (so you have to use two channels to talk to the > bookie) > Why not standardize the management endpoint in http endpoint and build the dashboard on it? It is a common approach for doing so. An RPC channel is more used for data channel. > - there is no way of getting the http address, as it is not advertised on > metadata discovery service > We should fix this, no? > - it does not support security (auth/tls..) yet > We should add TLS support and authentication to the http endpoint. I guess rather than choosing to implement duplicated work in a PRC endpoint, why can't we spend some time enhancing http endpoint? > > Given that I am mostly using Kerberos, > going to the http way will need separate security configurations: for > bookie RPC and for HTTP. > Jia implemented Kerberos for Pulsar http endpoint. Also HDFS supports kerberos at its http endpoint as well. So I don't think that is a problem as well. > > So that's why I would prefer to use the standard binary RPC protocol: only > one client, only one auth, security is already ok, no additional discovery > info. > > If you guys agree on the approach we can send a patch that includes the > support for publishing more information on that RPC > > Cheers > Enrico >
