Sijie, answers inline Enrico
Il giorno mar 3 dic 2019 alle ore 08:15 Sijie Guo <guosi...@gmail.com> ha scritto: > I am not sure it is a good idea to expose the stats via a RPC endpoint. > > Because most of the stats collection systems (e.g. prometheus) are > collecting stats via an http endpoints. > I don't think we should duplicate the work that a stats library is good at. > > Other thoughts inline. > > On Sat, Nov 30, 2019 at 4:31 AM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > Hi all, > > In my company we are developing a visual tool for Bookkeeper Management > and > > we would like to have more information about the status of a Bookie just > by > > using RPC calls. > > We already have getBookieInfo that is only reporting about disk space > > usage. > > > > Examples of useful info: > > - link to metrics endpoint > > - last minor/major gc execution timestamp > > - last jvm start timestamp (uptime) > > - configuration file > > > > We already have the http endpoint but it comes with several > disadvantages: > > - it is a secondary channel (so you have to use two channels to talk to > the > > bookie) > > > > Why not standardize the management endpoint in http endpoint and build the > dashboard on it? > Actually our project is a Web Application (WAR archive) so we can deploy it to any Servlet Container (Jetty, Tomcat), so in theory it could be deployed inside the bookie itself or at least distributed with BK binary distribution Once we have a stable release one good plan could be to contribute it to ASF BookKeeper Project (as StreamNative did with Pulsar Manager) https://github.com/diennea/bookkeeper-visual-manager > It is a common approach for doing so. An RPC channel is more used for data > channel. > > > > - there is no way of getting the http address, as it is not advertised on > > metadata discovery service > > > > We should fix this, no? > Yes we could, but actually it is more invasive change. I think such kind of enhancements should be done together with the change about using some bookieid that is is not host:port. > > > > - it does not support security (auth/tls..) yet > > > > We should add TLS support and authentication to the http endpoint. > Yes we could. Some problems: - (community) we have multiple implementations of the HTTP endpoint, we should decide whether to enhance only one, all of the them... - (private) In all of the clusters of my customers we are not using the http endpoint provided by the bookie, so actually having a new open endpoint will come with some disadvantages (new open port, security, network architecture change....) - The Bookie http server is not "read only", it can force the bookie to do things, we should also change this > > I guess rather than choosing to implement duplicated work in a PRC > endpoint, why can't we spend some time enhancing http endpoint? > > > > > > Given that I am mostly using Kerberos, > > going to the http way will need separate security configurations: for > > bookie RPC and for HTTP. > > > > Jia implemented Kerberos for Pulsar http endpoint. Also HDFS supports > kerberos at its http endpoint as well. > So I don't think that is a problem as well. > That would be interesting. > > > > > > So that's why I would prefer to use the standard binary RPC protocol: > only > > one client, only one auth, security is already ok, no additional > discovery > > info. > > > > If you guys agree on the approach we can send a patch that includes the > > support for publishing more information on that RPC > > > > Cheers > > Enrico > > >
