I am preparing a BP with the goal of enabling discovery of every service exposed by a Bookie using existing metadata service.
Stay tuned Enrico Il mar 3 dic 2019, 08:55 Enrico Olivelli <eolive...@gmail.com> ha scritto: > Sijie, > answers inline > > Enrico > > Il giorno mar 3 dic 2019 alle ore 08:15 Sijie Guo <guosi...@gmail.com> ha > scritto: > >> I am not sure it is a good idea to expose the stats via a RPC endpoint. >> >> Because most of the stats collection systems (e.g. prometheus) are >> collecting stats via an http endpoints. >> I don't think we should duplicate the work that a stats library is good >> at. >> >> Other thoughts inline. >> >> On Sat, Nov 30, 2019 at 4:31 AM Enrico Olivelli <eolive...@gmail.com> >> wrote: >> >> > Hi all, >> > In my company we are developing a visual tool for Bookkeeper Management >> and >> > we would like to have more information about the status of a Bookie >> just by >> > using RPC calls. >> > We already have getBookieInfo that is only reporting about disk space >> > usage. >> > >> > Examples of useful info: >> > - link to metrics endpoint >> > - last minor/major gc execution timestamp >> > - last jvm start timestamp (uptime) >> > - configuration file >> > >> > We already have the http endpoint but it comes with several >> disadvantages: >> > - it is a secondary channel (so you have to use two channels to talk to >> the >> > bookie) >> > >> >> Why not standardize the management endpoint in http endpoint and build the >> dashboard on it? >> > > Actually our project is a Web Application (WAR archive) so we can deploy > it to any Servlet Container (Jetty, Tomcat), so in theory it could be > deployed inside the bookie itself or at least distributed with BK binary > distribution > Once we have a stable release one good plan could be to contribute it to > ASF BookKeeper Project (as StreamNative did with Pulsar Manager) > > https://github.com/diennea/bookkeeper-visual-manager > > > >> It is a common approach for doing so. An RPC channel is more used for data >> channel. >> >> >> > - there is no way of getting the http address, as it is not advertised >> on >> > metadata discovery service >> > >> >> We should fix this, no? >> > > Yes we could, but actually it is more invasive change. > I think such kind of enhancements should be done together with the change > about using some bookieid that is is not host:port. > > > >> >> >> > - it does not support security (auth/tls..) yet >> > >> >> We should add TLS support and authentication to the http endpoint. >> > > Yes we could. > Some problems: > - (community) we have multiple implementations of the HTTP endpoint, we > should decide whether to enhance only one, all of the them... > - (private) In all of the clusters of my customers we are not using the > http endpoint provided by the bookie, so actually > having a new open endpoint will come with some disadvantages (new open > port, security, network architecture change....) > - The Bookie http server is not "read only", it can force the bookie to do > things, we should also change this > > > > >> >> I guess rather than choosing to implement duplicated work in a PRC >> endpoint, why can't we spend some time enhancing http endpoint? >> >> >> > >> > Given that I am mostly using Kerberos, >> > going to the http way will need separate security configurations: for >> > bookie RPC and for HTTP. >> > >> >> Jia implemented Kerberos for Pulsar http endpoint. Also HDFS supports >> kerberos at its http endpoint as well. >> So I don't think that is a problem as well. >> > > That would be interesting. > > >> >> >> > >> > So that's why I would prefer to use the standard binary RPC protocol: >> only >> > one client, only one auth, security is already ok, no additional >> discovery >> > info. >> > >> > If you guys agree on the approach we can send a patch that includes the >> > support for publishing more information on that RPC >> > >> > Cheers >> > Enrico >> > >> >
