Hi folks, I have experience enabling secure mode (Kerberos) for the HDFS layer on all of Hubspot's HBase clusters. The pattern in the Hadoop project is for clients to know what principal, or principal pattern, to expect a server to present. For example, see `dfs.namenode.kerberos.principal.pattern` on https://hadoop.apache.org/docs/r2.9.2/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml. Clients check the server's principal against the expected string or pattern to make sure it matches. Ideally, using Kerberos would provide a cryptographically verified guarantee to clients that they are connecting to the right server and not an imposter who might have hijacked the right server's domain name. I recommend that you add a property that the HBase client checks against the principal name that the server presents during the connection negotiation. If it doesn't match, the connection should fail. As a backwards compatibility measure, the pattern can default to "*", so it's really only used by users who care about this level of security. To be honest, Hubspot does not use this security feature very much, but since it seems to be standard in Hadoop, it would be nice to have it in HBase.
Charles On Tue, Jan 23, 2024 at 9:00 AM 张铎(Duo Zhang) <palomino...@gmail.com> wrote: > Thanks Bryan. > > If no other concerns, let me at least re-implement the PR for > HBASE-25051 based on the approach proposed here. > > Bryan Beaudreault <bbeaudrea...@apache.org> 于2024年1月23日周二 21:40写道： > > > > Thanks for pulling this together Duo. I'll take a closer look at this > after > > I finish up the 2.6.0 release. > > > > To me the only possibly controversial part is: > > > > > For HBASE-28321, it should be part of our rpc negotiation, where the > > server should return its server principal to the client, to let the > client > > acquire the necessary ticket for authentication. For me, I do not think > > this will increase the security risk. > > > > I'm not a security or kerberos expert, so can't really speak to whether > > that's safe without some deeper investigation. If we can get clarity on > > that, then the rest seems mostly good. > > > > If anyone with kerberos security experience has opinions on that quote, > > that'd be a big help to keep this moving forward. > > > > On Sun, Jan 21, 2024 at 9:10 AM 张铎(Duo Zhang) <palomino...@gmail.com> > wrote: > > > > > RpcConnectionRegistry was introduced in 2.5.0, and since it reduces > > > the load of zookeeper, I believe it has already been used by lots of > > > users. > > > We have already known that it can not work well with token(digest) > > > based authentication, and have already filed HBASE-25051 for it. and > > > recently, when fixing HBASE-28316, we found out that it could also be > > > broken if we choose to use different server principals for master and > > > region server, see HBASE-28321 for more details. > > > Since this is a very important feature and should have been widely > > > used among our users, I think we should fix these issues ASAP. > > > I've already worked on HBASE-25051 for a while and provided a workable > > > solution[1], so this time I took a deep thought and put up a design > > > doc[2], to address both HBASE-25051 and HBASE-28321. > > > > > > Please take a look. Any comments here or on the design doc are > welcomed. > > > > > > Thanks. > > > > > > 1. https://github.com/apache/hbase/pull/5631 > <https://github.com/apache/hbase/pull/5631> > > > 2. > > > > https://docs.google.com/document/d/1Cu-qzAdBGyBKM07aQP06RM0oeFSLPGtQFWuV_TDyBNg/edit?usp=sharing > <https://docs.google.com/document/d/1Cu-qzAdBGyBKM07aQP06RM0oeFSLPGtQFWuV_TDyBNg/edit?usp=sharing> > > > >
