Oh, I've opend HBASE-27632 for the refactoring work. Thanks.
张铎(Duo Zhang) <[email protected]> 于2023年2月10日周五 16:40写道： > While discussing how to deal with the problem in HBASE-27621, we proposed > to introduce two types of WAL readers, one for WAL splitting, and the other > for WAL replication, as replication needs to tail the WAL file which is > currently being written, so the logic is much more complicated. We do not > want to affect WAL splitting logic and performance while tweaking the > replication related things, as all HBase users need WAL splitting but not > everyone needs replication. > > But when reviewing the related code, I found that we have two > configurations for specifying the WAL reader class and WAL write class, > which indicates that we could only have one implementation for the WAL > reader. They are 'hbase.regionserver.hlog.reader.impl' and > 'hbase.regionserver.hlog.writer.impl'. > > We mentioned these two configurations several times in our ref guide. > > HBase 2.0+ can no longer read Sequence File based WAL file. > > HBase can no longer read the deprecated WAL files written in the Apache >> Hadoop Sequence File format. The hbase.regionserver.hlog.reader.impl and >> hbase.regionserver.hlog.writer.impl configuration entries should be set to >> use the Protobuf based WAL reader / writer classes. This implementation has >> been the default since HBase 0.96, so legacy WAL files should not be a >> concern for most downstream users. > > > Configure WAL encryption. > > Configure WAL encryption in every RegionServer’s hbase-site.xml, by >> setting the following properties. You can include these in the HMaster’s >> hbase-site.xml as well, but the HMaster does not have a WAL and will not >> use them. >> <property> >> <name>hbase.regionserver.hlog.reader.impl</name> >> >> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader</value> >> </property> >> <property> >> <name>hbase.regionserver.hlog.writer.impl</name> >> >> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter</value> >> </property> >> <property> >> <name>hbase.regionserver.wal.encryption</name> >> <value>true</value> >> </property> > > > So in fact, do not consider encryption, the configurations are useless as > we do not support reading sequence file format WAL any more, the only valid > options are protobuf based reader and write. And for security, I think the > configuration is redundant as if encryption is enabled, we should use > SecureProtobufLogWriter for writing, no matter what the configuration value > is. And for readers, I do not think we should use a configuration to > specify the implementation, we should detect whether the file is encrypted > and choose a secure or normal reader to read the file. > > So here, I propose we just deprecated these two configurations because > they are useless now. > > Thanks. > > >
