As you point out these configuration settings were introduced when we migrated from SequenceFile based WALs to the protobuf format. We needed to give users a way to manually migrate, although, arguably, an auto migration would have been better.
In theory these settings allow users to implement their own WAL readers and writers. However I do not believe users will do this. The WAL is critical for performance and correctness. If anyone is contemplating such wizard level changes they can patch the code themselves. It’s fine to document these settings as deprecated for sure, and I think ok also to claim them unsupported and ignored. > > On Feb 10, 2023, at 3:41 AM, 张铎 <[email protected]> wrote: > > While discussing how to deal with the problem in HBASE-27621, we proposed > to introduce two types of WAL readers, one for WAL splitting, and the other > for WAL replication, as replication needs to tail the WAL file which is > currently being written, so the logic is much more complicated. We do not > want to affect WAL splitting logic and performance while tweaking the > replication related things, as all HBase users need WAL splitting but not > everyone needs replication. > > But when reviewing the related code, I found that we have two > configurations for specifying the WAL reader class and WAL write class, > which indicates that we could only have one implementation for the WAL > reader. They are 'hbase.regionserver.hlog.reader.impl' and > 'hbase.regionserver.hlog.writer.impl'. > > We mentioned these two configurations several times in our ref guide. > > HBase 2.0+ can no longer read Sequence File based WAL file. > > HBase can no longer read the deprecated WAL files written in the Apache >> Hadoop Sequence File format. The hbase.regionserver.hlog.reader.impl and >> hbase.regionserver.hlog.writer.impl configuration entries should be set to >> use the Protobuf based WAL reader / writer classes. This implementation has >> been the default since HBase 0.96, so legacy WAL files should not be a >> concern for most downstream users. > > > Configure WAL encryption. > > Configure WAL encryption in every RegionServer’s hbase-site.xml, by setting >> the following properties. You can include these in the HMaster’s >> hbase-site.xml as well, but the HMaster does not have a WAL and will not >> use them. >> <property> >> <name>hbase.regionserver.hlog.reader.impl</name> >> >> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader</value> >> </property> >> <property> >> <name>hbase.regionserver.hlog.writer.impl</name> >> >> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter</value> >> </property> >> <property> >> <name>hbase.regionserver.wal.encryption</name> >> <value>true</value> >> </property> > > > So in fact, do not consider encryption, the configurations are useless as > we do not support reading sequence file format WAL any more, the only valid > options are protobuf based reader and write. And for security, I think the > configuration is redundant as if encryption is enabled, we should use > SecureProtobufLogWriter for writing, no matter what the configuration value > is. And for readers, I do not think we should use a configuration to > specify the implementation, we should detect whether the file is encrypted > and choose a secure or normal reader to read the file. > > So here, I propose we just deprecated these two configurations because they > are useless now. > > Thanks.
