Thanks Andrew for the feedback. If no other concerns, I will start the refactoring work and deprecated these two configs.
Thanks. Andrew Purtell <[email protected]> 于2023年2月10日周五 23:01写道： > As you point out these configuration settings were introduced when we > migrated from SequenceFile based WALs to the protobuf format. We needed to > give users a way to manually migrate, although, arguably, an auto migration > would have been better. > > In theory these settings allow users to implement their own WAL readers > and writers. However I do not believe users will do this. The WAL is > critical for performance and correctness. If anyone is contemplating such > wizard level changes they can patch the code themselves. It’s fine to > document these settings as deprecated for sure, and I think ok also to > claim them unsupported and ignored. > > > > > On Feb 10, 2023, at 3:41 AM, 张铎 <[email protected]> wrote: > > > > While discussing how to deal with the problem in HBASE-27621, we > proposed > > to introduce two types of WAL readers, one for WAL splitting, and the > other > > for WAL replication, as replication needs to tail the WAL file which is > > currently being written, so the logic is much more complicated. We do not > > want to affect WAL splitting logic and performance while tweaking the > > replication related things, as all HBase users need WAL splitting but not > > everyone needs replication. > > > > But when reviewing the related code, I found that we have two > > configurations for specifying the WAL reader class and WAL write class, > > which indicates that we could only have one implementation for the WAL > > reader. They are 'hbase.regionserver.hlog.reader.impl' and > > 'hbase.regionserver.hlog.writer.impl'. > > > > We mentioned these two configurations several times in our ref guide. > > > > HBase 2.0+ can no longer read Sequence File based WAL file. > > > > HBase can no longer read the deprecated WAL files written in the Apache > >> Hadoop Sequence File format. The hbase.regionserver.hlog.reader.impl and > >> hbase.regionserver.hlog.writer.impl configuration entries should be set > to > >> use the Protobuf based WAL reader / writer classes. This implementation > has > >> been the default since HBase 0.96, so legacy WAL files should not be a > >> concern for most downstream users. > > > > > > Configure WAL encryption. > > > > Configure WAL encryption in every RegionServer’s hbase-site.xml, by > setting > >> the following properties. You can include these in the HMaster’s > >> hbase-site.xml as well, but the HMaster does not have a WAL and will not > >> use them. > >> <property> > >> <name>hbase.regionserver.hlog.reader.impl</name> > >> > >> > <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader</value> > >> </property> > >> <property> > >> <name>hbase.regionserver.hlog.writer.impl</name> > >> > >> > <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter</value> > >> </property> > >> <property> > >> <name>hbase.regionserver.wal.encryption</name> > >> <value>true</value> > >> </property> > > > > > > So in fact, do not consider encryption, the configurations are useless as > > we do not support reading sequence file format WAL any more, the only > valid > > options are protobuf based reader and write. And for security, I think > the > > configuration is redundant as if encryption is enabled, we should use > > SecureProtobufLogWriter for writing, no matter what the configuration > value > > is. And for readers, I do not think we should use a configuration to > > specify the implementation, we should detect whether the file is > encrypted > > and choose a secure or normal reader to read the file. > > > > So here, I propose we just deprecated these two configurations because > they > > are useless now. > > > > Thanks. >
