[
https://issues.apache.org/jira/browse/HBASE-27632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-27632:
------------------------------
Release Note:
Introduced two types of WAL reader, WALStreamReader and WALTailingReader. The
former one is for most use cases where we only need to read closed WAL files,
such as splitting and replaying, and the second one is for replication, where
we need to tail a WAL file which is being written currently.
The config 'hbase.regionserver.hlog.reader.impl' is removed, as now we have two
types of WAL reader. And starting from at least 0.98, we do not support
sequence file format WAL file any more, the only accepted format is protobuf.
And for encrypted WAL files, now you do not need to specify
SecureProtobufLogReader either, as the new implementation will choose to use
SecureWALCellCodec based on the properties in WALHeader.
There is still a config called 'hbase.regionserver.wal.stream.reader.impl', for
specifying the implementation for WALStreamReader. This is only designed to be
used in HBase itself, for injecting some error when writing UTs, downstream
users should not use it as it is unstable.
> Refactor WAL.Reader implementation so we can better support WAL splitting and
> replication
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-27632
> URL: https://issues.apache.org/jira/browse/HBASE-27632
> Project: HBase
> Issue Type: Improvement
> Components: Replication, wal
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Major
>
> WAL splitting and replication has completely different requirements on WAL
> reader, the latter one is much comlicated as it wants to tail a WAL file
> which is currently being written.
> I think split them into two classes is better here.
> But we exposes two configurations out
> 'hbase.regionserver.hlog.reader.impl'
> 'hbase.regionserver.hlog.writer.impl'
> which indicates that we only have one implementation for wal reader.
> These two configurations are inrtoduced in 0.96, for reading the pre 0.94
> sequence file format WAL file. And starting from 2.0.0, we have already
> dropped the support of reading sequence file format WAL file, so I think the
> configurations are useless now.
> Now we stil require users to specify SecureProtobufLogReader and
> SecureProtobufLogWriter when enable WAL encryption, but actually they are
> both the only choice, so just use 'hbase.regionserver.wal.encryption' to
> determine whether we need to use SecureProtobufLogWriter to write WAL. And
> for reading, I do not think we should rely on the configuration, we should
> detect whether the WAL file itself is encrypted(I believe we have already
> done this in SecureProtobufLogReader.readHeader.
> So in general, I prefer we just mark these two configurations deprecated and
> do not use them in our code base, so we can have different Reader
> implementations for WAL splitting and replication.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
