[
https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13695978#comment-13695978
]
Himanshu Vashishtha commented on HBASE-8741:
--------------------------------------------
We want to scope sequenceIDs that are appended in a WALEdit to region.
Currently, they are scoped at regionserver level.
I took a first pass on the above problem and would like to gather feedback on
the approach in general.
High Level:
1) Generate sequenceId at region level, and use it for all Log#append ops.
2) Don't pass HRegionServerServices to HLog; this defeats the purpose of using
LogFactory. To pass regionlevel info from HRegionServer to HLog, use the
WALActionListener. HRegionServer currently uses LogRoller, which can pass the
necessary info to FSHLog instance from regionserver services.
3) For each WAL, contain an in-memory map of all regions:<current_seqId> at
time of rolling. This is used when deciding a WAL is eligible for rolling or
not. We could call this "snapshotting of regionSequenceIds" while rolling the
wal.
While cleaning up the WAL, we compare these snapshotted sequenceIds to the
corresponding entries in oldestUnflushedEntries (and oldestFlushingEntries). If
for all regions, the sequenceIds are lesser, then that wal is eligible for
archiving.
The above sequenceID snapshot should have an accurate info; so we create it at
time when we stopped writing to the old (current) wal.
Testing: Jenkins is green; also ran a patched local hbase instance and did
flushing/log-rolling. It works as expected.
Thanks.
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>
> Key: HBASE-8741
> URL: https://issues.apache.org/jira/browse/HBASE-8741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
> Affects Versions: 0.95.1
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
> Attachments: HBASE-8741-v0.patch
>
>
> Currently, when opening a region, we find the maximum sequence ID from all
> its HFiles and then set the LogSequenceId of the log (in case the later is at
> a small value). This works good in recovered.edits case as we are not writing
> to the region until we have replayed all of its previous edits.
> With distributed log replay, if we want to enable writes while a region is
> under recovery, we need to make sure that the logSequenceId > maximum
> logSequenceId of the old regionserver. Otherwise, we might have a situation
> where new edits have same (or smaller) sequenceIds.
> We can store region level information in the WALTrailer, than this scenario
> could be avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file
> which has a trailer and,
> b) completely reading the last wal file (this file would not have the
> trailer, so it needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all
> completed WAL files, and reading the remaining incomplete files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
