[
https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684977#comment-13684977
]
Himanshu Vashishtha commented on HBASE-8741:
--------------------------------------------
bq. Who would read the new info that is in tail of the WAL logs?
It is read in the SSH, while processing the dead region server.
bq. Do we need to read the tail of the second to last file if we are reading
the lsat unfinished file anyways since it will have an edit in excess of the
second-to-last files?
bq. Will we read this last file twice? Once when figuring last sequenceid and
then again when replaying the edits?
bq. How we transfer the sequence number to the regionserver that is opening the
region?
Yeah, also reading the last WAL file may delay the recovery process (recover
lease may take longer in case DN is also down). I think read the last
"completed" WAL trailer to get the sequenceId at time of rolling, and add a
delta big enough to avoid reading the last (and most probably incomplete) WAL
file. All the Regions that are to be deployed from this RegionServer will have
this long value in their RegionInfo object. While opening, it will set/compare
this value to the maximum SequenceId obtained from the store files. I am
working on a patch for this.
bq. In multiwall case we may have to read more-than-one file – all files that
are still open.
While implementing multi wal, we should make sure we are getting the
sequenceIds from one counter. That way, we would not have a sequenceIds
collision in the multi wals. We could try using the above approach (read the
trailer of the last completed wal file, and add a delta).
bq. enabling writes in the current Distributed Log Splitting.
Yes, I think we could enable that once we resolve sequenceId stuff. And, that
would be great.
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>
> Key: HBASE-8741
> URL: https://issues.apache.org/jira/browse/HBASE-8741
> Project: HBase
> Issue Type: Bug
> Components: MTTR
> Affects Versions: 0.95.1
> Reporter: Himanshu Vashishtha
> Assignee: Himanshu Vashishtha
>
> Currently, when opening a region, we find the maximum sequence ID from all
> its HFiles and then set the LogSequenceId of the log (in case the later is at
> a small value). This works good in recovered.edits case as we are not writing
> to the region until we have replayed all of its previous edits.
> With distributed log replay, if we want to enable writes while a region is
> under recovery, we need to make sure that the logSequenceId > maximum
> logSequenceId of the old regionserver. Otherwise, we might have a situation
> where new edits have same (or smaller) sequenceIds.
> We can store region level information in the WALTrailer, than this scenario
> could be avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file
> which has a trailer and,
> b) completely reading the last wal file (this file would not have the
> trailer, so it needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all
> completed WAL files, and reading the remaining incomplete files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
