[
https://issues.apache.org/jira/browse/HBASE-20724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francis Liu updated HBASE-20724:
--------------------------------
Description:
It is important that compacted storefiles of a given compaction execution are
wholly opened or archived to insure data consistency. ie a storefile containing
delete tombstones can be archived while older storefiles containing cells that
were supposed to be deleted are left unarchived thereby undeleting those cells.
When a server fails compaction markers (in the wal edit) are used to determine
which storefiles are compacted and should be excluded during region open
(during failover). But the WALs containing compaction markers can be
prematurely archived even though there are still compacted storefiles for that
particular compaction event that hasn't been archived yet. Thus losing
compaction information that needs to be replayed in the event of an RS crash.
This is because hlog archiving logic only keeps track of flushed storefiles and
not compacted ones.
https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680
was:
It is important that compacted storefiles of a given compaction execution are
wholly opened or archived to insure data consistency. ie a storefile containing
delete tombstones can be archived while older storefiles containing cells that
were supposed to be deleted are left unarchived thereby undeleting those cells.
When a server fails compaction markers (in the wal edit) are used to determine
which storefiles are compacted and should be excluded during region open
(during failover). But the WALs containing compaction markers can be
prematurely archived even though there are still compacted storefiles for that
particular compaction event that hasn't been archived yet. Thus losing
compaction information that needs to be replayed. This is because hlog
archiving logic only keeps track of flushed storefiles and not archived ones.
https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680
> Sometimes some compacted storefiles are still opened after region failover
> --------------------------------------------------------------------------
>
> Key: HBASE-20724
> URL: https://issues.apache.org/jira/browse/HBASE-20724
> Project: HBase
> Issue Type: Bug
> Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0
> Reporter: Francis Liu
> Assignee: Francis Liu
> Priority: Critical
>
> It is important that compacted storefiles of a given compaction execution are
> wholly opened or archived to insure data consistency. ie a storefile
> containing delete tombstones can be archived while older storefiles
> containing cells that were supposed to be deleted are left unarchived thereby
> undeleting those cells.
> When a server fails compaction markers (in the wal edit) are used to
> determine which storefiles are compacted and should be excluded during region
> open (during failover). But the WALs containing compaction markers can be
> prematurely archived even though there are still compacted storefiles for
> that particular compaction event that hasn't been archived yet. Thus losing
> compaction information that needs to be replayed in the event of an RS crash.
> This is because hlog archiving logic only keeps track of flushed storefiles
> and not compacted ones.
> https://issues.apache.org/jira/browse/HBASE-20704?focusedCommentId=16507680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16507680
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
