[
https://issues.apache.org/jira/browse/HBASE-13389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482550#comment-14482550
]
stack commented on HBASE-13389:
-------------------------------
[~lhofhansl] 3 days is arbitrary. If it so happens that someone has failed to
observe a backed up replication and don't notice it till a week has gone by,
will they lose data? Can we not have the optimization instead cut in when it is
guaranteed the mvcc is no longer needed?
In-cluster, it seems a matter of hours will do it as long as we check that
locally flushing is working. Below are a few 'statements' of why I think it
should be fine. For replication, as I read it, edits get a new seqid when
applied to the sink cluster so source cluster seqid doesn't factor in at all.
Maybe I misread.
As per Enis, I don't get how idempotency is effected.
Notes on in-cluster:
* For log replay in-cluster, when Distributed Log Replay, if a Region gets an
edit with a seqid that fits inside a range covered by an existing hfile, then
we can just drop it because it already persisted. This would be for case were
an old WAL is being replayed though the edits have been flushed out to hfiles
already (and the optimization dropping mvcc has been run).
* If a region can't flush, then we should not run the optimization (This is
probably ok... compaction will likely just not succeed if we can't flush but
optimization should check last flush time).
* If no replication, optimization can run on any file as long as no
outstanding scanners and read point is beyond the oldest edit in an hfile
(optimization does this now I believe).
> [REGRESSION] HBASE-12600 undoes skip-mvcc parse optimizations
> -------------------------------------------------------------
>
> Key: HBASE-13389
> URL: https://issues.apache.org/jira/browse/HBASE-13389
> Project: HBase
> Issue Type: Sub-task
> Components: Performance
> Reporter: stack
> Attachments: 13389.txt
>
>
> HBASE-12600 moved the edit sequenceid from tags to instead exploit the
> mvcc/sequenceid slot in a key. Now Cells near-always have an associated
> mvcc/sequenceid where previous it was rare or the mvcc was kept up at the
> file level. This is sort of how it should be many of us would argue but as a
> side-effect of this change, read-time optimizations that helped speed scans
> were undone by this change.
> In this issue, lets see if we can get the optimizations back -- or just
> remove the optimizations altogether.
> The parse of mvcc/sequenceid is expensive. It was noticed over in HBASE-13291.
> The optimizations undone by this changes are (to quote the optimizer himself,
> Mr [~lhofhansl]):
> {quote}
> Looks like this undoes all of HBASE-9751, HBASE-8151, and HBASE-8166.
> We're always storing the mvcc readpoints, and we never compare them against
> the actual smallestReadpoint, and hence we're always performing all the
> checks, tests, and comparisons that these jiras removed in addition to
> actually storing the data - which with up to 8 bytes per Cell is not trivial.
> {quote}
> This is the 'breaking' change:
> https://github.com/apache/hbase/commit/2c280e62530777ee43e6148fd6fcf6dac62881c0#diff-07c7ac0a9179cedff02112489a20157fR96
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
