[
https://issues.apache.org/jira/browse/HBASE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-6311:
--------------------------------
Description:
It is a big problem we found in 0.94, and you could reproduce the problem in
Trunk using the test case I uploaded.
When we do compaction, we will use region.getSmallestReadPoint() to keep MVCC
for opened scanners;
However,It will make data mistake after compaction because we will skip delete
type KV but keep the put type kv.
The following is the reason from code:
In StoreFileScanner, enforceMVCC is false when compaction, so we could read the
delete type KV,
However, we will skip this delete type KV in ScanQueryMatcher because following
code
{code}
if (kv.isDelete())
{
...
if (includeDeleteMarker
&& kv.getMemstoreTS() <= maxReadPointToTrackVersions) {
System.out.println("add deletes,maxReadPointToTrackVersions="
+ maxReadPointToTrackVersions);
this.deletes.add(bytes, offset, qualLength, timestamp, type);
}
...
}
{code}
Here maxReadPointToTrackVersions = region.getSmallestReadPoint();
and kv.getMemstoreTS() > maxReadPointToTrackVersions
So we won't add this to DeleteTracker.
Why test case passed if remove the line
MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
Because in the StoreFileScanner#skipKVsNewerThanReadpoint
{code}
if (cur.getMemstoreTS() <= readPoint) {
cur.setMemstoreTS(0);
}
{code}
So if we remove the line
MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will add
it to DeleteTracker in ScanQueryMatcher
was:
It is a big problem found in 0.94, and you could reproduce the problem in Trunk
using the test case I uploaded:
When we do compaction, we will use region.getSmallestReadPoint();
It will make data mistake after compaction because we will skip delete type KV
but keep the put type kv.
The following is the reason:
In StoreFileScanner, enforceMVCC is false when compaction, so we could read the
delete type KV,
However, we will skip this delete type KV in ScanQueryMatcher because following
code
{code}
if (kv.isDelete())
{
...
if (includeDeleteMarker
&& kv.getMemstoreTS() <= maxReadPointToTrackVersions) {
System.out.println("add deletes,maxReadPointToTrackVersions="
+ maxReadPointToTrackVersions);
this.deletes.add(bytes, offset, qualLength, timestamp, type);
}
...
}
{code}
Here maxReadPointToTrackVersions = region.getSmallestReadPoint();
and kv.getMemstoreTS() > maxReadPointToTrackVersions
So we won't add this to DeleteTracker.
Why test case passed if remove the line
MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
Because in the StoreFileScanner#skipKVsNewerThanReadpoint
{code}
if (cur.getMemstoreTS() <= readPoint) {
cur.setMemstoreTS(0);
}
{code}
So if we remove the line
MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will add
it to DeleteTracker in ScanQueryMatcher
> Data error after compaction because of scanners
> -----------------------------------------------
>
> Key: HBASE-6311
> URL: https://issues.apache.org/jira/browse/HBASE-6311
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.94.0
> Reporter: chunhui shen
> Assignee: chunhui shen
> Priority: Blocker
> Attachments: HBASE-6311-test.patch, HBASE-6311v1.patch
>
>
> It is a big problem we found in 0.94, and you could reproduce the problem in
> Trunk using the test case I uploaded.
> When we do compaction, we will use region.getSmallestReadPoint() to keep MVCC
> for opened scanners;
> However,It will make data mistake after compaction because we will skip
> delete type KV but keep the put type kv.
> The following is the reason from code:
> In StoreFileScanner, enforceMVCC is false when compaction, so we could read
> the delete type KV,
> However, we will skip this delete type KV in ScanQueryMatcher because
> following code
> {code}
> if (kv.isDelete())
> {
> ...
> if (includeDeleteMarker
> && kv.getMemstoreTS() <= maxReadPointToTrackVersions) {
> System.out.println("add deletes,maxReadPointToTrackVersions="
> + maxReadPointToTrackVersions);
> this.deletes.add(bytes, offset, qualLength, timestamp, type);
> }
> ...
> }
> {code}
> Here maxReadPointToTrackVersions = region.getSmallestReadPoint();
> and kv.getMemstoreTS() > maxReadPointToTrackVersions
> So we won't add this to DeleteTracker.
> Why test case passed if remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Because in the StoreFileScanner#skipKVsNewerThanReadpoint
> {code}
> if (cur.getMemstoreTS() <= readPoint) {
> cur.setMemstoreTS(0);
> }
> {code}
> So if we remove the line
> MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint);
> Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will
> add it to DeleteTracker in ScanQueryMatcher
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
