[ https://issues.apache.org/jira/browse/HBASE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chunhui shen updated HBASE-6311: -------------------------------- Summary: Data error after compaction because keep MVCC for opened scanners (was: Data error after compaction because of scanners) > Data error after compaction because keep MVCC for opened scanners > ----------------------------------------------------------------- > > Key: HBASE-6311 > URL: https://issues.apache.org/jira/browse/HBASE-6311 > Project: HBase > Issue Type: Bug > Components: regionserver > Affects Versions: 0.94.0 > Reporter: chunhui shen > Assignee: chunhui shen > Priority: Blocker > Attachments: HBASE-6311-test.patch, HBASE-6311v1.patch > > > It is a big problem we found in 0.94, and you could reproduce the problem in > Trunk using the test case I uploaded. > When we do compaction, we will use region.getSmallestReadPoint() to keep MVCC > for opened scanners; > However,It will make data mistake after compaction because we will skip > delete type KV but keep the put type kv in the compacted storefile. > The following is the reason from code: > In StoreFileScanner, enforceMVCC is false when compaction, so we could read > the delete type KV, > However, we will skip this delete type KV in ScanQueryMatcher because > following code > {code} > if (kv.isDelete()) > { > ... > if (includeDeleteMarker > && kv.getMemstoreTS() <= maxReadPointToTrackVersions) { > System.out.println("add deletes,maxReadPointToTrackVersions=" > + maxReadPointToTrackVersions); > this.deletes.add(bytes, offset, qualLength, timestamp, type); > } > ... > } > {code} > Here maxReadPointToTrackVersions = region.getSmallestReadPoint(); > and kv.getMemstoreTS() > maxReadPointToTrackVersions > So we won't add this to DeleteTracker. > Why test case passed if remove the line > MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint); > Because in the StoreFileScanner#skipKVsNewerThanReadpoint > {code} > if (cur.getMemstoreTS() <= readPoint) { > cur.setMemstoreTS(0); > } > {code} > So if we remove the line > MultiVersionConsistencyControl.setThreadReadPoint(smallestReadPoint); > Here readPoint is LONG.MAX_VALUE, we will set memStore ts as 0, so we will > add it to DeleteTracker in ScanQueryMatcher -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira