[ https://issues.apache.org/jira/browse/HBASE-19001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16207626#comment-16207626 ]
Duo Zhang commented on HBASE-19001: ----------------------------------- {quote} In the wrapper outer scanner, we will get all the Cells (even which might get removed by SQM logics). {quote} If you mean the SQM for flush/compaction then no, we can not get the cells which are removed by SQM as the InternalScanner is a StoreScanner actually and the returned cells have already been processed by SQM. So the solution here is that, if you only want to keep n versions but may have m versions for an intermediate state where m > n, then you need to set the MAX_VERSIONS option to m for the family, and filter out unnecessary versions by yourself. > Remove the hooks in RegionObserver which are designed to construct a > StoreScanner which is marked as IA.Private > --------------------------------------------------------------------------------------------------------------- > > Key: HBASE-19001 > URL: https://issues.apache.org/jira/browse/HBASE-19001 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors > Reporter: Duo Zhang > Assignee: Duo Zhang > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19001-v1.patch, HBASE-19001.patch > > > There are three methods here > {code} > KeyValueScanner > preStoreScannerOpen(ObserverContext<RegionCoprocessorEnvironment> c, > Store store, Scan scan, NavigableSet<byte[]> targetCols, > KeyValueScanner s, long readPt) > throws IOException; > InternalScanner > preFlushScannerOpen(ObserverContext<RegionCoprocessorEnvironment> c, > Store store, List<KeyValueScanner> scanners, InternalScanner s, long > readPoint) > throws IOException; > InternalScanner > preCompactScannerOpen(ObserverContext<RegionCoprocessorEnvironment> c, > Store store, List<? extends KeyValueScanner> scanners, ScanType > scanType, long earliestPutTs, > InternalScanner s, CompactionLifeCycleTracker tracker, > CompactionRequest request, > long readPoint) throws IOException; > {code} > For the flush and compact ones, we've discussed many times, it is not safe to > let user inject a Filter or even implement their own InternalScanner using > the store file scanners, as our correctness highly depends on the complicated > logic in SQM and StoreScanner. CP users are expected to wrap the original > InternalScanner(it is a StoreScanner anyway) in preFlush/preCompact methods > to do filtering or something else. > For preStoreScannerOpen it even returns a KeyValueScanner which is marked as > IA.Private... This is less hurt but still, we've decided to not expose > StoreScanner to CP users so here this method is useless. CP users can use > preGetOp and preScannerOpen method to modify the Get/Scan object passed in to > inject into the scan operation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)