hi, Hi John,
Thanks for the reply. I implemented #2 (*another way*) in ur above post: i debug the logs : in PostScannerOpen() , regionScanner method parameter object is null Also, in preScannerOpen() , i returned *return super.preScannerOpen(e, scan, new DelegateRegionScanner(s)); * in postScannerNext() , internalScanner object is *org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl* #1 way (Put scanner in local map) - may not be possible, cos if two different scan operation with and without attribute hits at the same time, how can we differentiate in postScannerNext. Thanks, Raju, (972)273-0155. On Tue, Jul 11, 2017 at 8:05 AM, Anoop John <[email protected]> wrote: > Ya. It is the same RegionScanner impl in use only being passed. Ya > the param type should have been RegionScanner I guess. We made that > mistake! > -Anoop- > > On Mon, Jul 10, 2017 at 8:37 PM, Ted Yu <[email protected]> wrote: > > The tricky part is that postScannerNext() passes InternalScanner > parameter > > instead of RegionScanner. > > > > FYI > > > > On Sun, Jul 9, 2017 at 10:57 PM, Anoop John <[email protected]> > wrote: > > > >> Ya as Ted said, u are not getting Scan object in the postScannerNext > >> and so can not make use of the attribute in Scan within this hook. > >> Just setting the sharedData variable will cause issue with concurrent > >> scans. (As u imagine) > >> > >> So I can think of solving this in 2 possible ways. (May be more ways > >> possible) > >> > >> 1. U keep a Map within ur CP impl. You implement postScannerOpen > >> where u will get the ref to Scanner been created as well as the Scan. > >> If the Scan is having attribute, keep that scanner within ur Map. > >> During postScannerNext check if the coming in scanner is there in ur > >> Map. If so that means this is the one where u can do the action. > >> Also dont forget to implement postScannerClose and remove that scanner > >> from the Map. Here u might have some perf penalty as u have to add > >> and get from Map which has to be a concurrent map too. > >> > >> Another way > >> > >> 2. Create a custom scanner implementing RegionScanner. The new one > >> has to take an original Region Scanner and just delegate the calls. On > >> postScannerOpen, u will get the original scanner been created and u > >> can just wrap it with ur new scanner object. ( If the Scan object is > >> having required attribute).. In postScannerNext() u can check for ur > >> own RegionScanner type and if so u can do action. > >> > >> > >> -Anoop- > >> > >> > >> On Sat, Jul 8, 2017 at 9:13 PM, Ted Yu <[email protected]> wrote: > >> > if (canUseGetOperation(e)) { > >> > //logic goes here > >> > > >> > Does your Get target the same region being scanned ? > >> > If not, issuing the Get is not advised since the other region may be > >> hosted > >> > on different region server. > >> > > >> > Cheers > >> > > >> > On Thu, Jul 6, 2017 at 7:14 AM, Veerraju Tadimeti <[email protected]> > >> wrote: > >> > > >> >> hi, > >> >> > >> >> I have few questions regarding scope of > *RegionCoprocessorEnvironment* > >> >> sharedData. > >> >> > >> >> > >> >> > >> >> - *Is sharedData map is shared accross all instances > simultaneously > >> ?* > >> >> - I am putting a variable in sharedData in preScannerOpen() > >> based on > >> >> scan attribute, > >> >> - check that variable exists in postScannerNext() then apply > >> logic, > >> >> - remove the variable postScannerClose(). > >> >> - If data is in multiple regions, when one coprocessor removes > >> >> variable in postScannerClose(), will the variable is NULL for > >> another > >> >> region coprocessor in postScannerNext() ? > >> >> > >> >> > >> >> - * Is sharedData map is shared across all the client request > >> >> operations ?* > >> >> > >> >> If a variable is set in sharedData for one client operation(say > SCAN), > >> will > >> >> the variable is available for another client operation(new SCAN) ? > >> >> > >> >> > >> >> - *Will the variables be garbage collected even if we dont > implement > >> >> (removed variables in sharedData) postScannerClose() method* > >> >> > >> >> > >> >> Please find below the logic that I am using currently > >> >> *CODE: * > >> >> > >> >> public RegionScanner > >> >> *preScannerOpen*(ObserverContext<RegionCoprocessorEnvironment> > >> >> e, Scan scan, RegionScanner s) throws IOException { > >> >> byte[] useGetInPostScannerNext = scan.getAttribute(USE_GET_ > >> >> OPERATION_IN_POST_SCANNER_NEXT); > >> >> String useGetInPostScannerNextStr = Bytes.toString( > >> >> useGetInPostScannerNext); > >> >> if (Boolean.parseBoolean(useGetInPostScannerNextStr)) { > >> >> e.getEnvironment().getSharedData().put(USE_GET_ > >> >> OPERATION_IN_POST_SCANNER_NEXT, useGetInPostScannerNextStr); > >> >> } > >> >> return super.preScannerOpen(e, scan, s); > >> >> } > >> >> > >> >> @Override > >> >> public boolean *postScannerNext*(final > >> >> ObserverContext<RegionCoprocessorEnvironment> > >> >> e, > >> >> final InternalScanner s, final List<Result> results, > final > >> int > >> >> limit, > >> >> final boolean hasMore) throws IOException { > >> >> try { > >> >> > >> >> if (canUseGetOperation(e)) { > >> >> > >> >> //logic goes here > >> >> } > >> >> } catch (Exception ex) { > >> >> logger.error("Exception in postScannerNext ", ex); > >> >> throw new IOException(ex); > >> >> } > >> >> return hasMore; > >> >> } > >> >> > >> >> @Override > >> >> public void > >> >> *postScannerClose*(ObserverContext<RegionCoprocessorEnvironment> > >> >> e, InternalScanner s) throws IOException { > >> >> if (canUseGetOperation(e)) { > >> >> e.getEnvironment().getSharedData().remove(USE_ > >> >> GET_OPERATION_IN_POST_SCANNER_NEXT); > >> >> } > >> >> super.postScannerClose(e, s); > >> >> } > >> >> > >> >> private boolean *canUseGetOperation*(final > >> >> ObserverContext<RegionCoprocessorEnvironment> > >> >> e) { > >> >> String useGetOperationInPostScannerNext = (String) > >> >> e.getEnvironment().getSharedData().get(USE_GET_ > >> OPERATION_IN_POST_SCANNER_ > >> >> NEXT); > >> >> return Boolean.parseBoolean(useGetOperationInPostScannerNe > xt); > >> >> } > >> >> > >> >> Thanks, > >> >> Raju, > >> >> (972)273-0155 <(972)%20273-0155>. > >> >> > >> >
