On Fri, Sep 16, 2011 at 12:01 PM, Douglas Campbell <[email protected]> wrote: > I'm reducing keys by regions and then building a Scan with > a. start/stop = minkey/max+1 > b. 100 cache rows > c. cache blocks false > d. configuring scan like this > public static void setRowFilters(Scan scan, Iterator<byte[]> keys) { > FilterList filterlist = new FilterList(Operator.MUST_PASS_ONE); > for (; keys.hasNext(); ) { > filterlist.addFilter(new RowFilter(CompareOp.EQUAL, new > BinaryComparator(keys.next()))); > } > scan.setFilter(filterlist); > } > > where the iterator contains the keys I want
The min/max keys are for each region right? Are they pretty big? > > Are you saying I should also re-instantiate the htable? No, I'm saying that the HTable should take care of this, not the user. > > Also, looking at Regionserver stack trace shows the following blocked threads > > "IPC Server handler 19 on 60020" daemon prio=10 tid=0x0000000053838000 > nid=0x3224 waiting for monitor entry [0x0000000045799000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2322) > - waiting to lock <0x0000000782a96ad8> (a > org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1823) > at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > > "IPC Server handler 18 on 60020" daemon prio=10 tid=0x0000000053835800 > nid=0x3223 waiting for monitor entry [0x0000000045698000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2322) > - waiting to lock <0x000000071d024b60> (a > org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1823) > at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039) > > "IPC Server handler 17 on 60020" daemon prio=10 tid=0x0000000053833800 > nid=0x3222 waiting for monitor entry [0x0000000045597000] > java.lang.Thread.State: BLOCKED (on object monitor) > at org.apache.hadoop.io.compress.CodecPool.payback(CodecPool.java:75) > - locked <0x00000006fbe527f8> (a java.util.HashMap) > at > org.apache.hadoop.io.compress.CodecPool.returnDecompressor(CodecPool.java:152) > at > org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.returnDecompressor(Compression.java:245) > at > org.apache.hadoop.hbase.io.hfile.HFile$Reader.decompress(HFile.java:1098) > at > org.apache.hadoop.hbase.io.hfile.HFile$Reader.readBlock(HFile.java:1036) > - locked <0x000000074aa2a950> (a [B) > at > org.apache.hadoop.hbase.io.hfile.HFile$Reader$Scanner.next(HFile.java:1276) > at > org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:115) > at > org.apache.hadoop.hbase.regionserver.KeyValueHeap.reseek(KeyValueHeap.java:255) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:394) Are you sharing scanners between multiple threads? J-D
