> On 2010-06-07 14:23:42, stack wrote: > > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java, line 96 > > <http://review.hbase.org/r/98/diff/5/?file=944#file944line96> > > > > getRowOrBefore is an expensive call. Are we sure we are not calling > > this too often?
I agree it is an expensive call. However I don't think it would bring any performance penalty for existing and potential use cases: Use case 1 -- existing MetaScanner users: since this method is newly added, existing users won't be affected; Use case 2 -- hbase clients when locating a region : 1) if prefetch is on, it calls this MetaScanner with [table + row combination], which calls getRowOrBefore() to get current region info, then number of following regions from meta. After that, the client can get the region info directly from cache. 2) if prefetch is disabled (current behavior), it eventually calls similar method getClosestRowBefore() to get desired region. So no matter prefetch is on or not, getRowOrBefore(or getClosestRowBefore) eventually is called. The only difference is whether to scan following regions from meta or not. For future MetaScanner users which scan from one region with desired use table row, it has to take the effort since it is the expected behavior. - Mingjie ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.hbase.org/r/98/#review144 ----------------------------------------------------------- On 2010-06-02 01:13:42, Mingjie Lai wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.hbase.org/r/98/ > ----------------------------------------------------------- > > (Updated 2010-06-02 01:13:42) > > > Review request for hbase. > > > Summary > ------- > > HBASE-2468: Improvements to prewarm META cache on clients. > > Changes: > 1. Add new HTable methods which support region info de/serialation, and > region cache prewarm: > - void serializeRegionInfo(): clients could perform a large scan for all the > meta for the table, serialize the meta to a file. MR job can ship a copy of > the meta for the table in the DistributedCache > - Map<HRegionInfo, HServerAddress> deserializeRegionInfo(): MR job can > deserialize the region info from the DistributedCache > - prewarmRegionCache(Map<HRegionInfo, HServerAddress> regionMap): MR job can > prewarm local region cache by the deserialized region info. > > 2. For each client, each region cache read-miss could trigger read-ahead some > number of rows in META. This option could be turned on and off for one > particular table. > > > This addresses bug HBASE-2468. > http://issues.apache.org/jira/browse/HBASE-2468 > > > Diffs > ----- > > src/main/java/org/apache/hadoop/hbase/client/HConnection.java 853164d > src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java > ed18092 > src/main/java/org/apache/hadoop/hbase/client/HTable.java 7ec95cb > src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java d3a0c07 > src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java > 95e494a > > Diff: http://review.hbase.org/r/98/diff > > > Testing > ------- > > Unit tests passed locally for me. > > > Thanks, > > Mingjie > >
