On Wed, Feb 10, 2010 at 9:03 AM, Vincent Barat <[email protected]> wrote: > Hello, > > I'm working on optimizing the HBase loader for PIG.
Thanks for doing this. > Here is what I have in hbase-default.xml: You should edit hbase-site.xml, not hbase-default.xml. > > <property> > <name>hbase.client.scanner.caching</Name> > <value>1000</value> > </property> > > Is there something I missed ? > It must be a CLASSPATH issue. Are you adding the conf dir to your CLASSPATH ahead of the hbase jar (hbase jar has hbase-*.xml in it so may be found before your customizations). There is also "hbase.client.scanner.max.result.size" which is not mentioned in the config. It allows you to set a maximum size rather than count of rows on what a scanner gets in one go. This might be more portable than an absolute count of rows for what you are doing; i.e. it'll work about the same if the PIG input is small or big rows. See https://issues.apache.org/jira/browse/HBASE-1996 for more on this value. St.Ack
