Julio, If I'm reading it right it looks like JIRA-3714 addresses this issue.
Thanks for including your workaround ( adding -c flag ). That seems to be a better workaround than the one we were using. -----Original Message----- From: Julio Lopez [mailto:[email protected]] Sent: Sunday, April 24, 2011 2:17 AM To: [email protected] Subject: Re: Help with NPE during bulk load (completebulkload) Hi Andy, Thanks for the reply. We are running 0.90.2. I tried the -Dhbase.cluster.distributed=true option as well and got a little further, but the main issue is that none of the configuration parameters are being loaded from hbase-site.xml. Below is what I posted to #hbase on IRC. Cheers, -Julio ----- After skimming over the code and running the client in a debugger, it seems that the tools run through the Hadoop/HBase driver (org.apache.hadoop.hbase.mapreduce.Driver) do not implicitly add hbase-site.xml to the configuration object. Also, the default configuration objects for these tools (or at least the completebulkload one) are of the org.apache.hadoop.conf.Configuration type and not org.apache.hadoop.hbase.HBaseConfiguration. Quick workaround (not documented as far as I can tell, and probably should be), use the -c option when executing the command to specify a configuration file (i.e., hbase-site.xml in this case) Alternatively, write a short java program that uses the LoadIncrementalHFiles class. The real solution IMHO would be to initialize LoadIncrementalHFiles in the driver with a org.apache.hadoop.hbase.HBaseConfiguration so hbase-site.xml gets loaded. ----- Begin forwarded message: > Date: April 24, 2011 3:26:41 AM EDT > Subject: RE: Help with NPE during bulk load (completebulkload) > Source: [email protected] Archives > Author: Andy Sautins > > http://mail-archives.apache.org/mod_mbox/hbase-user/201104.mbox/%3c18CDDC753AF02C40BF7BCFFECE23E981057B286EC3@rpcoex01.rpcorp.local%3e > > What version of HBase are you running? We had a similar issue and were able > to work around > it by passing -Dhbase.cluster.distributed=true on the command line to > completebulkload. > > -----Original Message----- > Sent: Saturday, April 23, 2011 2:26 PM > To: [email protected] > Subject: Help with NPE during bulk load (completebulkload) > > Folks, some help would be greatly appreciated. > > I'm getting the following NPE while trying a bulk load with the > completebulkload tool (see > command below). The error is in the zookeeper code that reads the zk config. > The possible > NPs seem to be: > - conf > - value > > The following is in hbase-site.xml > > --------------- > <property> > <name>hbase.cluster.distributed</name> > <value>true</value> > </property> > --------------- > > The hbase shell works fine, I can connect to HBase and get description of the > tables. > > > The zoo.cfg file is also included. > > Has anybody experienced a similar issue? > > Thanks in advance, > > -Julio > > ----- command ---- > > HADOOP_CLASSPATH=${HBASE_CONF_DIR} hadoop --config ${HADOOP_CONF_DIR} \ > jar /usr/local/sw/hbase/hbase-0.90.2.jar completebulkload \ > bulkgen/output/files usertable > > --- code snippet from > org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167) > --- > > > // Special case for 'hbase.cluster.distributed' property being 'true' > if (key.startsWith("server.")) { > 167: ====> if > (conf.get(HConstants.CLUSTER_DISTRIBUTED).equals(HConstants.CLUSTER_IS_DISTRIBUTED) > && value.startsWith("localhost")) { > --------------------- > > > > > /usr/local/sw/hadoop/bin/hadoopException in thread "main" > java.lang.NullPointerException > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.parseZooCfg(ZKConfig.java:167) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:69) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:250) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:113) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:998) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:303) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:294) > at > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:156) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:167) > at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:311) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > at > org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:318) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at org.apache.hadoop.hbase.mapreduce.Driver.main(Driver.java:49) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > > -------------- zoo.cfg -------------- > # The number of milliseconds of each tick > tickTime=2000 > # The number of ticks that the initial > # synchronization phase can take > initLimit=10 > # The number of ticks that can pass between # sending a request and getting > an acknowledgement > syncLimit=5 > # the directory where the snapshot is stored. > dataDir=/var/lib/zk/data > dataLogDir=/var/lib/zk/log > # the port at which the clients will connect > clientPort=2181 > #server.1=zk1:2888:3888 > server.2=zk2:2888:3888 > server.3=zk3:2888:3888 > server.4=zk4:2888:3888 > ---------------------------------------- > Read more... >
