Pig 0.12.0 with HBase 0.96.0-hadoop2 WritableByteArrayComparable class not found

[Tak-Lon (Stephen) Wu]

Hi,

I am wondering if anyone has used Pig with Yarn and HBase 0.96.0-hadoop2.

I got the following error message:

ERROR 2998: Unhandled internal error.
org/apache/hadoop/hbase/filter/WritableByteArrayComparable

java.lang.NoClassDefFoundError:
org/apache/hadoop/hbase/filter/WritableByteArrayComparable
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:270)
        at
org.apache.pig.impl.PigContext.resolveClassName(PigContext.java:635)
        at
org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:1257)
        at
org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:1245)
        at
org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:5069)
        at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3426)
        at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1536)
        at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1013)
        at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:553)
        at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421)
        at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1688)
        at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1421)
        at org.apache.pig.PigServer.parseAndBuild(PigServer.java:342)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:367)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:353)
        at
org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:202)
        at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:608)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.filter.WritableByteArrayComparable
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

Does it means the HBaseStorage in pig 0.12.0 has changed their interface
and is now incompatible with HBase 0.96.x?

In addition, I tried to include the previous hbase 0.94.17 jar file
(compiled in hadoop 2.2.0), but it got hangs when loading so data from
tables. So, if you have any suggestion, that would be so helpful.

Best,
Stephen



-- 
Stephen Wu
Pervasive Technology Institute
Indiana University, Bloomington