[
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827083#comment-13827083
]
Jarek Jarcec Cecho commented on PIG-3285:
-----------------------------------------
Thank you for the quick turnaround [~ndimiduk]!
I was able to run {{TestHBaseStorage}} unit tests for all the combinations
using following commands:
{code}
# Old path:
ant clean test -Dhbaseversion=94 -Dtestcase=TestHBaseStorage
ant clean test -Dhbaseversion=95 -Dtestcase=TestHBaseStorage
-Dprotobuf-java.version=2.5.0
# New path:
# Compile HBase 0.94 branch and install it to local maven cache (mvn clean
install -DskipTests)
ant clean test -Dhbaseversion=94 -Dtestcase=TestHBaseStorage
-Dhbase94.version=0.94.14
# Compile HBase 0.96 branch and install it to local maven cache (mvn clean
install -DskipTests)
ant clean test -Dhbaseversion=95 -Dtestcase=TestHBaseStorage
-Dprotobuf-java.version=2.5.0 -Dhbase95.version=0.96.1-SNAPSHOT
{code}
The only problem that I've encounter so far is the exception
{{ReflectiveOperationException}}
([javadoc|http://docs.oracle.com/javase/7/docs/api/java/lang/ReflectiveOperationException.html])
that has been added in JDK7, while Pig still requires JDK6 compatibility. I've
substituted the exception with {{Exception}} for the purpose of above tests.
I'll try to give it a spin on a real clusters to see if everything will work as
expected.
> Jobs using HBaseStorage fail to ship dependency jars
> ----------------------------------------------------
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
> Issue Type: Bug
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies.
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.<clinit>(HbaseObjectWritable.java:266)
> at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
> at $Proxy7.getProtocolVersion(Unknown Source)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
