[
https://issues.apache.org/jira/browse/PIG-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209982#comment-13209982
]
Travis Crawford commented on PIG-2532:
--------------------------------------
Status update:
I reproduced the issue in a unit test, and verified the patch fixes the issue.
The issue presents itself when a jar is registered at runtime, and a class in
that registered jar is stored in the UDFContext. Without the patch
deserilization fails.
To setup the test I store two java files as resources, and compile+jar them at
runtime (using javax classes). I took this approach to keep the test setup
isolated.
However, since it uses javax classes this approach may not be ideal. I'm
preparing another version that performs these setup steps in ant (generate the
test-specific jar, runs the test with an appropriate classpath. That version
will likely be the better choice to commit.
> Registered classes fail deserialization in frontend
> ---------------------------------------------------
>
> Key: PIG-2532
> URL: https://issues.apache.org/jira/browse/PIG-2532
> Project: Pig
> Issue Type: Bug
> Reporter: Travis Crawford
> Assignee: Travis Crawford
> Attachments: PIG-2532.patch
>
>
> This issue came up while integrating HCatalog with our environment. HCatalog
> jars are added to the pig command-line with {{-Dpig.additional.jars}} but
> fails (exception below). When added to the pig classpath the error goes away.
> We identified the issue as deserialization using the root class loader, not
> the context class loader set when the thread is created. This causes
> HCatSchema which is serialized into the context to fail deserialization in
> the thread.
> {code}
> 2012-02-14 21:55:53,936 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR
> 6017: java.io.IOException: Deserialization error:
> org.apache.hcatalog.data.schema.HCatSchema
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:181)
> at
> org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFContext(MapRedUtil.java:159)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:229)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
> at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:811)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
> at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
> at
> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.mainLoopAction(PigJobControl.java:144)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.run(PigJobControl.java:121)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hcatalog.data.schema.HCatSchema
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:247)
> at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
> at
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> at java.util.Hashtable.readObject(Hashtable.java:859)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> at java.util.HashMap.readObject(HashMap.java:1030)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
> ... 15 more
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
