Chris,
I can confirm the problem; just run into it myself :(
In my case I was able to run my code succesfully on MapR cluster (hadoop
0.20.2), with Pig 0.9.1 and Jython 2.5.2.
Now, I am getting same "Deserialization error" after upgrading to Pig 0.10.0
(see log snippets below).
Any ideas/luck so far? Is there anything I can try/test to help?
Marek
-----------------
2012-06-19 16:24:54,470 [main] INFO org.apache.pig.Main - Apache Pig
version 0.10.0 (r1328203) compiled Apr 20 2012, 05:54:07
2012-06-19 16:24:54,470 [main] INFO org.apache.pig.Main - Logging error
messages to: /transfer/marek/poc123-code/pig_1340141094466.log
(...)
2012-06-19 16:26:27,484 [main] ERROR
org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
2012-06-19 16:26:27,488 [main] INFO
org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt
Features
0.20.2-dev 0.10.0 marek 2012-06-19 16:25:13 2012-06-19
16:26:27 FILTER
Failed!
(...)
2012-06-19 16:26:27,488 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLaunch
er - Failed!
2012-06-19 16:26:27,544 [main] ERROR org.apache.pig.tools.grunt.GruntParser
- ERROR 2997: Unable to recreate exception from backed error:
java.io.IOException: Deserialization error: could not instantiate
'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]'
at
org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:
55)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBa
se.setup(PigGenericMapBase.java:177)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:654)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja
va:1109)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/transfer/marek/poc123-code/poc123-udf.py, parse_time_local]'
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:553)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.POUserFunc.instantiateFunc(POUserFunc.java:107)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperat
ors.POUserFunc.readObject(POUserFunc.java:439)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at java.util.ArrayList.readObject(ArrayList.java:593)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at java.util.HashMap.readObject(HashMap.java:1030)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at java.util.ArrayList.readObject(ArrayList.java:593)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at java.util.HashMap.readObject(HashMap.java:1029)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1946)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1870)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
at
java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at
org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:
53)
... 9 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcces
sorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
torAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:521)
... 77 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 0:
Could not initialize: /transfer/marek/poc123-code/poc123-udf.py
at
org.apache.pig.scripting.jython.JythonFunction.<init>(JythonFunction.java:87
)
... 82 more
Caused by: java.lang.IllegalStateException: Could not initialize interpreter
(from file system or classpath) with
/transfer/marek/poc123-code/poc123-udf.py
at
org.apache.pig.scripting.ScriptEngine.getScriptAsStream(ScriptEngine.java:14
5)
at
org.apache.pig.scripting.jython.JythonScriptEngine$Interpreter.init(JythonSc
riptEngine.java:153)
at
org.apache.pig.scripting.jython.JythonScriptEngine.getFunction(JythonScriptE
ngine.java:328)
at
org.apache.pig.scripting.jython.JythonFunction.<init>(JythonFunction.java:52
)
... 82 more
