Re: jython not working in cluster mode

[Mridul Muralidharan]

You might want to raise a JIRA on this - both abs and rel paths should 
be supported ...


Regards,
Mridul

On Friday 03 June 2011 11:15 PM, Daniel Eklund wrote:
Shawn... excellent!..  thank you. it worked.

interestingly, I remember having to use the absolute path in local mode....

daniel

On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan<shawn...@gmail.com>  wrote:

Hi Daniel,

We found it was the absolute path which didn't work. Try relative path
when register your travelocity-udfs.py.

Shawn

On Thu, Jun 2, 2011 at 10:49 PM, Daniel Eklund<doekl...@gmail.com>  wrote:
I am tracing through the code, and on the backend, the pig implementation
of
map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or
pig.reducePlan.

As a shot in the dark I installed jython.jar in the lib directories of
all
the nodes in the cluster, but nothing happened.  I am using the
jython-2.5.0
version.

As I understand, the Register command should be shipping out the udfs (in
py
form) into the distributed cache on the cluster.  Is this true?  Where
would
I see this in HDFS?

thanks,
daniel

On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund<doekl...@gmail.com>
wrote:

Does anyone have followup to this problem?
http://www.mail-archive.com/user@pig.apache.org/msg01502.html

I am getting:
Caused by: java.io.IOException: Deserialization error: could not
instantiate 'org.apache.pig.scripting.jython.JythonFunction' with
arguments
'[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
ant]'
         at

org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
         at

org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
         ... 8 more
Caused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
         at

org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
         at

org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
         at

org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
         at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
         at

java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)