Hi everyone, I am having some weird classpath issues with a UDF that returns a custom tuple. My custom tuple has an arraylist of custom objects. It looks like: class MyTuple private ArrayList<MyClass> list;
When the UDF is called, everything works fine: the tuples are created and the UDF returns successfully. But then in the next step when I try to dump the results of the UDF I see class not found exceptions. That is, MyClass can be found during UDF execution, but it cannot be found while dumping the UDF results. I tried setting PIG_CLASSPATH, but didn't work. The only working solution so far is to add my jar file to hadoop's classpath. I wonder whether this is a bug (PIG_CLASSPATH should work) or I am not setting something correctly. Any help will be greatly appreciated. Thanks, Nezih Here is my pig script: register 'my_jar.jar'; x = load '/etc/passwd' using PigStorage(':') as (username:chararray, f1: chararray, f2: chararray, f3:chararray, f4:chararray); parsed = foreach x generate username, MyParser(*); dump parsed; Here is the stack trace: Caused by: java.io.IOException: Could not find class MyClass, while attempting to de-serialize it at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:293) at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:422) at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:318) at org.apache.pig.data.utils.SedesHelper.readGenericTuple(SedesHelper.java:144) at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:349) at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:318) at org.apache.pig.data.utils.SedesHelper.readGenericTuple(SedesHelper.java:144) at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:349) at org.apache.pig.impl.io.InterRecordReader.nextKeyValue(InterRecordReader.java:113) at org.apache.pig.impl.io.InterStorage.getNext(InterStorage.java:77) at org.apache.pig.impl.io.ReadToEndLoader.getNextHelper(ReadToEndLoader.java:246) at org.apache.pig.impl.io.ReadToEndLoader.getNext(ReadToEndLoader.java:226) at org.apache.pig.backend.hadoop.executionengine.HJob$1.hasNext(HJob.java:111) ... 12 more