I've had the exact same problem with my own udf and joda. It turn out that an old version of joda is embedded inside the pig application. My workaround was limitying myself to the old joda api.
Niels On Aug 13, 2013 10:31 AM, "Darpan R" <darpa...@gmail.com> wrote: > Thanks Serega and Ankit. > > According to my understanding it looks that the actual joda-time jar is in > the classpath (otherwise it would have failed saying no classDef found). > But it is not able to find the "now" static method which is part of the > DateTime class. > Not sure if I am missing something. > > The transformation I am doing on the date field using joda date time API. > > Let me paste the complete stack trace : > > > 2013-08-11 13:01:06,911 [Thread-9] WARN > org.apache.hadoop.mapred.LocalJobRunner - job_local_0001 > java.lang.NoSuchMethodError: > > org.joda.time.DateTime.now(Lorg/joda/time/DateTimeZone;)Lorg/joda/time/DateTime; > at > com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:178) > at > com.myproject.pig.udf.ExtractDataByDates.exec(ExtractDataByDates.java:12) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:337) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:381) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:334) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:372) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:297) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308) > at > > org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:241) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.runPipeline(PigGenericMapReduce.java:465) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.processOnePackageOutput(PigGenericMapReduce.java:433) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:413) > at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Reduce.reduce(PigGenericMapReduce.java:257) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) > at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418) > at > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:262) > 2013-08-11 13:01:10,417 [main] WARN > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Ooops! Some job has failed! Specify -stop_on_fail > > > > Thanks again. > DR > > On 13 August 2013 11:28, Serega Sheypak <serega.shey...@gmail.com> wrote: > > > Pig uses reflection. The top exception says that there is no such method > > signature. The problem is in the way you are trying to call method. > > And its better to paste the whole stavktrace > > 13.08.2013 7:39 пользователь "Darpan R" <darpa...@gmail.com> написал: > > > > > I've a UDF which I use to do custom processing on the records. In the > > eval > > > function I am using a third party jar for processing. I saw the job jar > > > file, but it does not include this dependency. Is there any way to > > include > > > dependent jar in the job jar ? (For testing I am running the cluster in > > the > > > local mode). > > > > > > Or can I use distributed cache to make the dependent jar available to > the > > > UDF ? > > > > > > I've tried registering the dependent jars in the pig. For the first > > > registered jar (all udfs are bundled in this jar) I do not face the > > issues. > > > But for the second jar, I am facing issues when UDF tries to access the > > > class from it. > > > REGISTER '/home/user/pig/udfrepository/projectUDF.jar' > > > REGISTER '/home/user/thridpartyjars/xyz.jar'; > > > The logs I get on the console are like this : > > > 2013-08-11 10:35:02,485 [Thread-14] WARN > > > org.apache.hadoop.mapred.LocalJobRunner - job_local_0001 > > > java.lang.NoSuchMethodError: > > > org.xyz.abc.convertToOtherFormat(Lorg/DateTimeZone;)Lorg/DateTime; > > > at com.myproject.MyUDF.exec(MyUDF.java:70) > > > Any help on this is highly appreciated. > > > Thanks in advance. > > > > > > -Darpan > > > > > >
