As you know we've been testing Pig 0.11 vs 2.0.4-alpha as a part of Bigtop's validation for the latest hadoop release and it worked ok. Bigtop doesn't run unit tests though, so it seems like a build issue to me.
Cos On Sat, May 11, 2013 at 09:19AM, Andrew Purtell wrote: > I've tried that, thanks. I did a bit more investigation and it seems the > issue is recent Hadoop 2 releases. Has anyone tried running Pig unit tests > using a more recent Hadoop release than 2.0.0-alpha? Maybe my trouble is a > simple thing that someone with more experience with Pig internals would see > right away? Cluster testing seems ok. It's just unit tests that fail. But > that is concerning. > > I'm trying HEAD of branch-0.11. > > My Java is version "1.6.0_43" Java(TM) SE Runtime Environment (build > 1.6.0_43-b01) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed > mode). OS is Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64). > > With defaults and only -Dhadoopversion=23 on the Ant command line, it seems > ok. > > With build.properties of: > > hadoopversion=23 > hadoop-common.version=2.0.4-alpha > hadoop-hdfs.version=2.0.4-alpha > hadoop-mapreduce.version=2.0.4-alpha > > > or defined on the Ant command line, I'll see unit test failures like: > > Testcase: testAccumWithDistinct took 0.868 sec > Caused an ERROR > org/apache/hadoop/mapred/ResourceMgrDelegate > java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/ResourceMgrDelegate > at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112) > at > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:94) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81) > at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74) > at org.apache.hadoop.mapred.JobClient.init(JobClient.java:482) > at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152) > at org.apache.pig.PigServer.launchPlan(PigServer.java:1264) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249) > at org.apache.pig.PigServer.storeEx(PigServer.java:931) > at org.apache.pig.PigServer.store(PigServer.java:898) > at org.apache.pig.PigServer.openIterator(PigServer.java:811) > at > org.apache.pig.test.TestAccumulator.testAccumWithDistinct(TestAccumulator.java:424) > > That suggests a cause but I've not started spelunking code with the hope > this is something simple that someone has already encountered. > > > On Sat, May 11, 2013 at 1:31 AM, Johnny Zhang <xiao...@cloudera.com> wrote: > > > Hi, Andrew: > > Does something like "-Dhadoopversion=23" help ? eg. ant clean test > > -Dhadoopversion=23 -Dtest.junit.output.format=xml > > > > Johnny > > > > > > On Fri, May 10, 2013 at 3:39 AM, Andrew Purtell <apurt...@apache.org> > > wrote: > > > > > Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT > > against > > > Hadoop 2.0.4. 'ant package' and full cluster tests work fine, but I'm not > > > having much luck with running the unit tests, 'ant test-core' or 'ant > > > test'. The problem looks to be a MR app classpath issue. > > > > > > Sometimes: java.lang.NoClassDefFoundError: > > > org/apache/hadoop/yarn/client/YarnClientImpl > > > > > > Sometimes: java.lang.NoClassDefFoundError: > > > org/apache/hadoop/mapred/ResourceMgrDelegate > > > > > > A few Google searches have turned up no useful pointers. Maybe there is > > > something simple I am missing? How do you set up for running unit tests > > on > > > your dev boxes? > > > > > > -- > > > Best regards, > > > > > > - Andy > > > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > > (via Tom White) > > > > > > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White)
signature.asc
Description: Digital signature
