Thanks, Rohini, the tests all pass. As Cheolsoo suggested, trunk is fine with Hadoop 2.0.4-alpha.
Johnny On Mon, May 13, 2013 at 11:55 AM, Rohini Palaniswamy < [email protected]> wrote: > Johnny, > You need to increase PermGen space for 2.x else TestGrunt will hang with > the OOM. Run ant with -DdebugArgs="-XX:+CMSClassUnloadingEnabled > -XX:MaxPermSize=128M" > > Regards, > Rohini > > > On Mon, May 13, 2013 at 10:22 AM, Johnny Zhang <[email protected]> > wrote: > > > I run unit test against 2.0.4-alpha in the trunk. The tests hit > > "OutOfMemoryError: > > PermGen space" in the middle, but so far no failures or errors. I > re-kicked > > off the test. > > > > Johnny > > > > > > On Fri, May 10, 2013 at 6:49 PM, Johnny Zhang <[email protected]> > > wrote: > > > > > Hi, Andrew: > > > I just set up a job to run unit test against 2.0.4-alpha. I will > > > investigate failure and reply to thread. > > > > > > Thanks, > > > Johnny Zhang > > > > > > > > > On Fri, May 10, 2013 at 6:19 PM, Andrew Purtell <[email protected] > > >wrote: > > > > > >> I've tried that, thanks. I did a bit more investigation and it seems > the > > >> issue is recent Hadoop 2 releases. Has anyone tried running Pig unit > > tests > > >> using a more recent Hadoop release than 2.0.0-alpha? Maybe my trouble > > is a > > >> simple thing that someone with more experience with Pig internals > would > > >> see > > >> right away? Cluster testing seems ok. It's just unit tests that fail. > > But > > >> that is concerning. > > >> > > >> I'm trying HEAD of branch-0.11. > > >> > > >> My Java is version "1.6.0_43" Java(TM) SE Runtime Environment (build > > >> 1.6.0_43-b01) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, > mixed > > >> mode). OS is Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64). > > >> > > >> With defaults and only -Dhadoopversion=23 on the Ant command line, it > > >> seems > > >> ok. > > >> > > >> With build.properties of: > > >> > > >> hadoopversion=23 > > >> hadoop-common.version=2.0.4-alpha > > >> hadoop-hdfs.version=2.0.4-alpha > > >> hadoop-mapreduce.version=2.0.4-alpha > > >> > > >> > > >> or defined on the Ant command line, I'll see unit test failures like: > > >> > > >> Testcase: testAccumWithDistinct took 0.868 sec > > >> Caused an ERROR > > >> org/apache/hadoop/mapred/ResourceMgrDelegate > > >> java.lang.NoClassDefFoundError: > > >> org/apache/hadoop/mapred/ResourceMgrDelegate > > >> at > > org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112) > > >> at > > >> > > >> > > > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > > >> at > > org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:94) > > >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81) > > >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74) > > >> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:482) > > >> at > org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461) > > >> at > > >> > > >> > > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152) > > >> at org.apache.pig.PigServer.launchPlan(PigServer.java:1264) > > >> at > > >> > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249) > > >> at org.apache.pig.PigServer.storeEx(PigServer.java:931) > > >> at org.apache.pig.PigServer.store(PigServer.java:898) > > >> at org.apache.pig.PigServer.openIterator(PigServer.java:811) > > >> at > > >> > > >> > > > org.apache.pig.test.TestAccumulator.testAccumWithDistinct(TestAccumulator.java:424) > > >> > > >> That suggests a cause but I've not started spelunking code with the > hope > > >> this is something simple that someone has already encountered. > > >> > > >> > > >> On Sat, May 11, 2013 at 1:31 AM, Johnny Zhang <[email protected]> > > >> wrote: > > >> > > >> > Hi, Andrew: > > >> > Does something like "-Dhadoopversion=23" help ? eg. ant clean test > > >> > -Dhadoopversion=23 -Dtest.junit.output.format=xml > > >> > > > >> > Johnny > > >> > > > >> > > > >> > On Fri, May 10, 2013 at 3:39 AM, Andrew Purtell < > [email protected]> > > >> > wrote: > > >> > > > >> > > Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT > > >> > against > > >> > > Hadoop 2.0.4. 'ant package' and full cluster tests work fine, but > > I'm > > >> not > > >> > > having much luck with running the unit tests, 'ant test-core' or > > 'ant > > >> > > test'. The problem looks to be a MR app classpath issue. > > >> > > > > >> > > Sometimes: java.lang.NoClassDefFoundError: > > >> > > org/apache/hadoop/yarn/client/YarnClientImpl > > >> > > > > >> > > Sometimes: java.lang.NoClassDefFoundError: > > >> > > org/apache/hadoop/mapred/ResourceMgrDelegate > > >> > > > > >> > > A few Google searches have turned up no useful pointers. Maybe > there > > >> is > > >> > > something simple I am missing? How do you set up for running unit > > >> tests > > >> > on > > >> > > your dev boxes? > > >> > > > > >> > > -- > > >> > > Best regards, > > >> > > > > >> > > - Andy > > >> > > > > >> > > Problems worthy of attack prove their worth by hitting back. - > Piet > > >> Hein > > >> > > (via Tom White) > > >> > > > > >> > > > >> > > >> > > >> > > >> -- > > >> Best regards, > > >> > > >> - Andy > > >> > > >> Problems worthy of attack prove their worth by hitting back. - Piet > Hein > > >> (via Tom White) > > >> > > > > > > > > >
