Johnny, You need to increase PermGen space for 2.x else TestGrunt will hang with the OOM. Run ant with -DdebugArgs="-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M"
Regards, Rohini On Mon, May 13, 2013 at 10:22 AM, Johnny Zhang <xiao...@cloudera.com> wrote: > I run unit test against 2.0.4-alpha in the trunk. The tests hit > "OutOfMemoryError: > PermGen space" in the middle, but so far no failures or errors. I re-kicked > off the test. > > Johnny > > > On Fri, May 10, 2013 at 6:49 PM, Johnny Zhang <xiao...@cloudera.com> > wrote: > > > Hi, Andrew: > > I just set up a job to run unit test against 2.0.4-alpha. I will > > investigate failure and reply to thread. > > > > Thanks, > > Johnny Zhang > > > > > > On Fri, May 10, 2013 at 6:19 PM, Andrew Purtell <apurt...@apache.org > >wrote: > > > >> I've tried that, thanks. I did a bit more investigation and it seems the > >> issue is recent Hadoop 2 releases. Has anyone tried running Pig unit > tests > >> using a more recent Hadoop release than 2.0.0-alpha? Maybe my trouble > is a > >> simple thing that someone with more experience with Pig internals would > >> see > >> right away? Cluster testing seems ok. It's just unit tests that fail. > But > >> that is concerning. > >> > >> I'm trying HEAD of branch-0.11. > >> > >> My Java is version "1.6.0_43" Java(TM) SE Runtime Environment (build > >> 1.6.0_43-b01) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01, mixed > >> mode). OS is Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64). > >> > >> With defaults and only -Dhadoopversion=23 on the Ant command line, it > >> seems > >> ok. > >> > >> With build.properties of: > >> > >> hadoopversion=23 > >> hadoop-common.version=2.0.4-alpha > >> hadoop-hdfs.version=2.0.4-alpha > >> hadoop-mapreduce.version=2.0.4-alpha > >> > >> > >> or defined on the Ant command line, I'll see unit test failures like: > >> > >> Testcase: testAccumWithDistinct took 0.868 sec > >> Caused an ERROR > >> org/apache/hadoop/mapred/ResourceMgrDelegate > >> java.lang.NoClassDefFoundError: > >> org/apache/hadoop/mapred/ResourceMgrDelegate > >> at > org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112) > >> at > >> > >> > org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34) > >> at > org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:94) > >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81) > >> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74) > >> at org.apache.hadoop.mapred.JobClient.init(JobClient.java:482) > >> at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461) > >> at > >> > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:152) > >> at org.apache.pig.PigServer.launchPlan(PigServer.java:1264) > >> at > >> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249) > >> at org.apache.pig.PigServer.storeEx(PigServer.java:931) > >> at org.apache.pig.PigServer.store(PigServer.java:898) > >> at org.apache.pig.PigServer.openIterator(PigServer.java:811) > >> at > >> > >> > org.apache.pig.test.TestAccumulator.testAccumWithDistinct(TestAccumulator.java:424) > >> > >> That suggests a cause but I've not started spelunking code with the hope > >> this is something simple that someone has already encountered. > >> > >> > >> On Sat, May 11, 2013 at 1:31 AM, Johnny Zhang <xiao...@cloudera.com> > >> wrote: > >> > >> > Hi, Andrew: > >> > Does something like "-Dhadoopversion=23" help ? eg. ant clean test > >> > -Dhadoopversion=23 -Dtest.junit.output.format=xml > >> > > >> > Johnny > >> > > >> > > >> > On Fri, May 10, 2013 at 3:39 AM, Andrew Purtell <apurt...@apache.org> > >> > wrote: > >> > > >> > > Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT > >> > against > >> > > Hadoop 2.0.4. 'ant package' and full cluster tests work fine, but > I'm > >> not > >> > > having much luck with running the unit tests, 'ant test-core' or > 'ant > >> > > test'. The problem looks to be a MR app classpath issue. > >> > > > >> > > Sometimes: java.lang.NoClassDefFoundError: > >> > > org/apache/hadoop/yarn/client/YarnClientImpl > >> > > > >> > > Sometimes: java.lang.NoClassDefFoundError: > >> > > org/apache/hadoop/mapred/ResourceMgrDelegate > >> > > > >> > > A few Google searches have turned up no useful pointers. Maybe there > >> is > >> > > something simple I am missing? How do you set up for running unit > >> tests > >> > on > >> > > your dev boxes? > >> > > > >> > > -- > >> > > Best regards, > >> > > > >> > > - Andy > >> > > > >> > > Problems worthy of attack prove their worth by hitting back. - Piet > >> Hein > >> > > (via Tom White) > >> > > > >> > > >> > >> > >> > >> -- > >> Best regards, > >> > >> - Andy > >> > >> Problems worthy of attack prove their worth by hitting back. - Piet Hein > >> (via Tom White) > >> > > > > >
