Re: Unit test classpath trouble

2013-05-14 Thread Johnny Zhang
Thanks, Rohini, the tests all pass. As Cheolsoo suggested, trunk is fine with Hadoop 2.0.4-alpha. Johnny On Mon, May 13, 2013 at 11:55 AM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Johnny, You need to increase PermGen space for 2.x else TestGrunt will hang with the OOM. Run ant

Re: Unit test classpath trouble

2013-05-13 Thread Konstantin Boudnik
If unit tests can't be run - which seems to be the case - it sound like a release blocker to me. Any plans to back port this fix into 0.11.2? 0.11.1 doesn't have it, right? Cos On Sun, May 12, 2013 at 09:12PM, Cheolsoo Park wrote: Hi Andrew, I'm trying HEAD of branch-0.11. You need

Re: Unit test classpath trouble

2013-05-13 Thread Andrew Purtell
Cheolsoo, Thank you so much. That was the problem. On Mon, May 13, 2013 at 12:12 PM, Cheolsoo Park piaozhe...@gmail.comwrote: Hi Andrew, I'm trying HEAD of branch-0.11. You need PIG-3248: https://issues.apache.org/jira/browse/PIG-3248. It's not in branch-0.11 but in trunk. In

Re: Unit test classpath trouble

2013-05-13 Thread Cheolsoo Park
Hi Cos, I am happy to backport it to branch-011. But I am not entirely sure whether we should bump the Hadoop version in a minor release. As of now, branch-0.11 unit tests run fine with 2.0.0-alpha. Please let me know if anyone has an opinion on this. Thanks, Cheolsoo On Sun, May 12, 2013 at

Re: Unit test classpath trouble

2013-05-13 Thread Johnny Zhang
I run unit test against 2.0.4-alpha in the trunk. The tests hit OutOfMemoryError: PermGen space in the middle, but so far no failures or errors. I re-kicked off the test. Johnny On Fri, May 10, 2013 at 6:49 PM, Johnny Zhang xiao...@cloudera.com wrote: Hi, Andrew: I just set up a job to run

Re: Unit test classpath trouble

2013-05-13 Thread Rohini Palaniswamy
Johnny, You need to increase PermGen space for 2.x else TestGrunt will hang with the OOM. Run ant with -DdebugArgs=-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M Regards, Rohini On Mon, May 13, 2013 at 10:22 AM, Johnny Zhang xiao...@cloudera.com wrote: I run unit test against

Re: Unit test classpath trouble

2013-05-13 Thread Konstantin Boudnik
Cheolsoo, just to make sure I do not misinterpret your words: - there were a change in YARN packaging between 2.0.0 and 2.0.3 that move some APIs into separate artifact hadoop-yarn-client - adding hadoop-yarn-client is only relevant for 2.0.3 and later and won't work on 2.0.0-2.0.2

Re: Unit test classpath trouble

2013-05-12 Thread Andrew Purtell
It happens consistently on multiple systems and different operating systems and JDK versions, so while is a build (or more specifically a unit test environment) problem, it is unlikely an issue singular to me. On Saturday, May 11, 2013, Konstantin Boudnik wrote: As you know we've been testing

Re: Unit test classpath trouble

2013-05-12 Thread Cheolsoo Park
Hi Andrew, I'm trying HEAD of branch-0.11. You need PIG-3248: https://issues.apache.org/jira/browse/PIG-3248. It's not in branch-0.11 but in trunk. In particular, you need to add the following lines to ivy.xml: +dependency org=org.apache.hadoop name=hadoop-yarn-client +

Unit test classpath trouble

2013-05-10 Thread Andrew Purtell
Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT against Hadoop 2.0.4. 'ant package' and full cluster tests work fine, but I'm not having much luck with running the unit tests, 'ant test-core' or 'ant test'. The problem looks to be a MR app classpath issue. Sometimes:

Re: Unit test classpath trouble

2013-05-10 Thread Johnny Zhang
Hi, Andrew: Does something like -Dhadoopversion=23 help ? eg. ant clean test -Dhadoopversion=23 -Dtest.junit.output.format=xml Johnny On Fri, May 10, 2013 at 3:39 AM, Andrew Purtell apurt...@apache.org wrote: Please pardon the basic question. I'm building Pig 0.11.2-SNAPSHOT against Hadoop

Re: Unit test classpath trouble

2013-05-10 Thread Andrew Purtell
I've tried that, thanks. I did a bit more investigation and it seems the issue is recent Hadoop 2 releases. Has anyone tried running Pig unit tests using a more recent Hadoop release than 2.0.0-alpha? Maybe my trouble is a simple thing that someone with more experience with Pig internals would see

Re: Unit test classpath trouble

2013-05-10 Thread Johnny Zhang
Hi, Andrew: I just set up a job to run unit test against 2.0.4-alpha. I will investigate failure and reply to thread. Thanks, Johnny Zhang On Fri, May 10, 2013 at 6:19 PM, Andrew Purtell apurt...@apache.org wrote: I've tried that, thanks. I did a bit more investigation and it seems the issue

Re: Unit test classpath trouble

2013-05-10 Thread Konstantin Boudnik
As you know we've been testing Pig 0.11 vs 2.0.4-alpha as a part of Bigtop's validation for the latest hadoop release and it worked ok. Bigtop doesn't run unit tests though, so it seems like a build issue to me. Cos On Sat, May 11, 2013 at 09:19AM, Andrew Purtell wrote: I've tried that, thanks.