Re: Pig 0.10 on YARN

2013-04-09 Thread Konstantin Boudnik
In case you wanna know the actual cause of the issue, please look at BIGTOP-906. It seems that published Pig artifacts are missing pieces of what's available in the fat-pig-jar aka pig.jar. And seemingly the only way to deal with it is to avoid using Maven artifacts and start using the fat one. I

Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
Guys, I am trying to run TestPigTest against a fanctional Hadoop 2.0 YARN cluster - I can run MR jobs using Hive or plain MR examples. However, it is going somewhat unsuccessful. While using pig far jar file - e.g. with everything repacked inside of it - I seeminly can run TestPigTest against a

Re: Pig 0.10 on YARN

2013-03-29 Thread Dmitriy Ryaboy
Can you run `pig -secretDebugCmd` and look at your paths? I suspect there's something going on there, like not having the right configs or hadoop jars. On Thu, Mar 28, 2013 at 11:32 PM, Konstantin Boudnik c...@apache.org wrote: Guys, I am trying to run TestPigTest against a fanctional Hadoop

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
There's no problem with Pig itself. If I use Pig CLI I can run some PigLatin scripts and they go all the way to the completion. The problem arises while running TestPigTest _programmatically_ e.g. using JUnit runner. Is there any secret debug options in this case? I've looked through the code but

Re: Pig 0.10 on YARN

2013-03-29 Thread Dmitriy Ryaboy
I missed the part where you are running it from JUnit. Still suspect a classpath issue :). Can you dump your classpath as seen by JUnit, and what happens when you run -secretDebugCmd and see if something is missing? -D On Fri, Mar 29, 2013 at 10:38 AM, Konstantin Boudnik c...@apache.org wrote:

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
I certainly agree with you. I have managed to narrow it down to the difference between pig-0.10.0-withouthadoop.jar and maven pig-0.10.0.jar. Basically, I can run: java -cp \ /etc/hadoop/conf:\ /etc/pig/conf:\ /root/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:\

Re: Pig 0.10 on YARN

2013-03-29 Thread Johnny Zhang
Hi, cos: I met the same problem before. Can you try below in your Bigtop pig maven execution .pom ? It will actually resolve other problem as well (for example, you may see Pig cannot get kerberos principle in secure YARN cluster) dependencies dependency

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
Hey Johnny. Yup, I've been using pretty much this without -hs artifact. However, adding hadoop-mapreduce-client-hs didn't help ;( Cos On Fri, Mar 29, 2013 at 03:25PM, Johnny Zhang wrote: Hi, cos: I met the same problem before. Can you try below in your Bigtop pig maven execution .pom ? It