I certainly agree with you. I have managed to narrow it down to the difference between pig-0.10.0-withouthadoop.jar and maven pig-0.10.0.jar.
Basically, I can run: java -cp \ /etc/hadoop/conf:\ /etc/pig/conf:\ /root/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:\ /root/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:\ /root/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:\ /root/.m2/repository/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar:\ /root/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:\ /usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:\ /root/.m2/repository/org/apache/avro/avro/1.5.3/avro-1.5.3.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-common/2.0.3.1-SNAPSHOT/hadoop-common-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.3.1-SNAPSHOT/hadoop-auth-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.3.1-SNAPSHOT/hadoop-hdfs-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.0.3.1-SNAPSHOT/hadoop-yarn-api-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.0.3.1-SNAPSHOT/hadoop-yarn-client-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.0.3.1-SNAPSHOT/hadoop-yarn-common-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-core-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-common-2.0.3.1-SNAPSHOT.jar:\ /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-jobclient-2.0.3.1-SNAPSHOT.jar:\ /usr/lib/pig/pig-0.10.0-withouthadoop.jar:\ /root/.m2/repository/com/google/guava/guava/11.0/guava-11.0.jar:\ /root/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:\ /root/.m2/repository/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar:\ /root/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:\ /root/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:\ /root/.m2/repository/org/antlr/stringtemplate/3.2.1/stringtemplate-3.2.1.jar:\ /usr/lib/pig/lib/automaton.jar:\ build/ivy/lib/Pig/junit-4.5.jar:\ build/pig-0.10.0-SNAPSHOT-smoketests.jar:\ /root/.m2/repository/org/apache/pig/pigunit/0.10.0-SNAPSHOT/pigunit-0.10.0-SNAPSHOT.jar \ -Dpigunit.exectype.cluster=true \ org.junit.runner.JUnitCore org.apache.pig.test.pigunit.TestPigTest But once /usr/lib/pig/pig-0.10.0-withouthadoop.jar is replaced with /root/.m2/repository/org/apache/pig/pig/0.10.0/pig-0.10.0.jar all hell breaks loose and submission doesn't happen anymore ;( And as before - no error messages nor exceptions appear anywhere on the surface. Cos On Fri, Mar 29, 2013 at 11:52AM, Dmitriy Ryaboy wrote: > I missed the part where you are running it from JUnit. Still suspect a > classpath issue :). Can you dump your classpath as seen by JUnit, and what > happens when you run "-secretDebugCmd" and see if something is missing? > > -D > > > On Fri, Mar 29, 2013 at 10:38 AM, Konstantin Boudnik <[email protected]> wrote: > > > There's no problem with Pig itself. If I use Pig CLI I can run some > > PigLatin > > scripts and they go all the way to the completion. > > > > The problem arises while running TestPigTest _programmatically_ e.g. using > > JUnit runner. Is there any secret debug options in this case? I've looked > > through the code but didn't spot anything immediately. > > > > I will give "-secretDebugCmd" a try anyway - it might give me some ideas... > > > > Appreciate the help! > > Cos > > > > On Fri, Mar 29, 2013 at 09:08AM, Dmitriy Ryaboy wrote: > > > Can you run `pig -secretDebugCmd` and look at your paths? I suspect > > there's > > > something going on there, like not having the right configs or hadoop > > jars. > > > > > > > > > On Thu, Mar 28, 2013 at 11:32 PM, Konstantin Boudnik <[email protected]> > > wrote: > > > > > > > Guys, > > > > > > > > I am trying to run TestPigTest against a fanctional Hadoop 2.0 YARN > > > > cluster - > > > > I can run MR jobs using Hive or plain MR examples. However, it is going > > > > somewhat unsuccessful. While using pig "far" jar file - e.g. with > > > > everything > > > > repacked inside of it - I seeminly can run TestPigTest against a real > > > > cluster. > > > > > > > > However, while using pig-withouthadoop.jar and adding all needed > > > > dependencies > > > > to the classpath, Pig is failing to communicate with YARN nor submit > > any MR > > > > jobs. There's no exceptions, not error messages anywhere in the logs. > > Have > > > > anyone seen anything similar? > > > > > > > > Any input is highly appreciated! > > > > > > > > -- > > > > With regards, > > > > Cos > > > > > > > > > >
