Hi Daniel I was testing the pig local mode. Does pig need mini cluster when it runs in local mode?
Regards! Chen On Sun, May 25, 2014 at 1:40 PM, Daniel Dai <[email protected]> wrote: > Which Hadoop are you running against? Also 2.2? > > On Sun, May 25, 2014 at 6:52 AM, Chen He <[email protected]> wrote: > > Hi Daniel > > > > Thank you for the reply. I compiled Tez and Pig against hadoop 2.2. > > > > Regards! > > > > Chen > > > > > > On Thu, May 22, 2014 at 5:22 PM, Daniel Dai <[email protected]> > wrote: > > > >> Which version of Hadoop are you using? Seem LocalResource changed from > >> interface to class in 2.1 (YARN-711). If you are using pre-2.1, this > >> might be the case. > >> > >> Thanks, > >> Daniel > >> > >> On Thu, May 22, 2014 at 3:06 PM, Chen He <[email protected]> wrote: > >> > Hi Daniel > >> > > >> > Thank you for the reply. I did as you suggested but get following > errors: > >> > > >> > ERROR 2998: Unhandled internal error. Found interface > >> > org.apache.hadoop.yarn.api.records.LocalResource, but class was > expected > >> > > >> > java.lang.IncompatibleClassChangeError: Found interface > >> > org.apache.hadoop.yarn.api.records.LocalResource, but class was > expected > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezResourceManager.getTezResources(TezResourceManager.java:124) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezResourceManager.addTezResources(TezResourceManager.java:96) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezPlanContainer.getLocalResources(TezPlanContainer.java:104) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezJobControlCompiler.getJob(TezJobControlCompiler.java:106) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezJobControlCompiler.compile(TezJobControlCompiler.java:89) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:92) > >> > at > >> > > >> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:277) > >> > at org.apache.pig.PigServer.launchPlan(PigServer.java:1378) > >> > at > >> > > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1363) > >> > at org.apache.pig.PigServer.execute(PigServer.java:1352) > >> > at org.apache.pig.PigServer.executeBatch(PigServer.java:403) > >> > at org.apache.pig.PigServer.executeBatch(PigServer.java:386) > >> > at > >> > > org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170) > >> > at > >> > > >> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:232) > >> > at > >> > > >> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203) > >> > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > >> > at org.apache.pig.Main.run(Main.java:608) > >> > at org.apache.pig.Main.main(Main.java:156) > >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> > at > >> > > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > >> > at > >> > > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> > at java.lang.reflect.Method.invoke(Method.java:606) > >> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208) > >> > > >> > Is this a problem that caused by hadoop incompatiblilty? Which > version of > >> > Hadoop is the best? > >> > > >> > Regards! > >> > > >> > Chen > >> > > >> > > >> > On Thu, May 22, 2014 at 4:19 PM, Daniel Dai <[email protected]> > >> wrote: > >> > > >> >> Need to set PIG_CLASSPATH: > >> >> export PIG_CLASSPATH=$TEZ_HOME/*:$TEZ_HOME/lib/*:$TEZ_HOME/conf > >> >> > >> >> Thanks, > >> >> Daniel > >> >> > >> >> On Thu, May 22, 2014 at 2:11 PM, Chen He <[email protected]> wrote: > >> >> > Any one met this error before? > >> >> > > >> >> > [main] ERROR org.apache.pig.Main - ERROR 2998: Unhandled internal > >> error. > >> >> > org/apache/tez/common/counters/DAGCounter > >> >> > > >> >> > Any reply will be appreciated! > >> >> > > >> >> > Regards! > >> >> > > >> >> > Chen > >> >> > >> >> -- > >> >> CONFIDENTIALITY NOTICE > >> >> NOTICE: This message is intended for the use of the individual or > >> entity to > >> >> which it is addressed and may contain information that is > confidential, > >> >> privileged and exempt from disclosure under applicable law. If the > >> reader > >> >> of this message is not the intended recipient, you are hereby > notified > >> that > >> >> any printing, copying, dissemination, distribution, disclosure or > >> >> forwarding of this communication is strictly prohibited. If you have > >> >> received this communication in error, please contact the sender > >> immediately > >> >> and delete it from your system. Thank You. > >> >> > >> > >> -- > >> CONFIDENTIALITY NOTICE > >> NOTICE: This message is intended for the use of the individual or > entity to > >> which it is addressed and may contain information that is confidential, > >> privileged and exempt from disclosure under applicable law. If the > reader > >> of this message is not the intended recipient, you are hereby notified > that > >> any printing, copying, dissemination, distribution, disclosure or > >> forwarding of this communication is strictly prohibited. If you have > >> received this communication in error, please contact the sender > immediately > >> and delete it from your system. Thank You. > >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
