It seems that Pig was correctly compiled against Hadoop 23, but the Cassandra piece was not, check out the where the exception is coming from:
> Caused by: java.lang.IncompatibleClassChangeError: Found interface > org.apache.hadoop.mapreduce.JobContext, but class was expected > at > org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:113) So, I would say that you also need to get Hadoop 2 compatible Cassandra connector first. Jarcec On Thu, Oct 24, 2013 at 10:34:49PM -0300, Claudio Romo Otto wrote: > After change from hadoop20 to hadoop23 the warning dissapeared but I > got the same exception (Found interface > org.apache.hadoop.mapreduce.JobContext, but class was expected) > > I have tried over a fresh install: hadoop 2.2.0 and pig 0.12.1 > compiled by me, no other product nor configuration, just two > servers, one master with ResourceManager and NameNode, one slave > with DataNode and NodeManager. > > I can't understand why over this fresh cluster Pig 0.12 fails. Here > is the new trace: > > 2013-10-24 16:10:52,351 [JobControl] ERROR > org.apache.pig.backend.hadoop23.PigJobControl - Error while trying > to run jobs. > java.lang.RuntimeException: java.lang.reflect.InvocationTargetException > at > org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:130) > at > org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191) > at java.lang.Thread.run(Thread.java:724) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:257) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128) > ... 3 more > Caused by: java.lang.IncompatibleClassChangeError: Found interface > org.apache.hadoop.mapreduce.JobContext, but class was expected > at > org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:113) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265) > at > org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335) > ... 8 more > > > El 24/10/13 21:33, Prashant Kommireddi escribió: > >Yes it does. You need to recompile Pig for hadoop 2 > > > >ant clean jar-withouthadoop -Dhadoopversion=23 > > > > > >On Thu, Oct 24, 2013 at 5:37 AM, Claudio Romo Otto < > >[email protected]> wrote: > > > >>Does Pig support Hadoop 2.2? When I try Pig 0.12 and Hadoop 2.2 I get an > >>error even with simple operations like > >> > >>data = LOAD 'cql://keyspace1/testcf?' USING CqlStorage(); > >>dump data; > >> > >>I only got a warning first and then and exception: > >> > >>2013-10-24 09:35:19,300 [main] WARN > >>org.apache.pig.backend.**hadoop20.PigJobControl > >>- falling back to default JobControl (not using hadoop 0.20 ?) > >>java.lang.**NoSuchFieldException: runnerState > >> at java.lang.Class.**getDeclaredField(Class.java:**1938) > >> at org.apache.pig.backend.**hadoop20.PigJobControl.<** > >>clinit>(PigJobControl.java:51) > >> at org.apache.pig.backend.hadoop.**executionengine.shims.** > >>HadoopShims.newJobControl(**HadoopShims.java:97) > >> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.** > >>JobControlCompiler.compile(**JobControlCompiler.java:285) > >> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.** > >>MapReduceLauncher.launchPig(**MapReduceLauncher.java:177) > >> at org.apache.pig.PigServer.**launchPlan(PigServer.java:**1264) > >> at org.apache.pig.PigServer.**executeCompiledLogicalPlan(** > >>PigServer.java:1249) > >> at org.apache.pig.PigServer.**storeEx(PigServer.java:931) > >> at org.apache.pig.PigServer.**store(PigServer.java:898) > >> at org.apache.pig.PigServer.**openIterator(PigServer.java:**811) > >> at org.apache.pig.tools.grunt.**GruntParser.processDump(** > >>GruntParser.java:696) > >> at org.apache.pig.tools.**pigscript.parser.**PigScriptParser.parse(** > >>PigScriptParser.java:320) > >> at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(** > >>GruntParser.java:194) > >> at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(** > >>GruntParser.java:170) > >> at org.apache.pig.tools.grunt.**Grunt.run(Grunt.java:69) > >> at org.apache.pig.Main.run(Main.**java:538) > >> at org.apache.pig.Main.main(Main.**java:157) > >> at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method) > >> at sun.reflect.**NativeMethodAccessorImpl.**invoke(** > >>NativeMethodAccessorImpl.java:**57) > >> at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(** > >>DelegatingMethodAccessorImpl.**java:43) > >> at java.lang.reflect.Method.**invoke(Method.java:606) > >> at org.apache.hadoop.util.RunJar.**main(RunJar.java:212) > >> > >>------------------------------**-------- > >> > >>Backend error message during job submission > >>------------------------------**------------- > >>Unexpected System Error Occured: java.lang.**IncompatibleClassChangeError: > >>Found interface org.apache.hadoop.mapreduce.**JobContext, but class was > >>expected > >> at org.apache.pig.backend.hadoop.**executionengine.** > >>mapReduceLayer.**PigOutputFormat.**setupUdfEnvAndStores(** > >>PigOutputFormat.java:225) > >> at org.apache.pig.backend.hadoop.**executionengine.** > >>mapReduceLayer.**PigOutputFormat.**checkOutputSpecs(** > >>PigOutputFormat.java:186) > >> at org.apache.hadoop.mapreduce.**JobSubmitter.checkSpecs(** > >>JobSubmitter.java:456) > >> at org.apache.hadoop.mapreduce.**JobSubmitter.**submitJobInternal( > >>**JobSubmitter.java:342) > >> at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1268) > >> at org.apache.hadoop.mapreduce.**Job$10.run(Job.java:1265) > >> at java.security.**AccessController.doPrivileged(**Native Method) > >> at javax.security.auth.Subject.**doAs(Subject.java:415) > >> at org.apache.hadoop.security.**UserGroupInformation.doAs(** > >>UserGroupInformation.java:**1491) > >> at org.apache.hadoop.mapreduce.**Job.submit(Job.java:1265) > >> at org.apache.hadoop.mapreduce.**lib.jobcontrol.ControlledJob.** > >>submit(ControlledJob.java:335) > >> at org.apache.hadoop.mapreduce.**lib.jobcontrol.JobControl.run(** > >>JobControl.java:240) > >> at org.apache.pig.backend.**hadoop20.PigJobControl.run(** > >>PigJobControl.java:121) > >> at java.lang.Thread.run(Thread.**java:724) > >> at org.apache.pig.backend.hadoop.**executionengine.** > >>mapReduceLayer.**MapReduceLauncher$1.run(**MapReduceLauncher.java:257) > >> > >>Pig Stack Trace > >>--------------- > >>ERROR 1066: Unable to open iterator for alias data > >> > >>org.apache.pig.impl.**logicalLayer.**FrontendException: ERROR 1066: > >>Unable to open iterator for alias data > >> at org.apache.pig.PigServer.**openIterator(PigServer.java:**836) > >> at org.apache.pig.tools.grunt.**GruntParser.processDump(** > >>GruntParser.java:696) > >> at org.apache.pig.tools.**pigscript.parser.** > >>PigScriptParser.parse(**PigScriptParser.java:320) > >> at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(** > >>GruntParser.java:194) > >> at org.apache.pig.tools.grunt.**GruntParser.parseStopOnError(** > >>GruntParser.java:170) > >> at org.apache.pig.tools.grunt.**Grunt.run(Grunt.java:69) > >> at org.apache.pig.Main.run(Main.**java:538) > >> at org.apache.pig.Main.main(Main.**java:157) > >> at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method) > >> at sun.reflect.**NativeMethodAccessorImpl.**invoke(** > >>NativeMethodAccessorImpl.java:**57) > >> at sun.reflect.**DelegatingMethodAccessorImpl.**invoke(** > >>DelegatingMethodAccessorImpl.**java:43) > >> at java.lang.reflect.Method.**invoke(Method.java:606) > >> at org.apache.hadoop.util.RunJar.**main(RunJar.java:212) > >>Caused by: java.io.IOException: Job terminated with anomalous status FAILED > >> at org.apache.pig.PigServer.**openIterator(PigServer.java:**828) > >> ... 12 more > >> > >> > >> >
signature.asc
Description: Digital signature
