Your cluster includes a ResourceManager, JobHistoryManager etc (YARN) or is it on MR1 (JobTracker)?
I'm not familiar with CDH distro, but you need to make sure the client jars (hadoop, pig) and the server are all the same version. On Thu, Feb 13, 2014 at 11:02 AM, Russell Jurney <[email protected]>wrote: > The cluster is running 2.0.0-mr1-cdh4.4.0 which includes Hadoop 1 and > Hadoop 2. > > > On Wed, Feb 12, 2014 at 6:24 PM, Prashant Kommireddi < > [email protected]> wrote: > >> Is the cluster running hadoop 2? >> >> >> On Wed, Feb 12, 2014 at 5:47 PM, Russell Jurney <[email protected] >> > wrote: >> >>> Changing the jars I REGISTER to phoenix-2.1.2.jar (no client jar), I >>> still get the same error: >>> >>> 2014-02-12 17:45:52,262 [main] ERROR >>> org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Unable to recreate >>> exception from backed error: Error: Found interface >>> org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected >>> >>> 2014-02-12 17:45:52,262 [main] ERROR >>> org.apache.pig.tools.grunt.GruntParser - >>> org.apache.pig.backend.executionengine.ExecException: ERROR 2997: Unable to >>> recreate exception from backed error: Error: Found interface >>> org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected >>> >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:217) >>> >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:149) >>> >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:400) >>> >>> at org.apache.pig.PigServer.launchPlan(PigServer.java:1266) >>> >>> at >>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1251) >>> >>> at org.apache.pig.PigServer.execute(PigServer.java:1241) >>> >>> at org.apache.pig.PigServer.executeBatch(PigServer.java:335) >>> >>> at >>> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137) >>> >>> at >>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198) >>> >>> at >>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170) >>> >>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) >>> >>> at org.apache.pig.Main.run(Main.java:604) >>> >>> at org.apache.pig.Main.main(Main.java:157) >>> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> at java.lang.reflect.Method.invoke(Method.java:616) >>> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208) >>> >>> >>> On Wed, Feb 12, 2014 at 5:23 PM, James Taylor <[email protected]>wrote: >>> >>>> Phoenix will work with either Hadoop1 or Hadoop2. The >>>> phoenix-<version>-client.jar bundles the Hadoop1 jars, so if you want to >>>> use Hadoop2, don't use that jar. Instead you can use the >>>> phoenix-<version>.jar and include any other required jars on the classpath. >>>> On the client-side Phoenix depends on antlr and opencsv (if your doing bulk >>>> loading). >>>> >>>> Thanks, >>>> James >>>> >>>> >>>> On Wed, Feb 12, 2014 at 5:10 PM, Russell Jurney < >>>> [email protected]> wrote: >>>> >>>>> I am using CDH 4.4, with HBase hbase-0.94.6+132 and pig-0.11.0+33. My >>>>> Hadoop client lib is hadoop-2.0.0+1475. >>>>> >>>>> So it looks like my Pig is MR2, but Phoenix is expecting MR1? >>>>> >>>>> I'm not really sure how to go about resolving this issue. CDH is a bit >>>>> of a black box - I don't know if their Pig is using MR1/2. And I don't >>>>> have >>>>> source to recompile it. >>>>> >>>>> It looks like my Pig is using >>>>> >>>>> >>>>> On Tue, Feb 11, 2014 at 11:12 PM, Prashant Kommireddi < >>>>> [email protected]> wrote: >>>>> >>>>>> Yup, that seems like a classpath issue. Also, make sure to compile >>>>>> pig with the correct hadoop version if you are using the fat jar. >>>>>> >>>>>> >>>>>> On Tue, Feb 11, 2014 at 9:05 PM, Skanda >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi Russell, >>>>>>> >>>>>>> Which version of HBase and Hadoop are you using? The reason for this >>>>>>> issue is that TaskAttemptContext is an interface in Hadoop 2.x but is a >>>>>>> class in Hadoop 1.x. >>>>>>> >>>>>>> Regards, >>>>>>> Skanda >>>>>>> >>>>>>> >>>>>>> On Wed, Feb 12, 2014 at 10:06 AM, James Taylor < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> This is beyond my knowledge of Pig, but Prashant may know as he >>>>>>>> contributed our Pig integration. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> James >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Feb 11, 2014 at 4:34 PM, Russell Jurney < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> I am trying to store data into this table: >>>>>>>>> >>>>>>>>> CREATE TABLE IF NOT EXISTS BEACONING_ACTIVITY ( >>>>>>>>> >>>>>>>>> EVENT_TIME VARCHAR NOT NULL, >>>>>>>>> C_IP VARCHAR NOT NULL, >>>>>>>>> CS_HOST VARCHAR NOT NULL, >>>>>>>>> SLD VARCHAR NOT NULL, >>>>>>>>> CONFIDENCE DOUBLE NOT NULL, >>>>>>>>> RISK DOUBLE NOT NULL, >>>>>>>>> ANOMOLY DOUBLE NOT NULL, >>>>>>>>> INTERVAL DOUBLE NOT NULL >>>>>>>>> >>>>>>>>> CONSTRAINT PK PRIMARY KEY (EVENT_TIME, C_IP, CS_HOST) >>>>>>>>> ); >>>>>>>>> >>>>>>>>> >>>>>>>>> Using this Pig: >>>>>>>>> >>>>>>>>> hosts_and_risks = FOREACH hosts_and_anomaly GENERATE hour, c_ip, >>>>>>>>> cs_host, sld, confidence, (confidence * anomaly) AS risk:double, >>>>>>>>> anomaly, >>>>>>>>> interval; >>>>>>>>> --hosts_and_risks = ORDER hosts_and_risks BY risk DESC; >>>>>>>>> --STORE hosts_and_risks INTO '/tmp/beacons.txt'; >>>>>>>>> STORE hosts_and_risks into 'hbase://BEACONING_ACTIVITY' using >>>>>>>>> com.salesforce.phoenix.pig.PhoenixHBaseStorage('hiveapp1','-batchSize >>>>>>>>> 5000'); >>>>>>>>> >>>>>>>>> And the most helpful error message I get is this: >>>>>>>>> >>>>>>>>> 2014-02-11 16:24:13,831 FATAL org.apache.hadoop.mapred.Child: Error >>>>>>>>> running child : java.lang.IncompatibleClassChangeError: Found >>>>>>>>> interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class >>>>>>>>> was expected >>>>>>>>> at >>>>>>>>> com.salesforce.phoenix.pig.hadoop.PhoenixOutputFormat.getRecordWriter(PhoenixOutputFormat.java:75) >>>>>>>>> at >>>>>>>>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:597) >>>>>>>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:444) >>>>>>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:268) >>>>>>>>> at java.security.AccessController.doPrivileged(Native Method) >>>>>>>>> at javax.security.auth.Subject.doAs(Subject.java:415) >>>>>>>>> at >>>>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >>>>>>>>> at org.apache.hadoop.mapred.Child.main(Child.java:262) >>>>>>>>> >>>>>>>>> >>>>>>>>> What am I to do? >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Russell Jurney twitter.com/rjurney [email protected] >>>>>>>>> datasyndrome.com >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Russell Jurney twitter.com/rjurney [email protected] >>>>> datasyndrome.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Russell Jurney twitter.com/rjurney [email protected] datasyndrome >>> .com >>> >> >> > > > -- > Russell Jurney twitter.com/rjurney [email protected] datasyndrome. > com >
