I'm sorry, that sounded rude. I'm just trying to get an "out of the box" solution. So I either need a newer hadoop compiled for Windows, or I need to know which version of Pig was built against Hadoop 2.3.
Recompiling to me smells like Pig is making use of conditional compilation rather than dynamically loading the compatibility jar it needs. I did some more googling, and discovered this was more likely an issue of Pig trying to run against H1 when it is connected to H2. There's separate directories for h1/h2 jars in the lib folder, is it using the wrong one even though it listed the installed Hadoop version on output? Are there extra steps? If it's not compatible with a version why can't it error out with a clear message saying this version of Pig can't work with that version of Hadoop, please recompile. For the record, I am using an already compiled version of Pig. I'm trying not to set up the 3+ different build tools I seem to have come across just in the Hadoop world. For me, it would be easier to just have a known working configuration so I can just use it. On Wed, Jan 7, 2015 at 6:45 AM, Berin Loritsch <[email protected]> wrote: > Better yet, tell me where I can get the right Hadoop version precompiled > for Windows. I'm in a .net shop, and my goal is to set up a test > infrastructure, not a Java development stack. > > Pig wasn't working with the stock Hadoop 2.6 on windows, so I had to > downgrade to get this far. > > > On Wednesday, January 7, 2015, Lorand Bendig <[email protected]> wrote: > >> Pig 0.14 uses Hadoop 2.4.0 by default, but you have Hadoop 2.3.0 . >> You may change the Hadoop version in $PIG_HOME/ivy/libraries.properties >> to: >> hadoop-common.version=2.3.0 >> hadoop-hdfs.version=2.3.0 >> hadoop-mapreduce.version=2.3.0 >> >> Then try to recompile Pig: >> ant clean jar -Dhadoopversion=23 >> >> --Lorand >> >> On 07/01/15 03:41, Berin Loritsch wrote: >> >>> I'm trying to run a Pig Latin script on 0.14.0, and I've been having some >>> configuration issues. I'm assuming this is part of that. I have Hadoop >>> 2.3.0 on windows running as a single node. When I run my PIG script, I >>> get >>> this exception: >>> >>> Backend error message during job submission >>> ------------------------------------------- >>> Unexpected System Error Occured: java.lang.IncompatibleClassChangeError: >>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >>> expected >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer. >>> PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:235) >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer. >>> PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:183) >>> at >>> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs( >>> JobSubmitter.java:458) >>> at >>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal( >>> JobSubmitter.java:343) >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) >>> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:422) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs( >>> UserGroupInformation.java:1548) >>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) >>> at >>> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob. >>> submit(ControlledJob.java:335) >>> at >>> org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run( >>> JobControl.java:240) >>> at org.apache.pig.backend.hadoop20.PigJobControl.run( >>> PigJobControl.java:121) >>> at java.lang.Thread.run(Thread.java:745) >>> at >>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer. >>> MapReduceLauncher$1.run(MapReduceLauncher.java:276) >>> >>> Pig Stack Trace >>> --------------- >>> ERROR 2244: Job failed, hadoop does not return any error message >>> >>> org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job >>> failed, hadoop does not return any error message >>> at org.apache.pig.tools.grunt.GruntParser.executeBatch( >>> GruntParser.java:179) >>> at >>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError( >>> GruntParser.java:234) >>> at >>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError( >>> GruntParser.java:205) >>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) >>> at org.apache.pig.Main.run(Main.java:624) >>> at org.apache.pig.Main.main(Main.java:170) >>> >>> >>
