Ah, yes, that explains a lot. I heard this before, comparing Hadoop to the Linux model, but I didn't get what the impact was until now ;)
Thanks. On Sat, Jan 11, 2014 at 4:47 PM, Ruslan Al-Fakikh <metarus...@gmail.com>wrote: > Usually Hadoop is used within a distro. Those can be cloudera, hortonworks, > emr, etc > 11 янв. 2014 г. 2:05 пользователь "Mariano Kamp" <mariano.k...@gmail.com> > написал: > > > Hi Josh. > > > > Ok, got it. Interesting. > > > > Downloaded ant, recompiled and now it works. > > > > Thank you. > > > > > > On Fri, Jan 10, 2014 at 10:16 PM, Josh Elser <josh.el...@gmail.com> > wrote: > > > > > Mariano, > > > > > > Pig 0.12.0 does work with Hadoop-2.2.0, but you need to recompile Pig > > > first. > > > > > > In your $PIG_HOME, run the following command to rebuild Pig: > > > > > > `ant clean jar-withouthadoop -Dhadoopversion=23` > > > > > > Then, try re-running your script. > > > > > > > > > On 1/9/14, 5:13 PM, Mariano Kamp wrote: > > > > > >> Hi, > > >> > > >> I am trying to run the first pig sample from the Hadoop - The > definitive > > >> guide - book. > > >> > > >> Unfortunately that doesn't work for me. > > >> > > >> I downloaded 0.12.0 and got the impression it should work with Hadoop > > 2.2. > > >> > > >> http://pig.apache.org/releases.html#14+October%2C+ > > >>> 2013%3A+release+0.12.0+available > > >>> 14 October, 2013: release 0.12.0 available > > >>> This release include several new features such as ASSERT operator, > > >>> Streaming UDF, new AvroStorage, IN/CASE >operator, > > BigInteger/BigDecimal > > >>> data type, support for Windows. > > >>> Note > > >>> This release works with Hadoop 0.20.X, 1.X, 0.23.X and 2.X > > >>> > > >> > > >> I use Hadoop 2.x. > > >> > > >>> snow:bin mkamp$ which hadoop > > >>> /Users/mkamp/hadoop-2.2.0/bin//hadoop > > >>> > > >> > > >> snow:bin mkamp$ echo $HADOOP_HOME > > >>> /Users/mkamp/hadoop-2.2.0 > > >>> > > >> > > >> But no matter if HADOOP_HOME is set or not I get a couple of errors > and > > >> it doesn't work if I run the script: > > >> > > >> records = LOAD 'micro-tab/sample.txt' > > >>> AS (year:chararray, temperature:int, quality:int); > > >>> DUMP records; > > >>> > > >> > > >> All hell breaks lose and there is a lot of output, but most seems > > >> meaningless, warnings about settings that are deprecated in Hadoop, > but > > >> still delivered by default this way. > > >> > > >> Hard to say what is relevant. Here are some excerpts, full output > > >> attached as file. > > >> > > >> From the logfile: > > >> > > >> Unexpected System Error Occured: > > java.lang.IncompatibleClassChangeError: > > >>> Found interface >org.apache.hadoop.mapreduce.JobContext, but class > was > > >>> expected > > >>> at >org.apache.pig.backend.hadoop.executionengine.mapReduceLayer. > > >>> PigOutputFormat.setupUdfEnvAndStores(PigO>utputFormat.java:225) > > >>> > > >> > > >> > > >> ERROR 1066: Unable to open iterator for alias records > > >>> From the console: > > >>> > > >> > > >> 2014-01-09 22:24:45,976 [main] WARN > > org.apache.pig.backend.hadoop20.PigJobControl > > >>> - falling back to default >JobControl (not using hadoop 0.20 ?) > > >>> java.lang.NoSuchFieldException: runnerState > > >>> at java.lang.Class.getDeclaredField(Class.java:1918) > > >>> > > >> > > >> But as a little googling indicated, this is business as usual? > > >> > > >> 2014-01-09 22:24:49,228 [JobControl] ERROR > > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl > > >>> - Error >while trying to run jobs. > > >>> java.lang.IncompatibleClassChangeError: Found interface > > >>> org.apache.hadoop.mapreduce.JobContext, but class >was expected > > >>> > > >> > > >> Input(s): > > >>> Failed to read data from "hdfs://localhost/user/mkamp/ > > >>> micro-tab/sample.txt" > > >>> > > >> > > >> That last one looks interesting. Maybe I am using it wrong and the > > >> reported errors are not related? I wanted to read from the local file > > >> system. > > >> > > >> So I also changed the script to read from hdfs, but that didn't change > > >> the error. > > >> > > >> Any ideas where to go from here? > > >> > > >> Is it possible to run the latest Hadoop binary download and the latest > > >> Pig binary download together? > > >> > > >> > > >
