I've downloaded pig and am trying to get it to do the simplest things,
following the Apache Getting Started instructions:
The download and installation seemed to go fine. Pig -help, from
the newly created pig account, produced a list of helpful commands.
Then the problems began:
pig -x local vomits out:
2013-10-27 09:02:27,759 [main] INFO org.apache.pig.Main - Apache Pig
version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-10-27 09:02:27,760 [main] INFO org.apache.pig.Main - Logging error
messages to: /science/hadoop/pig/pig_1382878947757.log
2013-10-27 09:02:27,794 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /science/hadoop/pig/.pigbootup not found
2013-10-27 09:02:28,152 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: file:///
2013-10-27 09:02:28,157 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation -
mapred.used.genericoptionsparser is deprecated. Instead, use
mapreduce.client.genericoptionsparser.used
2013-10-27 09:02:28,161 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
deprecated. Instead, use fs.defaultFS
2013-10-27 09:02:28,161 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - mapred.job.tracker is
deprecated. Instead, use mapreduce.jobtracker.address
2013-10-27 09:02:28,821 [main] WARN
org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2013-10-27 09:02:28,964 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - io.bytes.per.checksum
is deprecated. Instead, use dfs.bytes-per-checksum
2013-10-27 09:02:28,968 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
deprecated. Instead, use fs.defaultFS
grunt>
Well, it seems to be running, so lets try it. As per the instructions,
I copied /etc/passwd to the pwd and then:
grunt> A = load 'passwd' using PigStorage(':');
2013-10-27 09:04:37,525 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
job.end.retry.interval is deprecated. Instead, use
mapreduce.job.end-noti ication.retry.interval
2013-10-27 09:04:37,525 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
mapred.job.tracker.retiredjobs.cache.size is deprecated. Instead, use
map educe.jobtracker.retiredjobs.cache.size
2013-10-27 09:04:37,525 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
mapred.task.profile.reduces is deprecated. Instead, use
mapreduce.task.pr file.reduces
2013-10-27 09:04:37,525 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use
mapreduce.job. vm.numtasks
...<About 50 lines of SNIP!>
2013-10-27 09:04:37,551 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion - hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
2013-10-27 09:04:37,552 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
mapreduce.job.counters.limit is deprecated. Instead, use
mapreduce.job.co nters.max
2013-10-27 09:04:37,552 [main] INFO
org.apache.hadoop.conf.Configuration.deprec tion -
mapred.job.reduce.input.buffer.percent is deprecated. Instead, use
mapred ce.reduce.input.buffer.percent
Not what I had expected.
If you've plodded along this far, and would like to take a stab at what
I'm doing wrong, TIA!
You'll need my environment:
Slackware linux 13.0.
uname -a gives:
Linux nebula 2.6.33.4 #3 SMP Wed May 12 23:13:09 CDT 2010 x86_64
Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz GenuineIntel GNU/Linux
$PATH:
/usr/local/bin/
/usr/bin
/bin
/science/hadoop/pig/bin
/science/hadoop/pig/pig-0.11.1/bin
/science/hadoop/bin
/usr/local/lib64/jdk1.7.0_25
/usr/local/lib64/jdk1.7.0_25/bin
/usr/local/lib64/jdk1.7.0_25/visualvm_134/bin
/usr/local/apache-maven/bin
/work/glxy/bin
/work/python
/usr/local/lib64/apache-ant-1.9.2/bin
$CLASSPATH:
/usr/local/lib64/jdk1.7.0_25
/usr/local/lib64/jdk1.7.0_25/bin/
/usr/local/lib64/jdk1.7.0_25/lib/junit-4.11.jar
/usr/local/lib64/apache-ant-1.9.2/bin
/science/hadoop/pig/pig-0.11.1/pig-0.11.1.jar
/science/hadoop/pig/classes
I've tried pig-0.12.0 as well, with similar results.
Thanks,
Tom
