Hello,
I have been trying to set up pig 0.8.1 to work with hadoop 0.20.203
without success. At the moment, if I run pig -x local I correctly get
access to the grunt shell but when I try to run the default mapreduce
option it fails with error 2999, see below:
miguel@euler:~$ pig
2011-05-23 19:35:09,843 [main] INFO org.apache.pig.Main - Logging
error messages to: /home/miguel/pig_1306168509838.log
2011-05-23 19:35:10,030 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://localhost:54310
2011-05-23 19:35:10,168 [main] ERROR org.apache.pig.Main - ERROR 2999:
Unexpected internal error. Failed to create DataStorage
Details at logfile: /home/miguel/pig_1306168509838.log
Hadoop itself runs normally on my system. Any ideas on what to do to
fix this? Is it an incompatibility issue or something I am doing
wrong? Below I am including my system's information, hopefully this
will be useful for a diagnosis.
The log file has the following:
Error before Pig is launched
----------------------------
ERROR 2999: Unexpected internal error. Failed to create DataStorage
java.lang.RuntimeException: Failed to create DataStorage
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
at org.apache.pig.PigServer.<init>(PigServer.java:226)
at org.apache.pig.PigServer.<init>(PigServer.java:215)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
at org.apache.pig.Main.run(Main.java:452)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Call to localhost/127.0.0.1:54310
failed on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
... 9 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
================================================================================
My system details:
Ubuntu 10.04 LTS
miguel@euler:~$ pig -version
Apache Pig version 0.8.1 (r1094835)
compiled Apr 18 2011, 19:26:53
miguel@euler:~$ hadoop version
Hadoop 0.20.203.0
Subversion
http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203
-r 1099333
Compiled by oom on Wed May 4 07:57:50 PDT 2011
My environment variables (I know I have more than I need but tutorials
on the web have been a little hazy on this):
miguel@euler:~$ printenv | grep "PIG\|HADOOP"
PIG_DIR=/home/miguel/pig/pig-0.8.1
PIG_HOME=/home/miguel/pig/pig-0.8.1
HADOOP_HOME=/usr/local/hadoop
PIGDIR=/home/miguel/pig/pig-0.8.1
PIG_INSTALL=/home/miguel/pig/pig-0.8.1
PIG_HADOOP_VERSION=20
HADOOP_INSTALL=/usr/local/hadoop
PIG_CLASSPATH=/usr/local/hadoop/conf
HADOOPDIR=/usr/local/hadoop/conf
Many thanks for any help,
Rui Miguel Forte
