Did you build pig-withouthadoop.jar and did you make sure that it and your hadoop*.jar are on the classpath?
If so, then when you make a pig script, begin it with: register /path/to/pig/build/ivy/lib/Pig/antlr-runtime-3.2.jar; 2011/5/26 Rui Miguel Forte <ruimiguelfo...@gmail.com> > Hi Jonathan, > > Sorry or the late response, First of all thanks for your email. I'm > afraid I still havent managed to get this to work. I am not clear on > what you mean by register the antlr jar script though. How do I do > this? > > Thanks, > Miguel > > > > On 23 May 2011 20:35, Jonathan Coveney <jcove...@gmail.com> wrote: > > Here is what I had to do to get pig running with a different version of > > Hadoop (in my case, the cloudera build but I'd try this as well): > > > > build pig-withouthadoop.jar by running "ant jar-withouthadoop". Then, > when > > you run pig, put the pig-withouthadoop.jar on your classpath as well as > your > > hadoop jar. In my case, I found that scripts only worked if I > additionally > > manually registered the antlr jar: > > > > register /path/to/pig/build/ivy/lib/Pig/antlr-runtime-3.2.jar; > > > > 2011/5/23 Rui Miguel Forte <ruimiguelfo...@gmail.com> > > > >> Hello, > >> > >> I have been trying to set up pig 0.8.1 to work with hadoop 0.20.203 > >> without success. At the moment, if I run pig -x local I correctly get > >> access to the grunt shell but when I try to run the default mapreduce > >> option it fails with error 2999, see below: > >> > >> miguel@euler:~$ pig > >> 2011-05-23 19:35:09,843 [main] INFO org.apache.pig.Main - Logging > >> error messages to: /home/miguel/pig_1306168509838.log > >> 2011-05-23 19:35:10,030 [main] INFO > >> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - > >> Connecting to hadoop file system at: hdfs://localhost:54310 > >> 2011-05-23 19:35:10,168 [main] ERROR org.apache.pig.Main - ERROR 2999: > >> Unexpected internal error. Failed to create DataStorage > >> Details at logfile: /home/miguel/pig_1306168509838.log > >> > >> Hadoop itself runs normally on my system. Any ideas on what to do to > >> fix this? Is it an incompatibility issue or something I am doing > >> wrong? Below I am including my system's information, hopefully this > >> will be useful for a diagnosis. > >> > >> The log file has the following: > >> > >> Error before Pig is launched > >> ---------------------------- > >> ERROR 2999: Unexpected internal error. Failed to create DataStorage > >> > >> java.lang.RuntimeException: Failed to create DataStorage > >> at > >> > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75) > >> at > >> > org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214) > >> at > >> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134) > >> at org.apache.pig.impl.PigContext.connect(PigContext.java:183) > >> at org.apache.pig.PigServer.<init>(PigServer.java:226) > >> at org.apache.pig.PigServer.<init>(PigServer.java:215) > >> at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55) > >> at org.apache.pig.Main.run(Main.java:452) > >> at org.apache.pig.Main.main(Main.java:107) > >> Caused by: java.io.IOException: Call to localhost/127.0.0.1:54310 > >> failed on local exception: java.io.EOFException > >> at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) > >> at org.apache.hadoop.ipc.Client.call(Client.java:743) > >> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > >> at $Proxy0.getProtocolVersion(Unknown Source) > >> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) > >> at > >> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) > >> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207) > >> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170) > >> at > >> > org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82) > >> at > >> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378) > >> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) > >> at > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) > >> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) > >> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) > >> at > >> > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) > >> at > >> > org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) > >> ... 9 more > >> Caused by: java.io.EOFException > >> at java.io.DataInputStream.readInt(DataInputStream.java:375) > >> at > >> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) > >> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) > >> > >> > ================================================================================ > >> > >> My system details: > >> > >> Ubuntu 10.04 LTS > >> miguel@euler:~$ pig -version > >> Apache Pig version 0.8.1 (r1094835) > >> compiled Apr 18 2011, 19:26:53 > >> miguel@euler:~$ hadoop version > >> Hadoop 0.20.203.0 > >> Subversion > >> > http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 > >> -r 1099333 > >> Compiled by oom on Wed May 4 07:57:50 PDT 2011 > >> > >> My environment variables (I know I have more than I need but tutorials > >> on the web have been a little hazy on this): > >> > >> miguel@euler:~$ printenv | grep "PIG\|HADOOP" > >> PIG_DIR=/home/miguel/pig/pig-0.8.1 > >> PIG_HOME=/home/miguel/pig/pig-0.8.1 > >> HADOOP_HOME=/usr/local/hadoop > >> PIGDIR=/home/miguel/pig/pig-0.8.1 > >> PIG_INSTALL=/home/miguel/pig/pig-0.8.1 > >> PIG_HADOOP_VERSION=20 > >> HADOOP_INSTALL=/usr/local/hadoop > >> PIG_CLASSPATH=/usr/local/hadoop/conf > >> HADOOPDIR=/usr/local/hadoop/conf > >> > >> Many thanks for any help, > >> Rui Miguel Forte > >> > > >
