I had a simular problem, you need to add the path to the rootClasspath, not just the class path.
On Fri, May 13, 2011 at 8:49 PM, Daniel Dai <jiany...@yahoo-inc.com> wrote: > Sounds like a hadoop job setup exception. Go to job tracker UI, you may > have chance to locate the job and check what happen in job setup. > > Daniel > > > On 05/11/2011 05:45 PM, Jianting Cao wrote: > >> I'm trying to embed pig into java program. I tried two approaches, none of >> them works. >> >> Approach 1: >> >> I followed http://pig.apache.org/docs/r0.6.0/setup.html#Embedded+Programs >> and then ran into the error: cannot find the hadoop-core.xml nor >> site-core.xml. Then I find the solution at >> http://mail-archives.apache.org/mod_mbox/pig-user/201104.mbox/%3CBANLkTinx >> G7VwS8hDjaMbR=8_2w7aj3_...@mail.gmail.com%3E, which is just add a line of >> System.setProperty("sun.boot.class.path", "/usr/local/hadoop/conf"); to >> tell the program the location of hadoop configuration files. Then I had >> another error: >> >> 11/05/11 17:18:32 INFO executionengine.HExecutionEngine: Connecting to >> hadoop file system at: hdfs://jianting-virtual-machine1:54310 >> 11/05/11 17:18:32 INFO executionengine.HExecutionEngine: Connecting to >> map-reduce job tracker at: jianting-virtual-machine1:54311 >> 11/05/11 17:18:32 INFO pigstats.ScriptState: Pig features used in the >> script: UNKNOWN >> 11/05/11 17:18:32 INFO executionengine.HExecutionEngine: >> pig.usenewlogicalplan is set to true. New logical plan will be used. >> 11/05/11 17:18:33 INFO executionengine.HExecutionEngine: (Name: B: >> Store(idout:org.apache.pig.builtin.PigStorage) - scope-4 Operator Key: >> scope-4) >> 11/05/11 17:18:33 INFO mapReduceLayer.MRCompiler: File concatenation >> threshold: 100 optimistic? false >> 11/05/11 17:18:33 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size >> before optimization: 1 >> 11/05/11 17:18:33 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size >> after optimization: 1 >> 11/05/11 17:18:33 INFO pigstats.ScriptState: Pig script settings are added >> to the job >> 11/05/11 17:18:33 INFO mapReduceLayer.JobControlCompiler: >> mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 >> 11/05/11 17:18:35 INFO mapReduceLayer.JobControlCompiler: Setting up >> single store job >> 11/05/11 17:18:35 INFO mapReduceLayer.MapReduceLauncher: 1 map-reduce >> job(s) waiting for submission. >> 11/05/11 17:18:35 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 11/05/11 17:18:35 INFO input.FileInputFormat: Total input paths to process >> : 1 >> 11/05/11 17:18:35 INFO util.MapRedUtil: Total input paths to process : 1 >> 11/05/11 17:18:35 INFO util.MapRedUtil: Total input paths (combined) to >> process : 1 >> 11/05/11 17:18:35 INFO mapReduceLayer.MapReduceLauncher: 0% complete >> org.apache.pig.PigException: ERROR 1002: Unable to store alias B >> >> This is where I am stuck. >> >> >> >> Then I tried a second approach. >> >> Approach 2: >> >> The code and everything is the same as in the tutorial of approach 1. The >> only difference is that instead of using vim, javac and java to edit, >> compile and run the example, I uses Netbeans. I added pig.jar file that is >> generated by running "ant" in pig folder. This time the error is: >> >> Have no file for >> /usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/modules/jdk.boot.jar >> 11/05/11 17:37:53 INFO executionengine.HExecutionEngine: Connecting to >> hadoop file system at: hdfs://jianting-virtual-machine1:54310 >> 11/05/11 17:37:53 INFO executionengine.HExecutionEngine: Connecting to >> map-reduce job tracker at: jianting-virtual-machine1:54311 >> 11/05/11 17:37:53 INFO pigstats.ScriptState: Pig features used in the >> script: UNKNOWN >> 11/05/11 17:37:53 INFO executionengine.HExecutionEngine: >> pig.usenewlogicalplan is set to true. New logical plan will be used. >> 11/05/11 17:37:54 INFO executionengine.HExecutionEngine: (Name: B: >> Store(idout:org.apache.pig.builtin.PigStorage) - scope-4 Operator Key: >> scope-4) >> 11/05/11 17:37:54 INFO mapReduceLayer.MRCompiler: File concatenation >> threshold: 100 optimistic? false >> 11/05/11 17:37:54 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size >> before optimization: 1 >> 11/05/11 17:37:54 INFO mapReduceLayer.MultiQueryOptimizer: MR plan size >> after optimization: 1 >> 11/05/11 17:37:54 INFO pigstats.ScriptState: Pig script settings are added >> to the job >> 11/05/11 17:37:54 INFO mapReduceLayer.JobControlCompiler: >> mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 >> 11/05/11 17:37:56 INFO mapReduceLayer.JobControlCompiler: Setting up >> single store job >> 11/05/11 17:37:57 INFO mapReduceLayer.MapReduceLauncher: 1 map-reduce >> job(s) waiting for submission. >> 11/05/11 17:37:57 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 11/05/11 17:37:57 INFO input.FileInputFormat: Total input paths to process >> : 1 >> 11/05/11 17:37:57 INFO util.MapRedUtil: Total input paths to process : 1 >> 11/05/11 17:37:57 INFO mapReduceLayer.MapReduceLauncher: 0% complete >> 11/05/11 17:37:57 INFO util.MapRedUtil: Total input paths (combined) to >> process : 1 >> 11/05/11 17:37:57 INFO mapReduceLayer.MapReduceLauncher: 100% complete >> 11/05/11 17:37:57 ERROR pigstats.PigStatsUtil: 0 map reduce job(s) failed! >> 11/05/11 17:37:57 INFO pigstats.PigStats: Script Statistics: >> >> HadoopVersion PigVersion UserId StartedAt FinishedAt >> Features >> 0.20.2 0.8.1-SNAPSHOT hadoop 2011-05-11 17:37:54 2011-05-11 >> 17:37:57 UNKNOWN >> >> Failed! >> >> Failed Jobs: >> JobId Alias Feature Message Outputs >> >> Input(s): >> >> Output(s): >> >> Counters: >> Total records written : 0 >> Total bytes written : 0 >> Spillable Memory Manager spill count : 0 >> Total bags proactively spilled: 0 >> Total records proactively spilled: 0 >> >> Job DAG: >> null >> >> 11/05/11 17:37:57 INFO mapReduceLayer.MapReduceLauncher: Success! >> java.io.IOException: Couldn't retrieve job. >> >> I'm stuck at here too. >> >> If I run the example line by line in grunt script mode, the program >> completes successfully without any error/warning. My guess is the >> warning:WARN mapred.JobClient: Use GenericOptionsParser for parsing the >> arguments. Applications should implement Tool for the same, which appears >> in both approaches and does not appear in grunt script mode. >> >> I'm using Pig 0.8.1, hadoop 0.20.2, java 1.6, Ubuntu 10.10. >> >> I searched the mailing list but without any luck. Any help is appreciated. >> >> Jianting Cao >> >> >> >> >
