Hi, Not sure what is going on here. Trying to get simple pig script to work when put on hdfs://
I am able to "hadoop fs -cat" the file but get an error when trying to read it in pig? Any ideas of what is causing this and how to resolve? I am able to get this working on local file system (see below). Thanks, John Details: # Data file hadoop fs -cat "hdfs://thadoop2/tmp/v.log" 12/07/13 10:23:23 INFO util.NativeCodeLoader: Loaded the native-hadoop library 1 6 2 8 3 10 4 12 # Pig file cat hdfs_dump.pig A = LOAD 'hdfs://thadoop2/tmp/v.log' AS (x:float,y:float) ; dump A; # Pig shell script cat hdfs_dump.sh pig hdfs_dump.pig 2>&1 | tee $0.out # Run pig shell script ./hdfs_dump.sh # Errors in log output grep -i err hdfs_dump.sh.out 2012-07-13 10:20:51,724 [main] INFO org.apache.pig.Main - Logging error messages to: /apps/dba/hadoop_benchmarks/jhm/pig_1342189251720.log 2012-07-13 10:21:11,663 [main] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backed error: Error initializing attempt_201207051157_0016_m_000000_3: 2012-07-13 10:21:11,663 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! 2012-07-13 10:21:11,680 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Unable to recreate exception from backed error: Error initializing attempt_201207051157_0016_m_000000_3: # Local version works # Local data file cat v.log 1 6 2 8 3 10 4 12 # Local pig script cat local_dump.sh pig -x local dump.pig # Local shell script cat local_dump.sh pig -x local dump.pig # Run local shell script ./local_dump.sh 2>&1 | tail job_local_0001 2012-07-13 10:59:58,304 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success! 2012-07-13 10:59:58,308 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 2012-07-13 10:59:58,308 [main] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 (1.0,6.0) (2.0,8.0) (3.0,10.0) (4.0,12.0)
