Hi I have a jar file which takes input from stdin and writes something on stdout. i.e. When I run
java -jar A.jar < input It prints the required output. However, when I run it as a mapper in hadoop streaming using the command $HADOOP_HOME/bin/hadoop jar ....streaming.jar -input .. -output ... -mapper 'java -jar A.jar' -reducer NONE i get the broken pipe exception. the error message is additionalConfSpec_:null null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming packageJobJar: [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/dir/hadoop-hadoop/hadoop-unjar45410/] [] /tmp/streamjob45411.jar tmpDir=null 08/11/11 23:20:14 INFO mapred.FileInputFormat: Total input paths to process : 1 08/11/11 23:20:14 INFO streaming.StreamJob: getLocalDirs(): [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/mapred] 08/11/11 23:20:14 INFO streaming.StreamJob: Running job: job_200811111724_0014 08/11/11 23:20:14 INFO streaming.StreamJob: To kill this job, run: 08/11/11 23:20:14 INFO streaming.StreamJob: /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014 08/11/11 23:20:15 INFO streaming.StreamJob: Tracking URL: http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014 08/11/11 23:20:16 INFO streaming.StreamJob: map 0% reduce 0% 08/11/11 23:21:00 INFO streaming.StreamJob: map 100% reduce 100% 08/11/11 23:21:00 INFO streaming.StreamJob: To kill this job, run: 08/11/11 23:21:00 INFO streaming.StreamJob: /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014 08/11/11 23:21:00 INFO streaming.StreamJob: Tracking URL: http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014 08/11/11 23:21:00 ERROR streaming.StreamJob: Job not Successful! 08/11/11 23:21:00 INFO streaming.StreamJob: killJob... Streaming Job Failed! Could some one please help me with any ideas or pointers. regards Amit -- View this message in context: http://www.nabble.com/Hadoop-Streaming----running-a-jar-file-tp20445877p20445877.html Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
