You should specify A.jar on the bin/hadoop command line with "-file A.jar", so that streaming knows to copy that file on the tasktracker node.
- milind On 11/11/08 10:50 AM, "Amit_Gupta" <[EMAIL PROTECTED]> wrote: > > > Hi > > I have a jar file which takes input from stdin and writes something on > stdout. i.e. When I run > > java -jar A.jar < input > > It prints the required output. > > However, when I run it as a mapper in hadoop streaming using the command > > $HADOOP_HOME/bin/hadoop jar ....streaming.jar -input .. -output ... -mapper > 'java -jar A.jar' -reducer NONE > > i get the broken pipe exception. > > > the error message is > > additionalConfSpec_:null > null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming > packageJobJar: > [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/dir/hadoop-hadoop/hadoop-unjar45410/] > [] /tmp/streamjob45411.jar tmpDir=null > 08/11/11 23:20:14 INFO mapred.FileInputFormat: Total input paths to process > : 1 > 08/11/11 23:20:14 INFO streaming.StreamJob: getLocalDirs(): > [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/mapred] > 08/11/11 23:20:14 INFO streaming.StreamJob: Running job: > job_200811111724_0014 > 08/11/11 23:20:14 INFO streaming.StreamJob: To kill this job, run: > 08/11/11 23:20:14 INFO streaming.StreamJob: > /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job > -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014 > 08/11/11 23:20:15 INFO streaming.StreamJob: Tracking URL: > http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014 > 08/11/11 23:20:16 INFO streaming.StreamJob: map 0% reduce 0% > 08/11/11 23:21:00 INFO streaming.StreamJob: map 100% reduce 100% > 08/11/11 23:21:00 INFO streaming.StreamJob: To kill this job, run: > 08/11/11 23:21:00 INFO streaming.StreamJob: > /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job > -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014 > 08/11/11 23:21:00 INFO streaming.StreamJob: Tracking URL: > http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014 > 08/11/11 23:21:00 ERROR streaming.StreamJob: Job not Successful! > 08/11/11 23:21:00 INFO streaming.StreamJob: killJob... > Streaming Job Failed! > > Could some one please help me with any ideas or pointers. > > regards > Amit > > > -- > View this message in context: > http://www.nabble.com/Hadoop-Streaming----running-a-jar-file-tp20445877p204458 > 77.html > Sent from the Hadoop lucene-users mailing list archive at Nabble.com. > -- Milind Bhandarkar Y!IM: GridSolutions 408-349-2136 ([EMAIL PROTECTED])
