You should specify A.jar on the bin/hadoop command line with -file A.jar,
so that streaming knows to copy that file on the tasktracker node.
- milind
On 11/11/08 10:50 AM, Amit_Gupta [EMAIL PROTECTED] wrote:
Hi
I have a jar file which takes input from stdin and writes something on
stdout. i.e. When I run
java -jar A.jar input
It prints the required output.
However, when I run it as a mapper in hadoop streaming using the command
$HADOOP_HOME/bin/hadoop jar streaming.jar -input .. -output ... -mapper
'java -jar A.jar' -reducer NONE
i get the broken pipe exception.
the error message is
additionalConfSpec_:null
null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming
packageJobJar:
[/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/dir/hadoop-hadoop/hadoop-unjar45410/]
[] /tmp/streamjob45411.jar tmpDir=null
08/11/11 23:20:14 INFO mapred.FileInputFormat: Total input paths to process
: 1
08/11/11 23:20:14 INFO streaming.StreamJob: getLocalDirs():
[/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/mapred]
08/11/11 23:20:14 INFO streaming.StreamJob: Running job:
job_20081724_0014
08/11/11 23:20:14 INFO streaming.StreamJob: To kill this job, run:
08/11/11 23:20:14 INFO streaming.StreamJob:
/mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job
-Dmapred.job.tracker=10.105.41.25:54311 -kill job_20081724_0014
08/11/11 23:20:15 INFO streaming.StreamJob: Tracking URL:
http://sayali:50030/jobdetails.jsp?jobid=job_20081724_0014
08/11/11 23:20:16 INFO streaming.StreamJob: map 0% reduce 0%
08/11/11 23:21:00 INFO streaming.StreamJob: map 100% reduce 100%
08/11/11 23:21:00 INFO streaming.StreamJob: To kill this job, run:
08/11/11 23:21:00 INFO streaming.StreamJob:
/mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job
-Dmapred.job.tracker=10.105.41.25:54311 -kill job_20081724_0014
08/11/11 23:21:00 INFO streaming.StreamJob: Tracking URL:
http://sayali:50030/jobdetails.jsp?jobid=job_20081724_0014
08/11/11 23:21:00 ERROR streaming.StreamJob: Job not Successful!
08/11/11 23:21:00 INFO streaming.StreamJob: killJob...
Streaming Job Failed!
Could some one please help me with any ideas or pointers.
regards
Amit
--
View this message in context:
http://www.nabble.com/Hadoop-Streamingrunning-a-jar-file-tp20445877p204458
77.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
--
Milind Bhandarkar
Y!IM: GridSolutions
408-349-2136
([EMAIL PROTECTED])