Hi All,
I am trying to use the hadoop straeming with "wget" to simulate a
distributed downloader.
The command line i use is
./bin/hadoop jar -D mapred.reduce.tasks=0
contrib/streaming/hadoop-0.19.0-streaming.jar -input urli -output urlo
-mapper /usr/bin/wget -outputformat
org.apache.hadoop.mapred.lib.MultipleTextOutputFormat
But it thrown an exception
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess
failed with code 1
at
org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:295)
at
org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:519)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:136)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child.main(Child.java:155)
can somebody point me a way of why this happend. thanks.
--
http://daily.appspot.com/food/