Hello,
I am trying to use hadoop with C++ while supplying myself a Reader and a
Writer. I compiled and ran wordcount-nopipe, however, it seems like when I
try to run it, the filename that the reader tries to open is "" (empty
string) - I print the filename it tries to open in the Reader's constructor.
I get the following error when running the program:
# bin/hadoop pipes -conf /tmp/words.xml -input words.txt -output outdir2
07/11/19 11:12:22 WARN mapred.JobClient: No job jar file set. User classes
may not be found. See JobConf(Class) or JobConf#setJar(String).
07/11/19 11:12:22 INFO mapred.FileInputFormat: Total input paths to process
: 1
07/11/19 11:12:22 INFO mapred.JobClient: Running job: job_200711161538_3778
07/11/19 11:12:23 INFO mapred.JobClient: map 0% reduce 0%
07/11/19 11:12:28 INFO mapred.JobClient: Task Id :
task_200711161538_3778_m_000003_0, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:134)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:83)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1760)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:243)
at
org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
at
org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
at
org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:112)
task_200711161538_3778_m_000003_0: FILENAME: <---- I print this to see
what is the file name accepted.
task_200711161538_3778_m_000003_0: Hadoop Pipes Exception: RecordReader
defined when not needed. at impl/HadoopPipes.cc:627 in virtual void
HadoopPipes::TaskContextImpl::runMap(std::string, int, bool)
07/11/19 11:12:28 INFO mapred.JobClient: Task Id :
task_200711161538_3778_m_000002_0, Status : FAILED
java.io.IOException: pipe child exception
at
org.apache.hadoop.mapred.pipes.Application.abort(Application.java:134)
at
org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:83)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:192)
at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1760)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:243)
at
org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:313)
at
org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:335)
at
org.apache.hadoop.mapred.pipes.BinaryProtocol$UplinkReaderThread.run(BinaryProtocol.java:112)
Also, can I use runTask several times in a C++ application, each time with a
different mapper and reducer, or can I run runTask at most once in a pipes
application?
Thanks for any help.
Jerr.
--
View this message in context:
http://www.nabble.com/pipe-application-error-tf4837436.html#a13839363
Sent from the Hadoop Dev mailing list archive at Nabble.com.
