Hi,
We are facing a weird error in a multi-stage mapreduce job (> 10 jobs).
A few stages run fine and then randomly 1 of the job would fail with the
following error:
2015-05-07 07:56:01,467 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.IllegalArgumentException: Can not create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
at org.apache.hadoop.fs.Path.<init>(Path.java:90)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.getPath(HiveInputFormat.java:110)
at org.apache.hadoop.mapred.MapTask.updateJobWithSplit(MapTask.java:451)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
2015-05-07 07:56:01,469 INFO org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
I checked the mapred.input.dir and it's reading from the intermediate
output of the previous job. It's a valid HDFS path, so not sure why
the child finds an empty string.
Any pointers to what else we can debug ?
Thanks,
Viral