When connecting to HDFS using an IP Task MapRed gets confused when checking
the output path.
----------------------------------------------------------------------------------------------
Key: HADOOP-5610
URL: https://issues.apache.org/jira/browse/HADOOP-5610
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.19.1, 0.19.0, 0.18.3
Environment: AIX 6.1 and Solaris
Reporter: Bill Habermaas
Priority: Minor
The exception appearing below happens if you are using an IP address to
reference a HDFS file system path. Although the URI authority for either path
(whether referenced by hostname:port or IP-address:port) refer to the same
filesystem, Task throws an exception because the stringified comparisons of the
authority in the URIs are not equal. The FileSystem holds the network location
of the namenode as a host name regardless as to whether an IP address or
hostname is used when the file system is connected (refer to HADOOP-5191 for an
earlier fix specifically for HDFS only).
2009-03-27 04:15:45,045 WARN [Thread-145]
org.apache.hadoop.mapred.LocalJobRunner: job_local_0002
java.io.IOException: Can not get the relative path: base =
hdfs://10.120.16.68:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0
child =
hdfs://p520aix61.mydomain.com:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0/part-00000
at org.apache.hadoop.mapred.Task.getFinalPath(Task.java:586)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:599)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:617)
at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:561)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:202)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
