[
https://issues.apache.org/jira/browse/HADOOP-5901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790989#action_12790989
]
Aaron Kimball commented on HADOOP-5901:
---------------------------------------
Yes, changing the signature for {{FileSystem.setDefaultUri(Configuration,
String)}} to throw an IOException if it cannot create a proper URI from the
String argument would be an incompatible change.
Digging into the changes necessary in the MapReduce codebase, it appears that
({{MiniMRCluster.configureJobConf()}}) is the only use of
FileSystem.setDefaultUri(). So that could be changed to itself throw
IOException easily enough. But that is called by
{{MiniMRCluster.createJobConf()}} which is used in several dozen places
throughout the test codebase.
On the HDFS codebase side, it looks as though all the uses of setDefaultUri are
in test cases which can all be modified to throw more exceptions than they do
now in a straightforward way; or else in places where the URI is generated
programmatically and we know that it's going to be valid. (Hence we can swallow
the IOException.)
So I think our options are as follows:
# Modify {{MiniMRCluster.createJobConf()}} to also throw IOException and modify
all call-sites to catch IOException.
# Early in this patch's development I was throwing an IllegalArgumentException
(unchecked) for this reason (preventing incompatibility). Konstantin S: Are you
still pro-IOException here?
# Make the {{FileSystem.checkName()}} method public. The {{namenode}} argument
in MiniMRCluster which is used to format the URI is passed into MiniMRCluster's
constructor. With a public checkName, we could call {{checkName(namenode)}} in
the MiniMRCluster constructor, which already throws IOException, so clients
already trap that. Then we could swallow the exception (maybe with a log4j
message) in {{createJobConf()}}, knowing that it should never be thrown since
we already verified the URI's sanity. This increases the size of the public
static FileSystem API.
> FileSystem.fixName() has unexpected behaviour
> ---------------------------------------------
>
> Key: HADOOP-5901
> URL: https://issues.apache.org/jira/browse/HADOOP-5901
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.21.0
> Reporter: Steve Loughran
> Assignee: Aaron Kimball
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: HADOOP-5901.2.patch, HADOOP-5901.3.patch,
> HADOOP-5901.patch
>
>
> {{FileSystem.fixName()}} tries to patch up fs.default.name values, but I'm
> not sure it helps that well.
> Has it been warning about deprecated values for long enough for it to be
> turned off?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
