[
https://issues.apache.org/jira/browse/HADOOP-5958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768900#action_12768900
]
Aaron Kimball commented on HADOOP-5958:
---------------------------------------
Hm. A casual inspection shows that uses of DF do not seem to depend on
superclass behavior. I concur that we're probably safe with just changing its
interface outright and flagging the change as incompatible in JIRA.
As for your other question... not so easy.
{{File}} provides a {{listRoots()}} method which doesn't exactly do the same
thing. A quick test shows that it just returns {{"/"}} for a file named
{{/home/aaron/foo}}, even though I have {{/home}} on a separate partition than
{{/}}.
The {{getMounts()}} method is used in
{{LocalFileSystem.reportChecksumFailure()}} to move files to a {{bad_files}}
directory on the same device as their original storage. Replacing
{{getMounts()}} naively with {{listRoots()}} could result in actual IO
transfer, vs. just metadata updates in the local fs.
In Java 7, the {{java.nio.file.FileSystem}} class (and friends like {{Path}})
will actually make all this trivial. It'll even let you change and inspect file
ownership and permissions. But we're not there yet. ::sigh::
So we can't eliminate getMounts() without changing the interface. Fortunately,
it seems as though this is only used in one method, so maybe changing
LFS.reportChecksumFailure() is the right answer?
Here's an alternative strategy:
* Make {{o.a.h.fs.DF}} an interface
* The current DF implementation is renamed {{ShellDF}}
* Add another implementation named {{JavaDF}} which uses {{File}} for what it
can.
** Throws {{IOException}} for things like {{getMounts()}}, etc.
* The {{NativeFileSystem}} specifically instantiates a {{ShellDF}} for the
situation where it needs mount-point info (inside reportChecksumFailure)
* All other uses of DF change to JavaDF
> Use JDK 1.6 File APIs in DF.java wherever possible
> --------------------------------------------------
>
> Key: HADOOP-5958
> URL: https://issues.apache.org/jira/browse/HADOOP-5958
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Devaraj Das
> Fix For: 0.21.0
>
>
> JDK 1.6 has File APIs like File.getFreeSpace() which should be used instead
> of spawning a command process for getting the various disk/partition related
> attributes. This would avoid spikes in memory consumption by tasks when
> things like LocalDirAllocator is used for creating paths on the filesystem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
