[
https://issues.apache.org/jira/browse/HADOOP-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592734#comment-13592734
]
Chris Nauroth commented on HADOOP-8973:
---------------------------------------
Here was the last discussion on this issue:
{quote}
Can we try the following approach that has the advantage of being common across
platforms.
Get FileStatus of the file/dir using FileSystem API
getPermission() from FileStatus
There are a bunch of impliesRead(), impliesWrite() functions that tell whether
a given FsPermission object allows read/write etc. I am sorry I don't remember
where these functions are.
Using these functions one can get the equivalent of isReadable/isWritable
{quote}
Digging into this, unfortunately no, we cannot reconstruct the logic of
{{DiskChecker}} in a cross-platform way by using {{FsPermission}} and
{{FsAction}} (with delegation to winutils ls when running on Windows). The
reason that methods like {{File#canRead}} are used here is that it checks if
the current process can read the file, with full consideration of the specific
user that launched the process. Our {{FsPermission}} model considers
permissions, but it does not consider how those permissions map to specific
users (nor would it be trivial to expand it to include this information).
For example, using {{FsPermission}} + winutils ls, we could see that a
particular directory has permissions 770, so the owner and members of the
primary group have full access to this directory. If the user that launched
the process is either the owner or a member of the primary group, then
{{DiskChecker}} must report that the disk is usable. If not, then
{{DiskChecker}} must report that the disk is unusable. However,
{{DiskChecker}} doesn't have enough information to choose the correct result.
It would be error-prone to go down the path of trying to duplicate the exact
permission checking logic enforced by the OS on the local file system. You
might argue that we could solve the example above by trying to combine more
calls to get the primary group and check if the current user's groups include
that group. However, that wouldn't be sufficient to cover more complex rules
enforced by local file systems, such as POSIX ACLs. It would be challenging to
get this exactly right, and since the logic is different on different OSes, it
wouldn't satisfy the goal of trying to keep this layer of code
platform-agnostic anyway.
I propose that we accept the patch I wrote earlier, still attached to the jira,
so that {{DiskChecker}} actually attempts a few file system operations for
verification when running on Windows. These work-arounds could be removed
whenever Hadoop upgrades to JDK7. Then, we can rewrite {{DiskChecker}} to use
the new Java {{FileSystem}} API, which doesn't have these bugs. I've confirmed
that the existing patch still applies cleanly to branch-trunk-win and fixes
{{TestDiskChecker}} so that it passes on both Mac and Windows.
> DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS
> ACLs
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-8973
> URL: https://issues.apache.org/jira/browse/HADOOP-8973
> Project: Hadoop Common
> Issue Type: Bug
> Components: util
> Affects Versions: trunk-win
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-8973-branch-trunk-win.patch
>
>
> DiskChecker.checkDir uses File.canRead, File.canWrite, and File.canExecute to
> check if a directory is inaccessible. These APIs are not reliable on Windows
> with NTFS ACLs due to a known JVM bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
