[
https://issues.apache.org/jira/browse/HADOOP-9631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683623#comment-13683623
]
Chris Nauroth commented on HADOOP-9631:
---------------------------------------
Hi Lohit,
I'm looking at version 2 of the patch, which always resolves to the home
directory to get server defaults. This might appear to be working for the
specific use case you need (YARN NodeManager log aggregation), but I don't
think it would be generally correct in all cases.
For example, imagine a viewfs configured with mount points mapping /user to
local file system and /data to an HDFS cluster. Then, creating a file in /data
would still get the undesired replication value.
That example is a bit contrived, but we can also consider the more realistic
example from the {{ViewFs}} javadocs:
{code}
* For example one could have a mount table that provides links such as
* <ul>
* <li> /user -> hdfs://nnContainingUserDir/user
* <li> /project/foo -> hdfs://nnProject1/projects/foo
* <li> /project/bar -> hdfs://nnProject2/projects/bar
* <li> /tmp -> hdfs://nnTmp/privateTmpForUserXXX
* </ul>
{code}
Now, assume the client is creating file /project/foo/myFile. The operation
would pick up server defaults associated with /user (configured on
nnContainingUserDir) instead of the desired /project/foo (configured on
nnProject1). The server defaults may differ between nnContainingUserDir and
nnProject1.
It seems like we need a patch that is sensitive to the actual {{Path}} accessed
in the client operation, similar to what Daryn mentioned earlier.
Also, I don't think the new unit test exercises this code, because
{{TestHDFSFileContextMainOperations}} doesn't configure a {{ViewFs}}.
> ViewFs should use underlying FileSystem's server side defaults
> --------------------------------------------------------------
>
> Key: HADOOP-9631
> URL: https://issues.apache.org/jira/browse/HADOOP-9631
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, viewfs
> Affects Versions: 2.0.4-alpha
> Reporter: Lohit Vijayarenu
> Attachments: HADOOP-9631.trunk.1.patch, HADOOP-9631.trunk.2.patch,
> TestFileContext.java
>
>
> On a cluster with ViewFS as default FileSystem, creating files using
> FileContext will always result with replication factor of 1, instead of
> underlying filesystem default (like HDFS)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
