[
https://issues.apache.org/jira/browse/HADOOP-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640074#comment-13640074
]
Ivan Mitic commented on HADOOP-9413:
------------------------------------
Thanks Arpit and Chris for preparing this patch! I kicked off a full test run
on Windows to validate the approach.
In parallel, I have a few questions/comments on the latest patch:
1. hadoop-project/pom.xml: It seems that the currently defined value for
LD_LIBRARY_PATH is invalid. I think it should have the following value:
{code}
<LD_LIBRARY_PATH>${env.LD_LIBRARY_PATH}:${project.build.directory}/native/target/usr/local/lib:${basedir}/../../hadoop-common-project/hadoop-common/target/bin</LD_LIBRARY_PATH>
{code}
2. hadoop-project/pom.xml: Assuming that we specified a valid value for
LD_LIBRARY_PATH above, the following section might not be needed:
{code}
<!-- Specify where to look for the native DLL on Windows -->
<PATH>${env.PATH};${basedir}/../../hadoop-common-project/hadoop-common/target/bin</PATH>
<PATH>${env.PATH};${hadoop.common.build.dir}/bin</PATH>
{code}
3. Other pom.xml files: Can we just define a new value for LD_LIBRARY_PATH next
to HADOOP_HOME and point it to the appropriate /target/bin folder in every
subproject? This would provide a nice symmetry with how we locate winutils on
Windows.
4. Might make sense to either remove hadoop.common.build.dir or use it for both
winutils.exe and hadoop.dll
> Introduce common utils for File#setReadable/Writable/Executable and
> File#canRead/Write/Execute that work cross-platform
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-9413
> URL: https://issues.apache.org/jira/browse/HADOOP-9413
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Fix For: 3.0.0
>
> Attachments: HADOOP-9413.commonfileutils.2.patch,
> HADOOP-9413.commonfileutils.3.patch, HADOOP-9413.commonfileutils.patch
>
>
> So far, we've seen many unittest and product bugs in Hadoop on Windows
> because Java's APIs that manipulate with permissions do not work as expected.
> We've addressed many of these problems on one-by-one basis (by either
> changing code a bit or disabling the test). While debugging the remaining
> unittest failures we continue to run into the same patterns of problems, and
> instead of addressing them one-by-one, I propose that we expose a set of
> equivalent wrapper APIs that will work well for all platforms.
> Scanning thru the codebase, this will actually be a simple change as there
> are very few places that use File#setReadable/Writable/Executable and
> File#canRead/Write/Execute (5 files in Common, 9 files in HDFS).
> HADOOP-8973 contains additional context on the problem.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
