[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687842#comment-13687842
]
Hudson commented on HADOOP-9637:
Integrated in Hadoop-Yarn-trunk #245 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/245/])
HADOOP-9637. Adding Native Fstat for Windows as needed by YARN. Contributed
by Chuan Liu. (Revision 1494341)
Result = FAILURE
cnauroth :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494341
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chmod.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chown.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/include/winutils.h
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/libwinutils.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/ls.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Fix For: 3.0.0, 2.1.0-beta
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.3.patch,
HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687943#comment-13687943
]
Hudson commented on HADOOP-9637:
Integrated in Hadoop-Hdfs-trunk #1435 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1435/])
HADOOP-9637. Adding Native Fstat for Windows as needed by YARN. Contributed
by Chuan Liu. (Revision 1494341)
Result = FAILURE
cnauroth :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494341
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chmod.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chown.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/include/winutils.h
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/libwinutils.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/ls.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Fix For: 3.0.0, 2.1.0-beta
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.3.patch,
HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687988#comment-13687988
]
Hudson commented on HADOOP-9637:
Integrated in Hadoop-Mapreduce-trunk #1462 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1462/])
HADOOP-9637. Adding Native Fstat for Windows as needed by YARN. Contributed
by Chuan Liu. (Revision 1494341)
Result = FAILURE
cnauroth :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494341
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chmod.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chown.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/include/winutils.h
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/libwinutils.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/ls.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Fix For: 3.0.0, 2.1.0-beta
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.3.patch,
HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13686432#comment-13686432
]
Hadoop QA commented on HADOOP-9637:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588295/HADOOP-9637-trunk.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2663//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2663//console
This message is automatically generated.
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13686983#comment-13686983
]
Chris Nauroth commented on HADOOP-9637:
---
Thanks for addressing my questions. It all makes sense. I have just one more
comment about the new patch.
{code}
if (Shell.WINDOWS) {
int i = owner.indexOf('\\');
if (i != -1)
this.owner = owner.substring(i + 1);
}
{code}
This code is identical for username and group, so let's refactor it behind a
helper function. We also can add comments stating that the reason for this is
to remove domain.
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687168#comment-13687168
]
Hadoop QA commented on HADOOP-9637:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12588442/HADOOP-9637-trunk.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2665//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2665//console
This message is automatically generated.
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.3.patch,
HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687292#comment-13687292
]
Hudson commented on HADOOP-9637:
Integrated in Hadoop-trunk-Commit #3971 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3971/])
HADOOP-9637. Adding Native Fstat for Windows as needed by YARN. Contributed
by Chuan Liu. (Revision 1494341)
Result = SUCCESS
cnauroth :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1494341
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/nativeio/NativeIO.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chmod.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/chown.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/include/winutils.h
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/libwinutils.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/winutils/ls.c
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/nativeio/TestNativeIO.java
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Attachments: HADOOP-9637-trunk.2.patch, HADOOP-9637-trunk.3.patch,
HADOOP-9637-trunk.patch
In the YARN, nodemanager need to enforce the log file can only be accessed by
the owner. At various places, {{SecureIOUtils.openForRead()}} was called to
enforce this check. We don't have {{NativeIO.Posix.getFstat()}} used by
{{SecureIOUtils.openForRead()}} on Windows, and this make the check fail on
Windows. The YARN unit tests
TestAggregatedLogFormat.testContainerLogsFileAccess and
TestContainerLogsPage.testContainerLogPageAccess fail on Windows because of
this.
The JIRA try to provide a Windows implementation of
{{NativeIO.Posix.getFstat()}}.
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9637) Adding Native Fstat for Windows as needed by YARN
[
https://issues.apache.org/jira/browse/HADOOP-9637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13685978#comment-13685978
]
Chuan Liu commented on HADOOP-9637:
---
May be you should edit the title to be something in the line of Adding Native
Fstat for Windows as needed by YARN.
Thanks for the suggestion! I have changed the title; will update the
description shortly.
Adding Native Fstat for Windows as needed by YARN
-
Key: HADOOP-9637
URL: https://issues.apache.org/jira/browse/HADOOP-9637
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0, 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Attachments: HADOOP-9637-trunk.patch
TestAggregatedLogFormat.testContainerLogsFileAccess test case fails on
Windows. The test case try to simulate a situation where first log file is
owned by different user (probably symlink) and second one by the user itself.
In this situation, the attempt to try to aggregate the logs should fail with
the error message Owner ... for path ... did not match expected owner
The check on file owner happens at {{AggregatedLogFormat.write()}} method.
The method calls {{SecureIOUtils.openForRead()}} to read the log files before
writing out to the OutputStream.
{{SecureIOUtils.openForRead()}} use {{NativeIO.Posix.getFstat()}} to get the
file owner and group. We don't have {{NativeIO.Posix.getFstat()}}
implementation on Windows; thus, the failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
