[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179829#comment-14179829 ] Hudson commented on YARN-2720: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #720 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/720/]) YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. (cnauroth: rev 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java * hadoop-yarn-project/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Classpath.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java Windows: Wildcard classpath variables not expanded against resources contained in archives -- Key: YARN-2720 URL: https://issues.apache.org/jira/browse/YARN-2720 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Craig Welch Assignee: Craig Welch Fix For: 2.6.0 Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch On windows there are limitations to the length of command lines and environment variables which prevent placing all classpath resources into these elements. Instead, a jar containing only a classpath manifest is created to provide the classpath. During this process wildcard references are expanded by inspecting the filesystem. Since archives are extracted to a different location and linked into the final location after the classpath jar is created, resources referred to via wildcards which exist in localized archives (.zip, tar.gz) are not added to the classpath manifest jar. Since these entries are removed from the final classpath for the container they are not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179917#comment-14179917 ] Hudson commented on YARN-2720: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1909 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1909/]) YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. (cnauroth: rev 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5) * hadoop-yarn-project/CHANGES.txt * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Classpath.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java Windows: Wildcard classpath variables not expanded against resources contained in archives -- Key: YARN-2720 URL: https://issues.apache.org/jira/browse/YARN-2720 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Craig Welch Assignee: Craig Welch Fix For: 2.6.0 Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch On windows there are limitations to the length of command lines and environment variables which prevent placing all classpath resources into these elements. Instead, a jar containing only a classpath manifest is created to provide the classpath. During this process wildcard references are expanded by inspecting the filesystem. Since archives are extracted to a different location and linked into the final location after the classpath jar is created, resources referred to via wildcards which exist in localized archives (.zip, tar.gz) are not added to the classpath manifest jar. Since these entries are removed from the final classpath for the container they are not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14180009#comment-14180009 ] Hudson commented on YARN-2720: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1934 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1934/]) YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. (cnauroth: rev 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Classpath.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java * hadoop-yarn-project/CHANGES.txt Windows: Wildcard classpath variables not expanded against resources contained in archives -- Key: YARN-2720 URL: https://issues.apache.org/jira/browse/YARN-2720 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Craig Welch Assignee: Craig Welch Fix For: 2.6.0 Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch On windows there are limitations to the length of command lines and environment variables which prevent placing all classpath resources into these elements. Instead, a jar containing only a classpath manifest is created to provide the classpath. During this process wildcard references are expanded by inspecting the filesystem. Since archives are extracted to a different location and linked into the final location after the classpath jar is created, resources referred to via wildcards which exist in localized archives (.zip, tar.gz) are not added to the classpath manifest jar. Since these entries are removed from the final classpath for the container they are not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[
https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178894#comment-14178894
]
Hadoop QA commented on YARN-2720:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12676140/YARN-2720.4.patch
against trunk revision 6f2028b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5485//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5485//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5485//console
This message is automatically generated.
Windows: Wildcard classpath variables not expanded against resources
contained in archives
--
Key: YARN-2720
URL: https://issues.apache.org/jira/browse/YARN-2720
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Reporter: Craig Welch
Assignee: Craig Welch
Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch
On windows there are limitations to the length of command lines and
environment variables which prevent placing all classpath resources into
these elements. Instead, a jar containing only a classpath manifest is
created to provide the classpath. During this process wildcard references
are expanded by inspecting the filesystem. Since archives are extracted to a
different location and linked into the final location after the classpath jar
is created, resources referred to via wildcards which exist in localized
archives (.zip, tar.gz) are not added to the classpath manifest jar. Since
these entries are removed from the final classpath for the container they are
not on the container's classpath as they should be.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178897#comment-14178897 ] Chris Nauroth commented on YARN-2720: - The Findbugs warnings are unrelated. I'll commit this. Windows: Wildcard classpath variables not expanded against resources contained in archives -- Key: YARN-2720 URL: https://issues.apache.org/jira/browse/YARN-2720 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Craig Welch Assignee: Craig Welch Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch On windows there are limitations to the length of command lines and environment variables which prevent placing all classpath resources into these elements. Instead, a jar containing only a classpath manifest is created to provide the classpath. During this process wildcard references are expanded by inspecting the filesystem. Since archives are extracted to a different location and linked into the final location after the classpath jar is created, resources referred to via wildcards which exist in localized archives (.zip, tar.gz) are not added to the classpath manifest jar. Since these entries are removed from the final classpath for the container they are not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2720) Windows: Wildcard classpath variables not expanded against resources contained in archives
[ https://issues.apache.org/jira/browse/YARN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178926#comment-14178926 ] Hudson commented on YARN-2720: -- SUCCESS: Integrated in Hadoop-trunk-Commit #6303 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6303/]) YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. (cnauroth: rev 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Classpath.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/WindowsSecureContainerExecutor.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileUtil.java Windows: Wildcard classpath variables not expanded against resources contained in archives -- Key: YARN-2720 URL: https://issues.apache.org/jira/browse/YARN-2720 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Reporter: Craig Welch Assignee: Craig Welch Fix For: 2.6.0 Attachments: YARN-2720.2.patch, YARN-2720.3.patch, YARN-2720.4.patch On windows there are limitations to the length of command lines and environment variables which prevent placing all classpath resources into these elements. Instead, a jar containing only a classpath manifest is created to provide the classpath. During this process wildcard references are expanded by inspecting the filesystem. Since archives are extracted to a different location and linked into the final location after the classpath jar is created, resources referred to via wildcards which exist in localized archives (.zip, tar.gz) are not added to the classpath manifest jar. Since these entries are removed from the final classpath for the container they are not on the container's classpath as they should be. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
