[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975155#comment-13975155 ] Hudson commented on YARN-1865: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1763 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1763/]) YARN-1865. ShellScriptBuilder does not check for some error conditions. Contributed by Remus Rusanu. (ivanmi: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588693) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Fix For: 2.5.0 > > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch, > YARN-1865.4.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975145#comment-13975145 ] Hudson commented on YARN-1865: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1738 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1738/]) YARN-1865. ShellScriptBuilder does not check for some error conditions. Contributed by Remus Rusanu. (ivanmi: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588693) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Fix For: 2.5.0 > > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch, > YARN-1865.4.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975107#comment-13975107 ] Hudson commented on YARN-1865: -- FAILURE: Integrated in Hadoop-Yarn-trunk #546 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/546/]) YARN-1865. ShellScriptBuilder does not check for some error conditions. Contributed by Remus Rusanu. (ivanmi: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588693) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Fix For: 2.5.0 > > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch, > YARN-1865.4.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974937#comment-13974937 ] Hudson commented on YARN-1865: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5541 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5541/]) YARN-1865. ShellScriptBuilder does not check for some error conditions. Contributed by Remus Rusanu. (ivanmi: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1588693) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Fix For: 2.5.0 > > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch, > YARN-1865.4.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974930#comment-13974930 ] Ivan Mitic commented on YARN-1865: -- +1 on the latest patch. Will commit now. > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch, > YARN-1865.4.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[
https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13974661#comment-13974661
]
Hadoop QA commented on YARN-1865:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12640911/YARN-1865.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3597//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3597//console
This message is automatically generated.
> ShellScriptBuilder does not check for some error conditions
> ---
>
> Key: YARN-1865
> URL: https://issues.apache.org/jira/browse/YARN-1865
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
>Priority: Minor
> Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch,
> YARN-1865.4.patch
>
>
> The WindowsShellScriptBuilder does not check for commands exceeding windows
> maximum shell command line length (8191 chars)
> Neither Unix nor Windows script builder do not check for error condition
> after mkdir nor link
> WindowsShellScriptBuilder mkdir is not safe with regard to paths containing
> spaces
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13962405#comment-13962405 ] Ivan Mitic commented on YARN-1865: -- Thanks Remus for the new patch. I have one more comment, can you please also include a unit test that validates errorCheck() behavior on Unix? You can either try to come up with a shell cmd line that will fail on both Unix and Windows, or add a separate unittest for Unix. > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[
https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957504#comment-13957504
]
Hadoop QA commented on YARN-1865:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12638219/YARN-1865.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3507//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3507//console
This message is automatically generated.
> ShellScriptBuilder does not check for some error conditions
> ---
>
> Key: YARN-1865
> URL: https://issues.apache.org/jira/browse/YARN-1865
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
>Priority: Minor
> Attachments: YARN-1865.1.patch, YARN-1865.2.patch, YARN-1865.3.patch
>
>
> The WindowsShellScriptBuilder does not check for commands exceeding windows
> maximum shell command line length (8191 chars)
> Neither Unix nor Windows script builder do not check for error condition
> after mkdir nor link
> WindowsShellScriptBuilder mkdir is not safe with regard to paths containing
> spaces
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948295#comment-13948295 ] Ivan Mitic commented on YARN-1865: -- Thanks Remus for the patch! I have two minor comments: 1. Shell#checkWindowsCommandLineLength: Can you please add a comment saying that caller is responsible to account for spaces between individual commands. It might also be useful to include info in the message on what the command actual length was and first 100 chars or so that it's even easier to pinpoint the problem. 2. ContainerLaunch: Would you mind also including unittest that validates that errorCheck() works as expected on Windows/Linux. Just do it for one random command (e.q. mkdirs with invalid chars in the name), no need to do it for all. Otherwise, the patch looks good to me. > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Attachments: YARN-1865.1.patch, YARN-1865.2.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[
https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945787#comment-13945787
]
Hadoop QA commented on YARN-1865:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636441/YARN-1865.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3447//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3447//console
This message is automatically generated.
> ShellScriptBuilder does not check for some error conditions
> ---
>
> Key: YARN-1865
> URL: https://issues.apache.org/jira/browse/YARN-1865
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
>Priority: Minor
> Attachments: YARN-1865.1.patch, YARN-1865.2.patch
>
>
> The WindowsShellScriptBuilder does not check for commands exceeding windows
> maximum shell command line length (8191 chars)
> Neither Unix nor Windows script builder do not check for error condition
> after mkdir nor link
> WindowsShellScriptBuilder mkdir is not safe with regard to paths containing
> spaces
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[
https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945707#comment-13945707
]
Hadoop QA commented on YARN-1865:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12636335/YARN-1865.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.yarn.server.nodemanager.TestContainerExecutor
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3445//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3445//console
This message is automatically generated.
> ShellScriptBuilder does not check for some error conditions
> ---
>
> Key: YARN-1865
> URL: https://issues.apache.org/jira/browse/YARN-1865
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
>Priority: Minor
> Attachments: YARN-1865.1.patch
>
>
> The WindowsShellScriptBuilder does not check for commands exceeding windows
> maximum shell command line length (8191 chars)
> Neither Unix nor Windows script builder do not check for error condition
> after mkdir nor link
> WindowsShellScriptBuilder mkdir is not safe with regard to paths containing
> spaces
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1865) ShellScriptBuilder does not check for some error conditions
[ https://issues.apache.org/jira/browse/YARN-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13945660#comment-13945660 ] Remus Rusanu commented on YARN-1865: Patch contains a bogus change to TestContainerExecutor.java and a space only change in shell.java. I'll upload a new iteration, sorry. > ShellScriptBuilder does not check for some error conditions > --- > > Key: YARN-1865 > URL: https://issues.apache.org/jira/browse/YARN-1865 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.0.0, 2.2.0, 2.3.0 >Reporter: Remus Rusanu >Assignee: Remus Rusanu >Priority: Minor > Attachments: YARN-1865.1.patch > > > The WindowsShellScriptBuilder does not check for commands exceeding windows > maximum shell command line length (8191 chars) > Neither Unix nor Windows script builder do not check for error condition > after mkdir nor link > WindowsShellScriptBuilder mkdir is not safe with regard to paths containing > spaces -- This message was sent by Atlassian JIRA (v6.2#6252)
