[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997977#comment-14997977 ] Pradeep Subrahmanion commented on YARN-1565: Can anybody help me on how to proceed on this one ? > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch, YARN-1565-004.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976669#comment-14976669 ] Pradeep Subrahmanion commented on YARN-1565: Gentle reminder for review. > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch, YARN-1565-004.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972618#comment-14972618 ] Pradeep Subrahmanion commented on YARN-1565: Findbugs doesn't list any new warnings. No new issues in latest result. [~ste...@apache.org] - Could you please take a look at latest patch ? > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch, YARN-1565-004.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971085#comment-14971085 ] Pradeep Subrahmanion commented on YARN-1565: Thank you for your review comments. I will update the patch with documentation. > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[
https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971905#comment-14971905
]
Hadoop QA commented on YARN-1565:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 24m 39s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 9m 39s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 43s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | site | 3m 37s | Site still builds. |
| {color:red}-1{color} | checkstyle | 1m 41s | The applied patch generated 1
new checkstyle issues (total was 212, now 212). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 46s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 37s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 29s | The patch appears to introduce 2
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 26s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 62m 57s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 121m 18s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-api |
| FindBugs | module:hadoop-yarn-server-resourcemanager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12768358/YARN-1565-004.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle site |
| git revision | trunk / 600ad7b |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9551/console |
This message was automatically generated.
> Add a way for YARN clients to get critical YARN system properties from the RM
> -
>
> Key: YARN-1565
> URL: https://issues.apache.org/jira/browse/YARN-1565
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.2.0
>Reporter: Steve Loughran
> Attachments: YARN-1565-001.patch, YARN-1565-002.patch,
> YARN-1565-003.patch, YARN-1565-004.patch
>
>
> If you are trying to build up an AM request, you need to know
> # the limits of memory, core for the chosen queue
> # the existing YARN classpath
> # the path separator for the target platform (so your classpath comes out
> right)
> # cluster OS: in case you need some OS-specific changes
> The classpath can be in yarn-site.xml, but a remote client may not have that.
> The site-xml file doesn't list Queue resource limits, cluster OS or the path
> separator.
> A way to query the RM for these values would make it easier for YARN clients
> to build up AM submissions with less guesswork and client-side config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969562#comment-14969562 ] Steve Loughran commented on YARN-1565: -- -ignore the yarn config one. like you say: not a new problem. And no, no findbugs errors listed. > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969568#comment-14969568 ] Steve Loughran commented on YARN-1565: -- Looking at the code, given the SystemPropertyInfo is to be a new structure that will evolve over time, let's keep it as simple as possible -making the classpath field public and skipping the explicit accessor method. Otherwise, I like this: Addresses a core problem on hadoop apps. I think later on I'd like more of the environment and things, but starting with classpath means we don't have to worry about exposing security information which we could otherwise unintentionally leak. (other than the list of artifacts on the CP, hence potentially known vulnerabilities). Now, we'll also need some documentation in the yarn-site pages, won't we? > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[
https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969360#comment-14969360
]
Hadoop QA commented on YARN-1565:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 36s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 1s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 54s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 31s | The applied patch generated 1
new checkstyle issues (total was 211, now 211). |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 13s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 23s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 62m 28s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 107m 55s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-api |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12768040/YARN-1565-003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2798723 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9522/console |
This message was automatically generated.
> Add a way for YARN clients to get critical YARN system properties from the RM
> -
>
> Key: YARN-1565
> URL: https://issues.apache.org/jira/browse/YARN-1565
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.2.0
>Reporter: Steve Loughran
> Attachments: YARN-1565-001.patch, YARN-1565-002.patch,
> YARN-1565-003.patch
>
>
> If you are trying to build up an AM request, you need to know
> # the limits of memory, core for the chosen queue
> # the existing YARN classpath
> # the path separator for the target platform (so your classpath comes out
> right)
> # cluster OS: in case you need some OS-specific changes
> The classpath can be in yarn-site.xml, but a remote client may not have that.
> The site-xml file doesn't list Queue resource limits, cluster OS or the path
> separator.
> A way to query the RM for these values would make it easier for YARN clients
> to build up AM submissions with less guesswork and client-side config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969401#comment-14969401 ] Pradeep Subrahmanion commented on YARN-1565: The YarnConfiguration class was already over 2000 lines before this patch. Can I ignore this warning or do I need to refactor the code ? Also Find bugs shows 1 warning in test result. But 'Findbugs Report' page is not displaying any warnings. > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch, > YARN-1565-003.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967430#comment-14967430 ] Pradeep Subrahmanion commented on YARN-1565: I have attached latest patch. But I couldn't 'Cancel Patch' and Submit new one. Cancel Patch option is not available. Am I missing some permission ? > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[
https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967619#comment-14967619
]
Hadoop QA commented on YARN-1565:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 48s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 53s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 25s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 31s | The applied patch generated 1
new checkstyle issues (total was 211, now 211). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 13s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 23s | Tests passed in
hadoop-yarn-api. |
| {color:red}-1{color} | yarn tests | 57m 56s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 103m 9s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-api |
| Failed unit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12767805/YARN-1565-002.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / e27c2ae |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9509/console |
This message was automatically generated.
> Add a way for YARN clients to get critical YARN system properties from the RM
> -
>
> Key: YARN-1565
> URL: https://issues.apache.org/jira/browse/YARN-1565
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.2.0
>Reporter: Steve Loughran
> Attachments: YARN-1565-001.patch, YARN-1565-002.patch
>
>
> If you are trying to build up an AM request, you need to know
> # the limits of memory, core for the chosen queue
> # the existing YARN classpath
> # the path separator for the target platform (so your classpath comes out
> right)
> # cluster OS: in case you need some OS-specific changes
> The classpath can be in yarn-site.xml, but a remote client may not have that.
> The site-xml file doesn't list Queue resource limits, cluster OS or the path
> separator.
> A way to query the RM for these values would make it easier for YARN clients
> to build up AM submissions with less guesswork and client-side config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967595#comment-14967595 ] Steve Loughran commented on YARN-1565: -- -looks like you are; I've resubmitted it for you > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch, YARN-1565-002.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[
https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965517#comment-14965517
]
Hadoop QA commented on YARN-1565:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 19m 4s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 13s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 46s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 28s | The applied patch generated 1
new checkstyle issues (total was 211, now 211). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 6s | The patch appears to introduce 1
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 25s | Tests passed in
hadoop-yarn-api. |
| {color:red}-1{color} | yarn tests | 62m 15s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 108m 3s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-yarn-api |
| Failed unit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12767621/YARN-1565-001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 9cb5d35 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/9492/console |
This message was automatically generated.
> Add a way for YARN clients to get critical YARN system properties from the RM
> -
>
> Key: YARN-1565
> URL: https://issues.apache.org/jira/browse/YARN-1565
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.2.0
>Reporter: Steve Loughran
> Attachments: YARN-1565-001.patch
>
>
> If you are trying to build up an AM request, you need to know
> # the limits of memory, core for the chosen queue
> # the existing YARN classpath
> # the path separator for the target platform (so your classpath comes out
> right)
> # cluster OS: in case you need some OS-specific changes
> The classpath can be in yarn-site.xml, but a remote client may not have that.
> The site-xml file doesn't list Queue resource limits, cluster OS or the path
> separator.
> A way to query the RM for these values would make it easier for YARN clients
> to build up AM submissions with less guesswork and client-side config.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965579#comment-14965579 ] Steve Loughran commented on YARN-1565: -- test failed as HTML came back. One thing I'd recommend is that if an assert isn't met (e.g. content type, status code), the test should try to either print out the HTTP Response text or (maybe) include it in the exception test. That way we can debug the failures from just the test results > Add a way for YARN clients to get critical YARN system properties from the RM > - > > Key: YARN-1565 > URL: https://issues.apache.org/jira/browse/YARN-1565 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.2.0 >Reporter: Steve Loughran > Attachments: YARN-1565-001.patch > > > If you are trying to build up an AM request, you need to know > # the limits of memory, core for the chosen queue > # the existing YARN classpath > # the path separator for the target platform (so your classpath comes out > right) > # cluster OS: in case you need some OS-specific changes > The classpath can be in yarn-site.xml, but a remote client may not have that. > The site-xml file doesn't list Queue resource limits, cluster OS or the path > separator. > A way to query the RM for these values would make it easier for YARN clients > to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1565) Add a way for YARN clients to get critical YARN system properties from the RM
[ https://issues.apache.org/jira/browse/YARN-1565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048704#comment-14048704 ] Steve Loughran commented on YARN-1565: -- I think this should be part of the REST API -we just publish some JSON that provides this information to local and remote systems # the values listed above # all the special expanded variables you can use in command creation # a select subset of YARN/Hadoop properties:defaultFS, yarn.vmem, some other props we think are useful for clients and debugging. We shouldn't publish the whole aggregate -site.xml values as that can leak private keys to object stores. Add a way for YARN clients to get critical YARN system properties from the RM - Key: YARN-1565 URL: https://issues.apache.org/jira/browse/YARN-1565 Project: Hadoop YARN Issue Type: Improvement Components: resourcemanager Affects Versions: 2.2.0 Reporter: Steve Loughran If you are trying to build up an AM request, you need to know # the limits of memory, core c for the chosen queue # the existing YARN classpath # the path separator for the target platform (so your classpath comes out right) # cluster OS: in case you need some OS-specific changes The classpath can be in yarn-site.xml, but a remote client may not have that. The site-xml file doesn't list Queue resource limits, cluster OS or the path separator. A way to query the RM for these values would make it easier for YARN clients to build up AM submissions with less guesswork and client-side config. -- This message was sent by Atlassian JIRA (v6.2#6252)
