[jira] [Commented] (YARN-3381) A typographical error in "InvalidStateTransitonException"
[
https://issues.apache.org/jira/browse/YARN-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525103#comment-14525103
]
Hadoop QA commented on YARN-3381:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 32s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 32s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 42s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 36s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 19s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | mapreduce tests | 8m 50s | Tests passed in
hadoop-mapreduce-client-app. |
| {color:green}+1{color} | yarn tests | 6m 47s | Tests passed in
hadoop-yarn-client. |
| {color:green}+1{color} | yarn tests | 1m 55s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 5m 50s | Tests passed in
hadoop-yarn-server-nodemanager. |
| {color:green}+1{color} | yarn tests | 52m 13s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 117m 46s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729924/YARN-3381-003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| hadoop-mapreduce-client-app test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
|
| hadoop-yarn-client test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/artifact/patchprocess/testrun_hadoop-yarn-client.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7630/console |
This message was automatically generated.
> A typographical error in "InvalidStateTransitonException"
> -
>
> Key: YARN-3381
> URL: https://issues.apache.org/jira/browse/YARN-3381
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: api
>Affects Versions: 2.6.0
>Reporter: Xiaoshuang LU
>Assignee: Brahma Reddy Battula
> Attachments: YARN-3381-002.patch, YARN-3381-003.patch, YARN-3381.patch
>
>
> Appears that "InvalidStateTransitonException" should be
> "InvalidStateTransitionException". Transition was misspelled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3565) NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String
[ https://issues.apache.org/jira/browse/YARN-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-3565: Attachment: YARN-3565-20150502-1.patch Hi [~Wangd], Attaching a patch with the modifications to support NodeLabel instead of string in NM HB/Register > NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object > instead of String > - > > Key: YARN-3565 > URL: https://issues.apache.org/jira/browse/YARN-3565 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, client, resourcemanager >Reporter: Wangda Tan >Assignee: Naganarasimha G R >Priority: Blocker > Attachments: YARN-3565-20150502-1.patch > > > Now NM HB/Register uses Set, it will be hard to add new fields if we > want to support specifying NodeLabel type such as exclusivity/constraints, > etc. We need to make sure rolling upgrade works. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens
[
https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525031#comment-14525031
]
Hadoop QA commented on YARN-2674:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684681/YARN-2674.2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7649/console |
This message was automatically generated.
> Distributed shell AM may re-launch containers if RM work preserving restart
> happens
> ---
>
> Key: YARN-2674
> URL: https://issues.apache.org/jira/browse/YARN-2674
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Chun Chen
> Attachments: YARN-2674.1.patch, YARN-2674.2.patch
>
>
> Currently, if RM work preserving restart happens while distributed shell is
> running, distribute shell AM may re-launch all the containers, including
> new/running/complete. We must make sure it won't re-launch the
> running/complete containers.
> We need to remove allocated containers from
> AMRMClientImpl#remoteRequestsTable once AM receive them from RM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2913) Fair scheduler should have ability to set MaxResourceDefault for each queue
[
https://issues.apache.org/jira/browse/YARN-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525037#comment-14525037
]
Hadoop QA commented on YARN-2913:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684708/YARN-2913.v1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7652/console |
This message was automatically generated.
> Fair scheduler should have ability to set MaxResourceDefault for each queue
> ---
>
> Key: YARN-2913
> URL: https://issues.apache.org/jira/browse/YARN-2913
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: YARN-2913.v1.patch
>
>
> Queues that are created on the fly have the max resource of the entire
> cluster. Fair Scheduler should have a default maxResource to control the
> maxResource of those queues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.
[
https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525039#comment-14525039
]
Hadoop QA commented on YARN-221:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12637905/YARN-221-trunk-v2.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7654/console |
This message was automatically generated.
> NM should provide a way for AM to tell it not to aggregate logs.
>
>
> Key: YARN-221
> URL: https://issues.apache.org/jira/browse/YARN-221
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Robert Joseph Evans
>Assignee: Ming Ma
> Attachments: YARN-221-trunk-v1.patch, YARN-221-trunk-v2.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the
> logs should not be aggregated, that they should be aggregated with a high
> priority, or that they should be aggregated but with a lower priority. The
> AM should be able to do this in the ContainerLaunch context to provide a
> default value, but should also be able to update the value when the container
> is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid
> connection to the NN at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2672) Improve Gridmix (synthetic generator + reservation support)
[
https://issues.apache.org/jira/browse/YARN-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525033#comment-14525033
]
Hadoop QA commented on YARN-2672:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12674061/YARN-2672.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7651/console |
This message was automatically generated.
> Improve Gridmix (synthetic generator + reservation support)
> ---
>
> Key: YARN-2672
> URL: https://issues.apache.org/jira/browse/YARN-2672
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, fairscheduler, resourcemanager
>Reporter: Carlo Curino
>Assignee: Carlo Curino
> Attachments: YARN-2672.patch
>
>
> This JIRA proposes an enhancement of Gridmix that contains:
> 1) a synthetic generator to produce load without the need of a trace, but
> based on distributions
> 2) include negotiation of reservations (to test YARN-1051).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[
https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525024#comment-14525024
]
Hadoop QA commented on YARN-2554:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12670251/YARN-2554.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7648/console |
This message was automatically generated.
> Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
> -
>
> Key: YARN-2554
> URL: https://issues.apache.org/jira/browse/YARN-2554
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 2.6.0
>Reporter: Jonathan Maron
> Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch,
> YARN-2554.3.patch
>
>
> If the HTTP policy to enable HTTPS is specified, the RM and AM are
> initialized with SSL listeners. The RM has a web app proxy servlet that acts
> as a proxy for incoming AM requests. In order to forward the requests to the
> AM the proxy servlet makes use of HttpClient. However, the HttpClient
> utilized is not initialized correctly with the necessary certs to allow for
> successful one way SSL invocations to the other nodes in the cluster (it is
> not configured to access/load the client truststore specified in
> ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be
> utilized to create an instance that can be assigned to the HttpClient.
> The symptoms of this issue are:
> AM: Displays "unknown_certificate" exception
> RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1391) Lost node list should be identify by NodeId
[
https://issues.apache.org/jira/browse/YARN-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525032#comment-14525032
]
Hadoop QA commented on YARN-1391:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12672452/YARN-1391.v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7650/console |
This message was automatically generated.
> Lost node list should be identify by NodeId
> ---
>
> Key: YARN-1391
> URL: https://issues.apache.org/jira/browse/YARN-1391
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.0.5-alpha
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: YARN-1391.v1.patch, YARN-1391.v2.patch
>
>
> in case of multiple node managers on a single machine. each of them should be
> identified by NodeId, which is more unique than just host name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2756) use static variable (Resources.none()) for not-running Node.resource in CommonNodeLabelsManager to save memory.
[
https://issues.apache.org/jira/browse/YARN-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525038#comment-14525038
]
Hadoop QA commented on YARN-2756:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677441/YARN-2756.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7653/console |
This message was automatically generated.
> use static variable (Resources.none()) for not-running Node.resource in
> CommonNodeLabelsManager to save memory.
> ---
>
> Key: YARN-2756
> URL: https://issues.apache.org/jira/browse/YARN-2756
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2756.000.patch
>
>
> use static variable (Resources.none()) for not-running Node.resource in
> CommonNodeLabelsManager to save memory. When a Node is not activated, the
> resource is never used, When a Node is activated, a new resource will be
> assigned to it in RMNodeLabelsManager#activateNode (nm.resource = resource)
> So it would be better to use static variable Resources.none() instead of
> allocating a new variable(Resource.newInstance(0, 0)) for each node
> deactivation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1813) Better error message for "yarn logs" when permission denied
[
https://issues.apache.org/jira/browse/YARN-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525010#comment-14525010
]
Hadoop QA commented on YARN-1813:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677648/YARN-1813.6.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7645/console |
This message was automatically generated.
> Better error message for "yarn logs" when permission denied
> ---
>
> Key: YARN-1813
> URL: https://issues.apache.org/jira/browse/YARN-1813
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.3.0, 2.4.1, 2.5.1
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Minor
> Attachments: YARN-1813.1.patch, YARN-1813.2.patch, YARN-1813.2.patch,
> YARN-1813.3.patch, YARN-1813.4.patch, YARN-1813.5.patch, YARN-1813.6.patch
>
>
> I ran some MR jobs as the "hdfs" user, and then forgot to sudo -u when
> grabbing the logs. "yarn logs" prints an error message like the following:
> {noformat}
> [andrew.wang@a2402 ~]$ yarn logs -applicationId application_1394482121761_0010
> 14/03/10 16:05:10 INFO client.RMProxy: Connecting to ResourceManager at
> a2402.halxg.cloudera.com/10.20.212.10:8032
> Logs not available at
> /tmp/logs/andrew.wang/logs/application_1394482121761_0010
> Log aggregation has not completed or is not enabled.
> {noformat}
> It'd be nicer if it said "Permission denied" or "AccessControlException" or
> something like that instead, since that's the real issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2761) potential race condition in SchedulingPolicy
[
https://issues.apache.org/jira/browse/YARN-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525016#comment-14525016
]
Hadoop QA commented on YARN-2761:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677844/YARN-2761.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7646/console |
This message was automatically generated.
> potential race condition in SchedulingPolicy
>
>
> Key: YARN-2761
> URL: https://issues.apache.org/jira/browse/YARN-2761
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Reporter: Hong Zhiguo
>Assignee: Hong Zhiguo
>Priority: Minor
> Attachments: YARN-2761.patch
>
>
> reported by findbug.
> In SchedulingPolicy.getInstance, ConcurrentHashMap.get and
> ConcurrentHashMap.put is called. These two operations together should be
> atomic, but using ConcurrentHashMap doesn't guarantee this.
> {code}
> public static SchedulingPolicy getInstance(Class
> clazz) {
> SchedulingPolicy policy = instances.get(clazz);
> if (policy == null) {
> policy = ReflectionUtils.newInstance(clazz, null);
>instances.put(clazz, policy);
> }
> return policy;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2691) User level API support for priority label
[
https://issues.apache.org/jira/browse/YARN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525009#comment-14525009
]
Hadoop QA commented on YARN-2691:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12683391/YARN-2691.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7644/console |
This message was automatically generated.
> User level API support for priority label
> -
>
> Key: YARN-2691
> URL: https://issues.apache.org/jira/browse/YARN-2691
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: client
>Reporter: Sunil G
>Assignee: Rohith
> Attachments: YARN-2691.patch, YARN-2691.patch
>
>
> Support for handling Application-Priority label coming from client to
> ApplicationSubmissionContext.
> Common api support for user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2757) potential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
[
https://issues.apache.org/jira/browse/YARN-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525006#comment-14525006
]
Hadoop QA commented on YARN-2757:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677472/YARN-2757.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7643/console |
This message was automatically generated.
> potential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
> ---
>
> Key: YARN-2757
> URL: https://issues.apache.org/jira/browse/YARN-2757
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2757.000.patch
>
>
> pontential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
> since we check the nodeLabels null at
> {code}
> if (!str.trim().isEmpty()
> && (nodeLabels == null || !nodeLabels.contains(str.trim( {
> return false;
> }
> {code}
> We should also check nodeLabels null at
> {code}
> if (!nodeLabels.isEmpty()) {
> return false;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2513) Host framework UIs in YARN for use with the ATS
[
https://issues.apache.org/jira/browse/YARN-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14525017#comment-14525017
]
Hadoop QA commented on YARN-2513:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7647/console |
This message was automatically generated.
> Host framework UIs in YARN for use with the ATS
> ---
>
> Key: YARN-2513
> URL: https://issues.apache.org/jira/browse/YARN-2513
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: YARN-2513-v1.patch, YARN-2513-v2.patch
>
>
> Allow for pluggable UIs as described by TEZ-8. Yarn can provide the
> infrastructure to host java script and possible java UIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2513) Host framework UIs in YARN for use with the ATS
[
https://issues.apache.org/jira/browse/YARN-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524987#comment-14524987
]
Hadoop QA commented on YARN-2513:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12681180/YARN-2513-v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7638/console |
This message was automatically generated.
> Host framework UIs in YARN for use with the ATS
> ---
>
> Key: YARN-2513
> URL: https://issues.apache.org/jira/browse/YARN-2513
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
> Attachments: YARN-2513-v1.patch, YARN-2513-v2.patch
>
>
> Allow for pluggable UIs as described by TEZ-8. Yarn can provide the
> infrastructure to host java script and possible java UIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-221) NM should provide a way for AM to tell it not to aggregate logs.
[
https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524991#comment-14524991
]
Hadoop QA commented on YARN-221:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12637905/YARN-221-trunk-v2.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7640/console |
This message was automatically generated.
> NM should provide a way for AM to tell it not to aggregate logs.
>
>
> Key: YARN-221
> URL: https://issues.apache.org/jira/browse/YARN-221
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Robert Joseph Evans
>Assignee: Ming Ma
> Attachments: YARN-221-trunk-v1.patch, YARN-221-trunk-v2.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the
> logs should not be aggregated, that they should be aggregated with a high
> priority, or that they should be aggregated but with a lower priority. The
> AM should be able to do this in the ContainerLaunch context to provide a
> default value, but should also be able to update the value when the container
> is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid
> connection to the NN at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2913) Fair scheduler should have ability to set MaxResourceDefault for each queue
[
https://issues.apache.org/jira/browse/YARN-2913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524993#comment-14524993
]
Hadoop QA commented on YARN-2913:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684708/YARN-2913.v1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7642/console |
This message was automatically generated.
> Fair scheduler should have ability to set MaxResourceDefault for each queue
> ---
>
> Key: YARN-2913
> URL: https://issues.apache.org/jira/browse/YARN-2913
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: YARN-2913.v1.patch
>
>
> Queues that are created on the fly have the max resource of the entire
> cluster. Fair Scheduler should have a default maxResource to control the
> maxResource of those queues
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2691) User level API support for priority label
[
https://issues.apache.org/jira/browse/YARN-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524990#comment-14524990
]
Hadoop QA commented on YARN-2691:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12683391/YARN-2691.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7639/console |
This message was automatically generated.
> User level API support for priority label
> -
>
> Key: YARN-2691
> URL: https://issues.apache.org/jira/browse/YARN-2691
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: client
>Reporter: Sunil G
>Assignee: Rohith
> Attachments: YARN-2691.patch, YARN-2691.patch
>
>
> Support for handling Application-Priority label coming from client to
> ApplicationSubmissionContext.
> Common api support for user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2674) Distributed shell AM may re-launch containers if RM work preserving restart happens
[
https://issues.apache.org/jira/browse/YARN-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524992#comment-14524992
]
Hadoop QA commented on YARN-2674:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684681/YARN-2674.2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7641/console |
This message was automatically generated.
> Distributed shell AM may re-launch containers if RM work preserving restart
> happens
> ---
>
> Key: YARN-2674
> URL: https://issues.apache.org/jira/browse/YARN-2674
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Chun Chen
> Attachments: YARN-2674.1.patch, YARN-2674.2.patch
>
>
> Currently, if RM work preserving restart happens while distributed shell is
> running, distribute shell AM may re-launch all the containers, including
> new/running/complete. We must make sure it won't re-launch the
> running/complete containers.
> We need to remove allocated containers from
> AMRMClientImpl#remoteRequestsTable once AM receive them from RM.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1813) Better error message for "yarn logs" when permission denied
[
https://issues.apache.org/jira/browse/YARN-1813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524981#comment-14524981
]
Hadoop QA commented on YARN-1813:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677648/YARN-1813.6.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7634/console |
This message was automatically generated.
> Better error message for "yarn logs" when permission denied
> ---
>
> Key: YARN-1813
> URL: https://issues.apache.org/jira/browse/YARN-1813
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.3.0, 2.4.1, 2.5.1
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Minor
> Attachments: YARN-1813.1.patch, YARN-1813.2.patch, YARN-1813.2.patch,
> YARN-1813.3.patch, YARN-1813.4.patch, YARN-1813.5.patch, YARN-1813.6.patch
>
>
> I ran some MR jobs as the "hdfs" user, and then forgot to sudo -u when
> grabbing the logs. "yarn logs" prints an error message like the following:
> {noformat}
> [andrew.wang@a2402 ~]$ yarn logs -applicationId application_1394482121761_0010
> 14/03/10 16:05:10 INFO client.RMProxy: Connecting to ResourceManager at
> a2402.halxg.cloudera.com/10.20.212.10:8032
> Logs not available at
> /tmp/logs/andrew.wang/logs/application_1394482121761_0010
> Log aggregation has not completed or is not enabled.
> {noformat}
> It'd be nicer if it said "Permission denied" or "AccessControlException" or
> something like that instead, since that's the real issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2756) use static variable (Resources.none()) for not-running Node.resource in CommonNodeLabelsManager to save memory.
[
https://issues.apache.org/jira/browse/YARN-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524984#comment-14524984
]
Hadoop QA commented on YARN-2756:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677441/YARN-2756.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7636/console |
This message was automatically generated.
> use static variable (Resources.none()) for not-running Node.resource in
> CommonNodeLabelsManager to save memory.
> ---
>
> Key: YARN-2756
> URL: https://issues.apache.org/jira/browse/YARN-2756
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2756.000.patch
>
>
> use static variable (Resources.none()) for not-running Node.resource in
> CommonNodeLabelsManager to save memory. When a Node is not activated, the
> resource is never used, When a Node is activated, a new resource will be
> assigned to it in RMNodeLabelsManager#activateNode (nm.resource = resource)
> So it would be better to use static variable Resources.none() instead of
> allocating a new variable(Resource.newInstance(0, 0)) for each node
> deactivation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2757) potential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
[
https://issues.apache.org/jira/browse/YARN-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524982#comment-14524982
]
Hadoop QA commented on YARN-2757:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677472/YARN-2757.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7635/console |
This message was automatically generated.
> potential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
> ---
>
> Key: YARN-2757
> URL: https://issues.apache.org/jira/browse/YARN-2757
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2757.000.patch
>
>
> pontential NPE in checkNodeLabelExpression of SchedulerUtils for nodeLabels.
> since we check the nodeLabels null at
> {code}
> if (!str.trim().isEmpty()
> && (nodeLabels == null || !nodeLabels.contains(str.trim( {
> return false;
> }
> {code}
> We should also check nodeLabels null at
> {code}
> if (!nodeLabels.isEmpty()) {
> return false;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2761) potential race condition in SchedulingPolicy
[
https://issues.apache.org/jira/browse/YARN-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524985#comment-14524985
]
Hadoop QA commented on YARN-2761:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12677844/YARN-2761.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7637/console |
This message was automatically generated.
> potential race condition in SchedulingPolicy
>
>
> Key: YARN-2761
> URL: https://issues.apache.org/jira/browse/YARN-2761
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
>Reporter: Hong Zhiguo
>Assignee: Hong Zhiguo
>Priority: Minor
> Attachments: YARN-2761.patch
>
>
> reported by findbug.
> In SchedulingPolicy.getInstance, ConcurrentHashMap.get and
> ConcurrentHashMap.put is called. These two operations together should be
> atomic, but using ConcurrentHashMap doesn't guarantee this.
> {code}
> public static SchedulingPolicy getInstance(Class
> clazz) {
> SchedulingPolicy policy = instances.get(clazz);
> if (policy == null) {
> policy = ReflectionUtils.newInstance(clazz, null);
>instances.put(clazz, policy);
> }
> return policy;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2672) Improve Gridmix (synthetic generator + reservation support)
[
https://issues.apache.org/jira/browse/YARN-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524972#comment-14524972
]
Hadoop QA commented on YARN-2672:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12674061/YARN-2672.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7633/console |
This message was automatically generated.
> Improve Gridmix (synthetic generator + reservation support)
> ---
>
> Key: YARN-2672
> URL: https://issues.apache.org/jira/browse/YARN-2672
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, fairscheduler, resourcemanager
>Reporter: Carlo Curino
>Assignee: Carlo Curino
> Attachments: YARN-2672.patch
>
>
> This JIRA proposes an enhancement of Gridmix that contains:
> 1) a synthetic generator to produce load without the need of a trace, but
> based on distributions
> 2) include negotiation of reservations (to test YARN-1051).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1391) Lost node list should be identify by NodeId
[
https://issues.apache.org/jira/browse/YARN-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524966#comment-14524966
]
Hadoop QA commented on YARN-1391:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12672452/YARN-1391.v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7632/console |
This message was automatically generated.
> Lost node list should be identify by NodeId
> ---
>
> Key: YARN-1391
> URL: https://issues.apache.org/jira/browse/YARN-1391
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Affects Versions: 2.0.5-alpha
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: YARN-1391.v1.patch, YARN-1391.v2.patch
>
>
> in case of multiple node managers on a single machine. each of them should be
> identified by NodeId, which is more unique than just host name
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[
https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524962#comment-14524962
]
Hadoop QA commented on YARN-2554:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12670251/YARN-2554.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7631/console |
This message was automatically generated.
> Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
> -
>
> Key: YARN-2554
> URL: https://issues.apache.org/jira/browse/YARN-2554
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 2.6.0
>Reporter: Jonathan Maron
> Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch,
> YARN-2554.3.patch
>
>
> If the HTTP policy to enable HTTPS is specified, the RM and AM are
> initialized with SSL listeners. The RM has a web app proxy servlet that acts
> as a proxy for incoming AM requests. In order to forward the requests to the
> AM the proxy servlet makes use of HttpClient. However, the HttpClient
> utilized is not initialized correctly with the necessary certs to allow for
> successful one way SSL invocations to the other nodes in the cluster (it is
> not configured to access/load the client truststore specified in
> ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be
> utilized to create an instance that can be assigned to the HttpClient.
> The symptoms of this issue are:
> AM: Displays "unknown_certificate" exception
> RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1912) ResourceLocalizer started without any jvm memory control
[
https://issues.apache.org/jira/browse/YARN-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524947#comment-14524947
]
Hadoop QA commented on YARN-1912:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12642478/YARN-1912-1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7628/console |
This message was automatically generated.
> ResourceLocalizer started without any jvm memory control
>
>
> Key: YARN-1912
> URL: https://issues.apache.org/jira/browse/YARN-1912
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.2.0
>Reporter: stanley shi
> Attachments: YARN-1912-0.patch, YARN-1912-1.patch
>
>
> In the LinuxContainerExecutor.java#startLocalizer, it does not specify any
> "-Xmx" configurations in the command, this caused the ResourceLocalizer to be
> started with default memory setting.
> In an server-level hardware, it will use 25% of the system memory as the max
> heap size, this will cause memory issue in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3381) A typographical error in "InvalidStateTransitonException"
[ https://issues.apache.org/jira/browse/YARN-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524951#comment-14524951 ] Brahma Reddy Battula commented on YARN-3381: Thanks for taking a look into this issue..Attached patch which includes little more modifications..looking for jenkins result.. > A typographical error in "InvalidStateTransitonException" > - > > Key: YARN-3381 > URL: https://issues.apache.org/jira/browse/YARN-3381 > Project: Hadoop YARN > Issue Type: Improvement > Components: api >Affects Versions: 2.6.0 >Reporter: Xiaoshuang LU >Assignee: Brahma Reddy Battula > Attachments: YARN-3381-002.patch, YARN-3381-003.patch, YARN-3381.patch > > > Appears that "InvalidStateTransitonException" should be > "InvalidStateTransitionException". Transition was misspelled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2130) Cleanup: Adding getRMAppManager, getQueueACLsManager, getApplicationACLsManager to RMContext
[
https://issues.apache.org/jira/browse/YARN-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524946#comment-14524946
]
Hadoop QA commented on YARN-2130:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12666181/YARN-2130.9.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7627/console |
This message was automatically generated.
> Cleanup: Adding getRMAppManager, getQueueACLsManager,
> getApplicationACLsManager to RMContext
>
>
> Key: YARN-2130
> URL: https://issues.apache.org/jira/browse/YARN-2130
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Attachments: YARN-2130.1.patch, YARN-2130.2.patch, YARN-2130.3.patch,
> YARN-2130.4.patch, YARN-2130.5.patch, YARN-2130.6.patch, YARN-2130.7-2.patch,
> YARN-2130.7.patch, YARN-2130.8.patch, YARN-2130.8.patch, YARN-2130.9.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[
https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524950#comment-14524950
]
Hadoop QA commented on YARN-2046:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12666320/YARN-2046.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7629/console |
This message was automatically generated.
> Out of band heartbeats are sent only on container kill and possibly too early
> -
>
> Key: YARN-2046
> URL: https://issues.apache.org/jira/browse/YARN-2046
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.10, 2.4.0
>Reporter: Jason Lowe
>Assignee: Ming Ma
> Attachments: YARN-2046.patch
>
>
> [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM
> is currently sending out of band heartbeats only when stopContainer is
> called. In addition those heartbeats might be sent too early because the
> container kill event is asynchronously posted then the heartbeat monitor is
> notified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3381) A typographical error in "InvalidStateTransitonException"
[ https://issues.apache.org/jira/browse/YARN-3381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated YARN-3381: --- Attachment: YARN-3381-003.patch > A typographical error in "InvalidStateTransitonException" > - > > Key: YARN-3381 > URL: https://issues.apache.org/jira/browse/YARN-3381 > Project: Hadoop YARN > Issue Type: Improvement > Components: api >Affects Versions: 2.6.0 >Reporter: Xiaoshuang LU >Assignee: Brahma Reddy Battula > Attachments: YARN-3381-002.patch, YARN-3381-003.patch, YARN-3381.patch > > > Appears that "InvalidStateTransitonException" should be > "InvalidStateTransitionException". Transition was misspelled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1912) ResourceLocalizer started without any jvm memory control
[
https://issues.apache.org/jira/browse/YARN-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524927#comment-14524927
]
Hadoop QA commented on YARN-1912:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12642478/YARN-1912-1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7626/console |
This message was automatically generated.
> ResourceLocalizer started without any jvm memory control
>
>
> Key: YARN-1912
> URL: https://issues.apache.org/jira/browse/YARN-1912
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.2.0
>Reporter: stanley shi
> Attachments: YARN-1912-0.patch, YARN-1912-1.patch
>
>
> In the LinuxContainerExecutor.java#startLocalizer, it does not specify any
> "-Xmx" configurations in the command, this caused the ResourceLocalizer to be
> started with default memory setting.
> In an server-level hardware, it will use 25% of the system memory as the max
> heap size, this will cause memory issue in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2046) Out of band heartbeats are sent only on container kill and possibly too early
[
https://issues.apache.org/jira/browse/YARN-2046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524923#comment-14524923
]
Hadoop QA commented on YARN-2046:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12666320/YARN-2046.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7625/console |
This message was automatically generated.
> Out of band heartbeats are sent only on container kill and possibly too early
> -
>
> Key: YARN-2046
> URL: https://issues.apache.org/jira/browse/YARN-2046
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 0.23.10, 2.4.0
>Reporter: Jason Lowe
>Assignee: Ming Ma
> Attachments: YARN-2046.patch
>
>
> [~mingma] pointed out in the review discussion for MAPREDUCE-5465 that the NM
> is currently sending out of band heartbeats only when stopContainer is
> called. In addition those heartbeats might be sent too early because the
> container kill event is asynchronously posted then the heartbeat monitor is
> notified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2142) Add one service to check the nodes' TRUST status
[
https://issues.apache.org/jira/browse/YARN-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524919#comment-14524919
]
Hadoop QA commented on YARN-2142:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12654924/final.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7623/console |
This message was automatically generated.
> Add one service to check the nodes' TRUST status
> -
>
> Key: YARN-2142
> URL: https://issues.apache.org/jira/browse/YARN-2142
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, resourcemanager, scheduler, webapp
> Environment: OS:Ubuntu 13.04;
> JAVA:OpenJDK 7u51-2.4.4-0
> Only in branch-2.2.0.
>Reporter: anders
>Priority: Minor
> Labels: features
> Attachments: final.patch, trust.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> Because of critical computing environment ,we must test every node's TRUST
> status in the cluster (We can get the TRUST status by the API of OAT
> sever),So I add this feature into hadoop's schedule .
> By the TRUST check service ,node can get the TRUST status of itself,
> then through the heartbeat ,send the TRUST status to resource manager for
> scheduling.
> In the scheduling step,if the node's TRUST status is 'false', it will be
> abandoned until it's TRUST status turn to 'true'.
> ***The logic of this feature is similar to node's health checkservice.
> ***Only in branch-2.2.0 , not in trunk***
> OAT wiki link:
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-641) Make AMLauncher in RM Use NMClient
[
https://issues.apache.org/jira/browse/YARN-641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524915#comment-14524915
]
Hadoop QA commented on YARN-641:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12587395/YARN-641.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7622/console |
This message was automatically generated.
> Make AMLauncher in RM Use NMClient
> --
>
> Key: YARN-641
> URL: https://issues.apache.org/jira/browse/YARN-641
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-641.1.patch, YARN-641.2.patch, YARN-641.3.patch
>
>
> YARN-422 adds NMClient. RM's AMLauncher is responsible for the interactions
> with an application's AM container. AMLauncher should also replace the raw
> ContainerManager proxy with NMClient.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2130) Cleanup: Adding getRMAppManager, getQueueACLsManager, getApplicationACLsManager to RMContext
[
https://issues.apache.org/jira/browse/YARN-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524921#comment-14524921
]
Hadoop QA commented on YARN-2130:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12666181/YARN-2130.9.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7624/console |
This message was automatically generated.
> Cleanup: Adding getRMAppManager, getQueueACLsManager,
> getApplicationACLsManager to RMContext
>
>
> Key: YARN-2130
> URL: https://issues.apache.org/jira/browse/YARN-2130
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
>Reporter: Tsuyoshi Ozawa
>Assignee: Tsuyoshi Ozawa
> Attachments: YARN-2130.1.patch, YARN-2130.2.patch, YARN-2130.3.patch,
> YARN-2130.4.patch, YARN-2130.5.patch, YARN-2130.6.patch, YARN-2130.7-2.patch,
> YARN-2130.7.patch, YARN-2130.8.patch, YARN-2130.8.patch, YARN-2130.9.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3534) Collect node resource utilization
[
https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524886#comment-14524886
]
Hadoop QA commented on YARN-3534:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 39s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 30s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 28s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 23s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 30s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 5m 51s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 44m 26s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729911/YARN-3534-8.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7594/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7594/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7594/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7594/console |
This message was automatically generated.
> Collect node resource utilization
> -
>
> Key: YARN-3534
> URL: https://issues.apache.org/jira/browse/YARN-3534
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Affects Versions: 2.7.0
>Reporter: Inigo Goiri
>Assignee: Inigo Goiri
> Attachments: YARN-3534-1.patch, YARN-3534-2.patch, YARN-3534-3.patch,
> YARN-3534-3.patch, YARN-3534-4.patch, YARN-3534-5.patch, YARN-3534-6.patch,
> YARN-3534-7.patch, YARN-3534-8.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> YARN should be aware of the resource utilization of the nodes when scheduling
> containers. For this, this task will implement the NodeResourceMonitor and
> send this information to the Resource Manager in the heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2151) FairScheduler option for global preemption within hierarchical queues
[
https://issues.apache.org/jira/browse/YARN-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524897#comment-14524897
]
Hadoop QA commented on YARN-2151:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12649887/YARN-2151.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7620/console |
This message was automatically generated.
> FairScheduler option for global preemption within hierarchical queues
> -
>
> Key: YARN-2151
> URL: https://issues.apache.org/jira/browse/YARN-2151
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Reporter: Andrey Stepachev
> Attachments: YARN-2151.patch
>
>
> FairScheduler has hierarchical queues, but fair share calculation and
> preemption still works withing a limited range and effectively still
> nonhierarchical.
> This patch solves this incompleteness in two aspects:
> 1. Currently MinShare is not propagated to upper queue, that leads to
> fair share calculation ignores all Min Shares in deeper queues.
> Lets take an example
> (implemented as test case TestFairScheduler#testMinShareInHierarchicalQueues)
> {code}
>
>
>
> 10240mb, 10vcores
>
>
> fair
>
> 6192mb, 6vcores
>
>
>
>
>
>
> {code}
> Then bigApp started within queue1.big with 10x1GB containers.
> That effectively eats all maximum allowed resources for queue1.
> Subsequent requests for app1 (queue1.sub1.sub11) and
> app2 (queue1.sub2) (5x1GB each) will wait for free resources.
> Take a note, that sub11 has min share requirements for 6x1GB.
> Without given patch fair share will be calculated with no knowledge
> about min share requirements and app1 and app2 will get equal
> number of containers.
> With the patch resources will split according to min share ( in test
> it will be 5 for app1 and 1 for app2)
> That behaviour controlled by the same parameter as ‘globalPreemtion’,
> but that can be changed easily.
> Implementation is a bit awkward, but seems that method for min share
> recalculation can be exposed as public or protected api and constructor
> in FSQueue can call it before using minShare getter. But right now
> current implementation with nulls should work too.
> 2. Preemption doesn’t works between queues on different level for the
> queues hierarchy. Moreover, it is not possible to override various
> parameters for children queues.
> This patch adds parameter ‘globalPreemption’, which enables global
> preemption algorithm modifications.
> In a nutshell patch adds function shouldAttemptPreemption(queue),
> which can calculate usage for nested queues, and if queue with usage more
> that specified threshold is found, preemption can be triggered.
> Aggregated minShare does the rest of work and preemption will work
> as expected within hierarchy of queues with different MinShare/MaxShare
> specifications on different levels.
> Test case TestFairScheduler#testGlobalPreemption depicts how it works.
> One big app gets resources above its fair share and app1 has a declared
> min share. On submission code finds that starvation and preempts enough
> containers to give enough room for app1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler
[
https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524887#comment-14524887
]
Hadoop QA commented on YARN-2325:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12656795/YARN-2325.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7619/console |
This message was automatically generated.
> need check whether node is null in nodeUpdate for FairScheduler
>
>
> Key: YARN-2325
> URL: https://issues.apache.org/jira/browse/YARN-2325
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2325.000.patch
>
>
> need check whether node is null in nodeUpdate for FairScheduler.
> If nodeUpdate is called after removeNode, the getFSSchedulerNode will be
> null. If the node is null, we should return with error message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1515) Provide ContainerManagementProtocol#signalContainer processing a batch of signals
[
https://issues.apache.org/jira/browse/YARN-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524905#comment-14524905
]
Hadoop QA commented on YARN-1515:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12645519/YARN-1515.v08.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7621/console |
This message was automatically generated.
> Provide ContainerManagementProtocol#signalContainer processing a batch of
> signals
> --
>
> Key: YARN-1515
> URL: https://issues.apache.org/jira/browse/YARN-1515
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, nodemanager
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Attachments: YARN-1515.v01.patch, YARN-1515.v02.patch,
> YARN-1515.v03.patch, YARN-1515.v04.patch, YARN-1515.v05.patch,
> YARN-1515.v06.patch, YARN-1515.v07.patch, YARN-1515.v08.patch
>
>
> This is needed to implement MAPREDUCE-5044 to enable thread diagnostics for
> timed-out task attempts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1515) Provide ContainerManagementProtocol#signalContainer processing a batch of signals
[
https://issues.apache.org/jira/browse/YARN-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524856#comment-14524856
]
Hadoop QA commented on YARN-1515:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12645519/YARN-1515.v08.patch |
| Optional Tests | javac unit findbugs checkstyle javadoc |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7615/console |
This message was automatically generated.
> Provide ContainerManagementProtocol#signalContainer processing a batch of
> signals
> --
>
> Key: YARN-1515
> URL: https://issues.apache.org/jira/browse/YARN-1515
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: api, nodemanager
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Attachments: YARN-1515.v01.patch, YARN-1515.v02.patch,
> YARN-1515.v03.patch, YARN-1515.v04.patch, YARN-1515.v05.patch,
> YARN-1515.v06.patch, YARN-1515.v07.patch, YARN-1515.v08.patch
>
>
> This is needed to implement MAPREDUCE-5044 to enable thread diagnostics for
> timed-out task attempts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-641) Make AMLauncher in RM Use NMClient
[
https://issues.apache.org/jira/browse/YARN-641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524867#comment-14524867
]
Hadoop QA commented on YARN-641:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12587395/YARN-641.3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7617/console |
This message was automatically generated.
> Make AMLauncher in RM Use NMClient
> --
>
> Key: YARN-641
> URL: https://issues.apache.org/jira/browse/YARN-641
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Zhijie Shen
>Assignee: Zhijie Shen
> Attachments: YARN-641.1.patch, YARN-641.2.patch, YARN-641.3.patch
>
>
> YARN-422 adds NMClient. RM's AMLauncher is responsible for the interactions
> with an application's AM container. AMLauncher should also replace the raw
> ContainerManager proxy with NMClient.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2325) need check whether node is null in nodeUpdate for FairScheduler
[
https://issues.apache.org/jira/browse/YARN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524869#comment-14524869
]
Hadoop QA commented on YARN-2325:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12656795/YARN-2325.000.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7618/console |
This message was automatically generated.
> need check whether node is null in nodeUpdate for FairScheduler
>
>
> Key: YARN-2325
> URL: https://issues.apache.org/jira/browse/YARN-2325
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: YARN-2325.000.patch
>
>
> need check whether node is null in nodeUpdate for FairScheduler.
> If nodeUpdate is called after removeNode, the getFSSchedulerNode will be
> null. If the node is null, we should return with error message.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2142) Add one service to check the nodes' TRUST status
[
https://issues.apache.org/jira/browse/YARN-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524861#comment-14524861
]
Hadoop QA commented on YARN-2142:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12654924/final.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7616/console |
This message was automatically generated.
> Add one service to check the nodes' TRUST status
> -
>
> Key: YARN-2142
> URL: https://issues.apache.org/jira/browse/YARN-2142
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager, resourcemanager, scheduler, webapp
> Environment: OS:Ubuntu 13.04;
> JAVA:OpenJDK 7u51-2.4.4-0
> Only in branch-2.2.0.
>Reporter: anders
>Priority: Minor
> Labels: features
> Attachments: final.patch, trust.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> Because of critical computing environment ,we must test every node's TRUST
> status in the cluster (We can get the TRUST status by the API of OAT
> sever),So I add this feature into hadoop's schedule .
> By the TRUST check service ,node can get the TRUST status of itself,
> then through the heartbeat ,send the TRUST status to resource manager for
> scheduling.
> In the scheduling step,if the node's TRUST status is 'false', it will be
> abandoned until it's TRUST status turn to 'true'.
> ***The logic of this feature is similar to node's health checkservice.
> ***Only in branch-2.2.0 , not in trunk***
> OAT wiki link:
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2151) FairScheduler option for global preemption within hierarchical queues
[
https://issues.apache.org/jira/browse/YARN-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524855#comment-14524855
]
Hadoop QA commented on YARN-2151:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12649887/YARN-2151.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7614/console |
This message was automatically generated.
> FairScheduler option for global preemption within hierarchical queues
> -
>
> Key: YARN-2151
> URL: https://issues.apache.org/jira/browse/YARN-2151
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
>Reporter: Andrey Stepachev
> Attachments: YARN-2151.patch
>
>
> FairScheduler has hierarchical queues, but fair share calculation and
> preemption still works withing a limited range and effectively still
> nonhierarchical.
> This patch solves this incompleteness in two aspects:
> 1. Currently MinShare is not propagated to upper queue, that leads to
> fair share calculation ignores all Min Shares in deeper queues.
> Lets take an example
> (implemented as test case TestFairScheduler#testMinShareInHierarchicalQueues)
> {code}
>
>
>
> 10240mb, 10vcores
>
>
> fair
>
> 6192mb, 6vcores
>
>
>
>
>
>
> {code}
> Then bigApp started within queue1.big with 10x1GB containers.
> That effectively eats all maximum allowed resources for queue1.
> Subsequent requests for app1 (queue1.sub1.sub11) and
> app2 (queue1.sub2) (5x1GB each) will wait for free resources.
> Take a note, that sub11 has min share requirements for 6x1GB.
> Without given patch fair share will be calculated with no knowledge
> about min share requirements and app1 and app2 will get equal
> number of containers.
> With the patch resources will split according to min share ( in test
> it will be 5 for app1 and 1 for app2)
> That behaviour controlled by the same parameter as ‘globalPreemtion’,
> but that can be changed easily.
> Implementation is a bit awkward, but seems that method for min share
> recalculation can be exposed as public or protected api and constructor
> in FSQueue can call it before using minShare getter. But right now
> current implementation with nulls should work too.
> 2. Preemption doesn’t works between queues on different level for the
> queues hierarchy. Moreover, it is not possible to override various
> parameters for children queues.
> This patch adds parameter ‘globalPreemption’, which enables global
> preemption algorithm modifications.
> In a nutshell patch adds function shouldAttemptPreemption(queue),
> which can calculate usage for nested queues, and if queue with usage more
> that specified threshold is found, preemption can be triggered.
> Aggregated minShare does the rest of work and preemption will work
> as expected within hierarchy of queues with different MinShare/MaxShare
> specifications on different levels.
> Test case TestFairScheduler#testGlobalPreemption depicts how it works.
> One big app gets resources above its fair share and app1 has a declared
> min share. On submission code finds that starvation and preempts enough
> containers to give enough room for app1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT
[
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524847#comment-14524847
]
Hadoop QA commented on YARN-126:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12580129/YARN-126.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7613/console |
This message was automatically generated.
> yarn rmadmin help message contains reference to hadoop cli and JT
> -
>
> Key: YARN-126
> URL: https://issues.apache.org/jira/browse/YARN-126
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 2.0.3-alpha
>Reporter: Thomas Graves
>Assignee: Rémy SAISSY
> Labels: usability
> Attachments: YARN-126.patch
>
>
> has option to specify a job tracker and the last line for general command
> line syntax had "bin/hadoop command [genericOptions] [commandOptions]"
> ran "yarn rmadmin" to get usage:
> RMAdmin
> Usage: java RMAdmin
>[-refreshQueues]
>[-refreshNodes]
>[-refreshUserToGroupsMappings]
>[-refreshSuperUserGroupsConfiguration]
>[-refreshAdminAcls]
>[-refreshServiceAcl]
>[-help [cmd]]
> Generic options supported are
> -conf specify an application configuration file
> -D use value for given property
> -fs specify a namenode
> -jt specify a job tracker
> -files specify comma separated files to be
> copied to the map reduce cluster
> -libjars specify comma separated jar files
> to include in the classpath.
> -archives specify comma separated
> archives to be unarchived on the compute machines.
> The general command line syntax is
> bin/hadoop command [genericOptions] [commandOptions]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1287) Consolidate MockClocks
[
https://issues.apache.org/jira/browse/YARN-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524835#comment-14524835
]
Hadoop QA commented on YARN-1287:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12621781/YARN-1287-3.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7611/console |
This message was automatically generated.
> Consolidate MockClocks
> --
>
> Key: YARN-1287
> URL: https://issues.apache.org/jira/browse/YARN-1287
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Sandy Ryza
>Assignee: Sebastian Wong
> Labels: newbie
> Attachments: YARN-1287-3.patch
>
>
> A bunch of different tests have near-identical implementations of MockClock.
> TestFairScheduler, TestFSSchedulerApp, and TestCgroupsLCEResourcesHandler for
> example. They should be consolidated into a single MockClock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1782) CLI should let users to query cluster metrics
[
https://issues.apache.org/jira/browse/YARN-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524840#comment-14524840
]
Hadoop QA commented on YARN-1782:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12650527/YARN-1782.patch |
| Optional Tests | shellcheck javadoc javac unit findbugs checkstyle site |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7612/console |
This message was automatically generated.
> CLI should let users to query cluster metrics
> -
>
> Key: YARN-1782
> URL: https://issues.apache.org/jira/browse/YARN-1782
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Zhijie Shen
>Assignee: Kenji Kikushima
> Attachments: YARN-1782.patch
>
>
> Like RM webUI and RESTful services, YARN CLI should also enable users to
> query the cluster metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2120) Coloring queues running over minShare on RM Scheduler page
[
https://issues.apache.org/jira/browse/YARN-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524817#comment-14524817
]
Hadoop QA commented on YARN-2120:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12648681/YARN-2120.v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7610/console |
This message was automatically generated.
> Coloring queues running over minShare on RM Scheduler page
> --
>
> Key: YARN-2120
> URL: https://issues.apache.org/jira/browse/YARN-2120
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: 76AD6A72-9A0D-4F3A-A7B8-6EC1DCBD543A.png,
> YARN-2120.v1.patch, YARN-2120.v2.patch
>
>
> Today RM Scheduler page shows FairShare, Used, Used (over fair share) and
> MaxCapacity.
> Since fairShare is displaying with dotted line, I think we can stop
> displaying orange when a queue over its fairshare.
> It would be better to show a queue running over minShare with orange color,
> so that we know queue is running more than its min share.
> Also, we can display a queue running at maxShare with red color.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager
[
https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524810#comment-14524810
]
Hadoop QA commented on YARN-1805:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12643371/YARN-1805.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7609/console |
This message was automatically generated.
> Signal container request delivery from resourcemanager to nodemanager
> -
>
> Key: YARN-1805
> URL: https://issues.apache.org/jira/browse/YARN-1805
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Ming Ma
>Assignee: Ming Ma
> Attachments: YARN-1805.patch
>
>
> 1. Update ResourceTracker's HeartbeatResponse to include the list of
> SignalContainerRequest.
> 2. Upon receiving the request, NM's NodeStatusUpdater will deliver the
> request to ContainerManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1427) yarn-env.cmd should have the analog comments that are in yarn-env.sh
[
https://issues.apache.org/jira/browse/YARN-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524802#comment-14524802
]
Hadoop QA commented on YARN-1427:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12646616/YARN-1427.1.patch |
| Optional Tests | |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7607/console |
This message was automatically generated.
> yarn-env.cmd should have the analog comments that are in yarn-env.sh
>
>
> Key: YARN-1427
> URL: https://issues.apache.org/jira/browse/YARN-1427
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Zhijie Shen
> Labels: newbie, windows
> Attachments: YARN-1427.1.patch
>
>
> There're the paragraphs of about RM/NM env vars (probably AHS as well soon)
> in yarn-env.sh. Should the windows version script provide the similar
> comments?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-445) Ability to signal containers
[
https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524805#comment-14524805
]
Hadoop QA commented on YARN-445:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12633748/YARN-445-signal-container-via-rm.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7608/console |
This message was automatically generated.
> Ability to signal containers
>
>
> Key: YARN-445
> URL: https://issues.apache.org/jira/browse/YARN-445
> Project: Hadoop YARN
> Issue Type: Task
> Components: nodemanager
>Reporter: Jason Lowe
>Assignee: Andrey Klochkov
> Attachments: MRJob.png, MRTasks.png, YARN-445--n2.patch,
> YARN-445--n3.patch, YARN-445--n4.patch,
> YARN-445-signal-container-via-rm.patch, YARN-445.patch, YARNContainers.png
>
>
> It would be nice if an ApplicationMaster could send signals to contaniers
> such as SIGQUIT, SIGUSR1, etc.
> For example, in order to replicate the jstack-on-task-timeout feature
> implemented by MAPREDUCE-1119 in Hadoop 0.21 the NodeManager needs an
> interface for sending SIGQUIT to a container. For that specific feature we
> could implement it as an additional field in the StopContainerRequest.
> However that would not address other potential features like the ability for
> an AM to trigger jstacks on arbitrary tasks *without* killing them. The
> latter feature would be a very useful debugging tool for users who do not
> have shell access to the nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1803) Signal container support in nodemanager
[
https://issues.apache.org/jira/browse/YARN-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524800#comment-14524800
]
Hadoop QA commented on YARN-1803:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12643173/YARN-1803.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7606/console |
This message was automatically generated.
> Signal container support in nodemanager
> ---
>
> Key: YARN-1803
> URL: https://issues.apache.org/jira/browse/YARN-1803
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Ming Ma
>Assignee: Ming Ma
> Attachments: YARN-1803.patch
>
>
> It could include the followings.
> 1. ContainerManager is able to process a new event type
> ContainerManagerEventType.SIGNAL_CONTAINERS coming from NodeStatusUpdater and
> deliver the request to ContainerExecutor.
> 2. Translate the platform independent signal command to Linux specific
> signals. Windows support will be tracked by another task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1427) yarn-env.cmd should have the analog comments that are in yarn-env.sh
[
https://issues.apache.org/jira/browse/YARN-1427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524785#comment-14524785
]
Hadoop QA commented on YARN-1427:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12646616/YARN-1427.1.patch |
| Optional Tests | |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7603/console |
This message was automatically generated.
> yarn-env.cmd should have the analog comments that are in yarn-env.sh
>
>
> Key: YARN-1427
> URL: https://issues.apache.org/jira/browse/YARN-1427
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Zhijie Shen
> Labels: newbie, windows
> Attachments: YARN-1427.1.patch
>
>
> There're the paragraphs of about RM/NM env vars (probably AHS as well soon)
> in yarn-env.sh. Should the windows version script provide the similar
> comments?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1782) CLI should let users to query cluster metrics
[
https://issues.apache.org/jira/browse/YARN-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524792#comment-14524792
]
Hadoop QA commented on YARN-1782:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12650527/YARN-1782.patch |
| Optional Tests | shellcheck javadoc javac unit findbugs checkstyle site |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7605/console |
This message was automatically generated.
> CLI should let users to query cluster metrics
> -
>
> Key: YARN-1782
> URL: https://issues.apache.org/jira/browse/YARN-1782
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.4.0
>Reporter: Zhijie Shen
>Assignee: Kenji Kikushima
> Attachments: YARN-1782.patch
>
>
> Like RM webUI and RESTful services, YARN CLI should also enable users to
> query the cluster metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2120) Coloring queues running over minShare on RM Scheduler page
[
https://issues.apache.org/jira/browse/YARN-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524790#comment-14524790
]
Hadoop QA commented on YARN-2120:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12648681/YARN-2120.v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7604/console |
This message was automatically generated.
> Coloring queues running over minShare on RM Scheduler page
> --
>
> Key: YARN-2120
> URL: https://issues.apache.org/jira/browse/YARN-2120
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.3.0
>Reporter: Siqi Li
>Assignee: Siqi Li
> Attachments: 76AD6A72-9A0D-4F3A-A7B8-6EC1DCBD543A.png,
> YARN-2120.v1.patch, YARN-2120.v2.patch
>
>
> Today RM Scheduler page shows FairShare, Used, Used (over fair share) and
> MaxCapacity.
> Since fairShare is displaying with dotted line, I think we can stop
> displaying orange when a queue over its fairshare.
> It would be better to show a queue running over minShare with orange color,
> so that we know queue is running more than its min share.
> Also, we can display a queue running at maxShare with red color.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1803) Signal container support in nodemanager
[
https://issues.apache.org/jira/browse/YARN-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524784#comment-14524784
]
Hadoop QA commented on YARN-1803:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12643173/YARN-1803.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7602/console |
This message was automatically generated.
> Signal container support in nodemanager
> ---
>
> Key: YARN-1803
> URL: https://issues.apache.org/jira/browse/YARN-1803
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
>Reporter: Ming Ma
>Assignee: Ming Ma
> Attachments: YARN-1803.patch
>
>
> It could include the followings.
> 1. ContainerManager is able to process a new event type
> ContainerManagerEventType.SIGNAL_CONTAINERS coming from NodeStatusUpdater and
> deliver the request to ContainerExecutor.
> 2. Translate the platform independent signal command to Linux specific
> signals. Windows support will be tracked by another task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1805) Signal container request delivery from resourcemanager to nodemanager
[
https://issues.apache.org/jira/browse/YARN-1805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524779#comment-14524779
]
Hadoop QA commented on YARN-1805:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12643371/YARN-1805.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7600/console |
This message was automatically generated.
> Signal container request delivery from resourcemanager to nodemanager
> -
>
> Key: YARN-1805
> URL: https://issues.apache.org/jira/browse/YARN-1805
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Reporter: Ming Ma
>Assignee: Ming Ma
> Attachments: YARN-1805.patch
>
>
> 1. Update ResourceTracker's HeartbeatResponse to include the list of
> SignalContainerRequest.
> 2. Upon receiving the request, NM's NodeStatusUpdater will deliver the
> request to ContainerManager.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-445) Ability to signal containers
[
https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524782#comment-14524782
]
Hadoop QA commented on YARN-445:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 1s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12633748/YARN-445-signal-container-via-rm.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7601/console |
This message was automatically generated.
> Ability to signal containers
>
>
> Key: YARN-445
> URL: https://issues.apache.org/jira/browse/YARN-445
> Project: Hadoop YARN
> Issue Type: Task
> Components: nodemanager
>Reporter: Jason Lowe
>Assignee: Andrey Klochkov
> Attachments: MRJob.png, MRTasks.png, YARN-445--n2.patch,
> YARN-445--n3.patch, YARN-445--n4.patch,
> YARN-445-signal-container-via-rm.patch, YARN-445.patch, YARNContainers.png
>
>
> It would be nice if an ApplicationMaster could send signals to contaniers
> such as SIGQUIT, SIGUSR1, etc.
> For example, in order to replicate the jstack-on-task-timeout feature
> implemented by MAPREDUCE-1119 in Hadoop 0.21 the NodeManager needs an
> interface for sending SIGQUIT to a container. For that specific feature we
> could implement it as an additional field in the StopContainerRequest.
> However that would not address other potential features like the ability for
> an AM to trigger jstacks on arbitrary tasks *without* killing them. The
> latter feature would be a very useful debugging tool for users who do not
> have shell access to the nodes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-126) yarn rmadmin help message contains reference to hadoop cli and JT
[
https://issues.apache.org/jira/browse/YARN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524776#comment-14524776
]
Hadoop QA commented on YARN-126:
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12580129/YARN-126.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7599/console |
This message was automatically generated.
> yarn rmadmin help message contains reference to hadoop cli and JT
> -
>
> Key: YARN-126
> URL: https://issues.apache.org/jira/browse/YARN-126
> Project: Hadoop YARN
> Issue Type: Bug
> Components: client
>Affects Versions: 2.0.3-alpha
>Reporter: Thomas Graves
>Assignee: Rémy SAISSY
> Labels: usability
> Attachments: YARN-126.patch
>
>
> has option to specify a job tracker and the last line for general command
> line syntax had "bin/hadoop command [genericOptions] [commandOptions]"
> ran "yarn rmadmin" to get usage:
> RMAdmin
> Usage: java RMAdmin
>[-refreshQueues]
>[-refreshNodes]
>[-refreshUserToGroupsMappings]
>[-refreshSuperUserGroupsConfiguration]
>[-refreshAdminAcls]
>[-refreshServiceAcl]
>[-help [cmd]]
> Generic options supported are
> -conf specify an application configuration file
> -D use value for given property
> -fs specify a namenode
> -jt specify a job tracker
> -files specify comma separated files to be
> copied to the map reduce cluster
> -libjars specify comma separated jar files
> to include in the classpath.
> -archives specify comma separated
> archives to be unarchived on the compute machines.
> The general command line syntax is
> bin/hadoop command [genericOptions] [commandOptions]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1517) AMFilterInitializer with configurable AMIpFilter
[
https://issues.apache.org/jira/browse/YARN-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524741#comment-14524741
]
Hadoop QA commented on YARN-1517:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12619611/AmFilterInitializer.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7597/console |
This message was automatically generated.
> AMFilterInitializer with configurable AMIpFilter
>
>
> Key: YARN-1517
> URL: https://issues.apache.org/jira/browse/YARN-1517
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.2.0
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
> Attachments: AmFilterInitializer.patch
>
>
> We need to implement custom logic in a filter for our webservice similar to
> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter and it would be
> convenient if we extended AmIpFilter as the proxy locations already
> available.
> We would need to specify a filter initializer for this filter. The
> initializer would be same as AmFilterInitializer except that it would add our
> filter instead of AmIpFilter and it would be better if we could reuse
> AmlFilterInitializer. Can AmFilterInitializer be updated to specify a filter
> name and filter class.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1287) Consolidate MockClocks
[
https://issues.apache.org/jira/browse/YARN-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524747#comment-14524747
]
Hadoop QA commented on YARN-1287:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12621781/YARN-1287-3.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7598/console |
This message was automatically generated.
> Consolidate MockClocks
> --
>
> Key: YARN-1287
> URL: https://issues.apache.org/jira/browse/YARN-1287
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Sandy Ryza
>Assignee: Sebastian Wong
> Labels: newbie
> Attachments: YARN-1287-3.patch
>
>
> A bunch of different tests have near-identical implementations of MockClock.
> TestFairScheduler, TestFSSchedulerApp, and TestCgroupsLCEResourcesHandler for
> example. They should be consolidated into a single MockClock.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1019) YarnConfiguration validation for local disk path and http addresses.
[
https://issues.apache.org/jira/browse/YARN-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524736#comment-14524736
]
Hadoop QA commented on YARN-1019:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12616524/YARN-1019.0.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7596/console |
This message was automatically generated.
> YarnConfiguration validation for local disk path and http addresses.
>
>
> Key: YARN-1019
> URL: https://issues.apache.org/jira/browse/YARN-1019
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.0.5-alpha
>Reporter: Omkar Vinit Joshi
>Priority: Minor
> Labels: newbie
> Attachments: YARN-1019.0.patch
>
>
> Today we are not validating certain configuration parameters set in
> yarn-site.xml. 1) Configurations related to paths... such as local-dirs,
> log-dirs.. Our NM crashes during startup if they are set to relative paths
> rather than absolute paths. To avoid such failures we can enforce checks
> (absolute paths) before startup . i.e. before we actually startup...( i.e.
> directory handler creating directories).
> 2) Also for all the parameters using "hostname:port" unless we are ok with
> default port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1329) yarn-config.sh overwrites YARN_CONF_DIR indiscriminately
[
https://issues.apache.org/jira/browse/YARN-1329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524730#comment-14524730
]
Hadoop QA commented on YARN-1329:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12613087/YARN-1329.patch |
| Optional Tests | shellcheck |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7595/console |
This message was automatically generated.
> yarn-config.sh overwrites YARN_CONF_DIR indiscriminately
> -
>
> Key: YARN-1329
> URL: https://issues.apache.org/jira/browse/YARN-1329
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager, resourcemanager
>Reporter: Aaron Gottlieb
>Assignee: haosdent
> Labels: easyfix
> Attachments: YARN-1329.patch
>
>
> The script yarn-daemons.sh calls
> {code}${HADOOP_LIBEXEC_DIR}/yarn-config.sh{code}
> yarn-config.sh overwrites any previously set value of environment variable
> YARN_CONF_DIR starting at line 40:
> {code:title=yarn-config.sh|borderStyle=solid}
> #check to see if the conf dir is given as an optional argument
> if [ $# -gt 1 ]
> then
> if [ "--config" = "$1" ]
> then
> shift
> confdir=$1
> shift
> YARN_CONF_DIR=$confdir
> fi
> fi
>
> # Allow alternate conf dir location.
> export YARN_CONF_DIR="${HADOOP_CONF_DIR:-$HADOOP_YARN_HOME/conf}"
> {code}
> The last line should check for the existence of YARN_CONF_DIR first.
> {code}
> DEFAULT_CONF_DIR="${HADOOP_CONF_DIR:-$YARN_HOME/conf}"
> export YARN_CONF_DIR="${YARN_CONF_DIR:-$DEFAULT_CONF_DIR}"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1772) Fair Scheduler documentation should indicate that admin ACLs also give submit permissions
[ https://issues.apache.org/jira/browse/YARN-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524523#comment-14524523 ] Naren Koneru commented on YARN-1772: Hi Jian, I won't be able to.. Feel free to take it.. > Fair Scheduler documentation should indicate that admin ACLs also give submit > permissions > - > > Key: YARN-1772 > URL: https://issues.apache.org/jira/browse/YARN-1772 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Reporter: Sandy Ryza >Assignee: Naren Koneru > > I can submit to a Fair Scheduler queue if I'm in the submit ACL OR if I'm in > the administer ACL. The Fair Scheduler docs seem to leave out the second > part. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2137) Add support for logaggregation to a path on non-default filecontext
[ https://issues.apache.org/jira/browse/YARN-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524511#comment-14524511 ] Sumit Kumar commented on YARN-2137: --- Apologies for the delay, i will rebase this patch and look into the required testing that [~vinodkv] recommended. > Add support for logaggregation to a path on non-default filecontext > --- > > Key: YARN-2137 > URL: https://issues.apache.org/jira/browse/YARN-2137 > Project: Hadoop YARN > Issue Type: Improvement > Components: log-aggregation >Affects Versions: 2.4.0 >Reporter: Sumit Kumar >Assignee: Sumit Kumar > Attachments: YARN-2137.patch > > > Current log-aggregation implementation supports logaggregation to default > filecontext only. This patch is to support logaggregation to any of the > supported filesystems within hadoop eco-system (hdfs, s3, swiftfs etc). So > for example a customer could use hdfs as default filesystem but use s3 or > swiftfs for logaggregation. Current implementation makes mixed usages of > FileContext+AbstractFileSystem apis as well as FileSystem apis which is > confusing. > This patch does two things: > # moves logaggregation implementation to use only FileContext apis > # adds support for doing log aggregation on non-default filesystem as well. > # changes TestLogAggregationService to use local filesystem itself instead of > mocking the behavior -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-1917) Add "waitForApplicationState" interface to YarnClient
[
https://issues.apache.org/jira/browse/YARN-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524493#comment-14524493
]
Hadoop QA commented on YARN-1917:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 58s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 2 new or modified test files. |
| {color:green}+1{color} | javac | 7m 44s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 53s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 44s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 29s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | mapreduce tests | 73m 26s | Tests failed in
hadoop-mapreduce-client-jobclient. |
| {color:red}-1{color} | yarn tests | 6m 57s | Tests failed in
hadoop-yarn-client. |
| {color:green}+1{color} | yarn tests | 6m 8s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 125m 51s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.mapred.TestMapRed |
| | hadoop.yarn.client.api.impl.TestYarnClient |
| Timed out tests | org.apache.hadoop.mapred.TestMiniMRClasspath |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729871/YARN-1917.20150501.1.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6f541ed |
| hadoop-mapreduce-client-jobclient test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7590/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
|
| hadoop-yarn-client test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7590/artifact/patchprocess/testrun_hadoop-yarn-client.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7590/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7590/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7590/console |
This message was automatically generated.
> Add "waitForApplicationState" interface to YarnClient
> -
>
> Key: YARN-1917
> URL: https://issues.apache.org/jira/browse/YARN-1917
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: client
>Affects Versions: 2.4.0
>Reporter: Wangda Tan
>Assignee: Wangda Tan
> Attachments: YARN-1917.20150501.1.patch, YARN-1917.patch,
> YARN-1917.patch, YARN-1917.patch
>
>
> Currently, YARN dosen't have this method. Users needs to write
> implementations like UnmanagedAMLauncher.monitorApplication or
> mapreduce.Job.monitorAndPrintJob on their own. This feature should be helpful
> to end users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2465) Make YARN unit tests work when pseudo distributed cluster is running
[
https://issues.apache.org/jira/browse/YARN-2465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524492#comment-14524492
]
Hadoop QA commented on YARN-2465:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 9s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 3 new or modified test files. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 1s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 5m 41s | Tests failed in
hadoop-yarn-server-nodemanager. |
| | | 22m 21s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12665165/YARN-2465.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7593/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7593/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7593/console |
This message was automatically generated.
> Make YARN unit tests work when pseudo distributed cluster is running
>
>
> Key: YARN-2465
> URL: https://issues.apache.org/jira/browse/YARN-2465
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Ming Ma
>Assignee: Ming Ma
> Attachments: YARN-2465.patch
>
>
> This is useful for development where you might have some pseudo distributed
> cluster in the background and don't want to stop it to run unit test cases.
> Most YARN test cases pass, except some tests that use localization service
> try to bind to the default localization service port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2893) AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
[ https://issues.apache.org/jira/browse/YARN-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524486#comment-14524486 ] zhihai xu commented on YARN-2893: - thanks [~adhoot] for the review and thanks [~jira.shegalov] for the review and committing the patch ! Greatly appreciated. > AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream > -- > > Key: YARN-2893 > URL: https://issues.apache.org/jira/browse/YARN-2893 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Gera Shegalov >Assignee: zhihai xu > Fix For: 2.8.0 > > Attachments: YARN-2893.000.patch, YARN-2893.001.patch, > YARN-2893.002.patch, YARN-2893.003.patch, YARN-2893.004.patch, > YARN-2893.005.patch > > > MapReduce jobs on our clusters experience sporadic failures due to corrupt > tokens in the AM launch context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2454) The function compareTo of variable UNBOUNDED in org.apache.hadoop.yarn.util.resource.Resources is definited wrong.
[
https://issues.apache.org/jira/browse/YARN-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524488#comment-14524488
]
Hadoop QA commented on YARN-2454:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 37s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 36s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 39s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 54s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 31s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 24s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 1m 55s | Tests passed in
hadoop-yarn-common. |
| | | 38m 37s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12664364/YARN-2454%20-v2.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7592/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7592/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7592/console |
This message was automatically generated.
> The function compareTo of variable UNBOUNDED in
> org.apache.hadoop.yarn.util.resource.Resources is definited wrong.
> --
>
> Key: YARN-2454
> URL: https://issues.apache.org/jira/browse/YARN-2454
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.4.0, 2.5.0, 2.4.1
>Reporter: Xu Yang
>Assignee: Xu Yang
> Attachments: YARN-2454 -v2.patch, YARN-2454-patch.diff,
> YARN-2454.patch
>
>
> The variable UNBOUNDED implement the abstract class Resources, and override
> the function compareTo. But there is something wrong in this function. We
> should not compare resources with zero as the same as the variable NONE. We
> should change 0 to Integer.MAX_VALUE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3375) NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner
[
https://issues.apache.org/jira/browse/YARN-3375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524465#comment-14524465
]
Hadoop QA commented on YARN-3375:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 30s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 35s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 43s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 43s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 22m 38s | Tests passed in
hadoop-common. |
| {color:green}+1{color} | yarn tests | 5m 49s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 67m 10s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12705561/YARN-3375.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6f541ed |
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7591/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7591/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7591/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7591/console |
This message was automatically generated.
> NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting
> NodeHealthScriptRunner
> --
>
> Key: YARN-3375
> URL: https://issues.apache.org/jira/browse/YARN-3375
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Devaraj K
>Assignee: Devaraj K
>Priority: Critical
> Attachments: YARN-3375.patch
>
>
> 1. NodeHealthScriptRunner.shouldRun() check is happening 3 times for starting
> the NodeHealthScriptRunner.
> {code:title=NodeManager.java|borderStyle=solid}
> if(!NodeHealthScriptRunner.shouldRun(nodeHealthScript)) {
> LOG.info("Abey khali");
> return null;
> }
> {code}
> {code:title=NodeHealthCheckerService.java|borderStyle=solid}
> if (NodeHealthScriptRunner.shouldRun(
> conf.get(YarnConfiguration.NM_HEALTH_CHECK_SCRIPT_PATH))) {
> addService(nodeHealthScriptRunner);
> }
> {code}
> {code:title=NodeHealthScriptRunner.java|borderStyle=solid}
> if (!shouldRun(nodeHealthScript)) {
> LOG.info("Not starting node health monitor");
> return;
> }
> {code}
> 2. If we don't configure node health script or configured health script
> doesn't execute permission, NM logs with the below message.
> {code:xml}
> 2015-03-19 19:55:45,713 INFO
> org.apache.hadoop.yarn.server.nodemanager.NodeManager: Abey khali
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3006) Improve the error message when attempting manual failover with auto-failover enabled
[
https://issues.apache.org/jira/browse/YARN-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524454#comment-14524454
]
Hudson commented on YARN-3006:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #7716 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7716/])
YARN-3006. Improve the error message when attempting manual failover with
auto-failover enabled. (Akira AJISAKA via wangda) (wangda: rev
7d46a806e71de6692cd230e64e7de18a8252019d)
* hadoop-yarn-project/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ha/HAAdmin.java
> Improve the error message when attempting manual failover with auto-failover
> enabled
>
>
> Key: YARN-3006
> URL: https://issues.apache.org/jira/browse/YARN-3006
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 2.6.0
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
>Priority: Minor
> Fix For: 2.8.0, 2.7.1
>
> Attachments: YARN-3006.001.patch
>
>
> When executing manual failover with automatic failover enabled,
> UnsupportedOperationException is thrown.
> {code}
> # yarn rmadmin -failover rm1 rm2
> Exception in thread "main" java.lang.UnsupportedOperationException:
> RMHAServiceTarget doesn't have a corresponding ZKFC address
> at
> org.apache.hadoop.yarn.client.RMHAServiceTarget.getZKFCAddress(RMHAServiceTarget.java:51)
> at
> org.apache.hadoop.ha.HAServiceTarget.getZKFCProxy(HAServiceTarget.java:94)
> at
> org.apache.hadoop.ha.HAAdmin.gracefulFailoverThroughZKFCs(HAAdmin.java:311)
> at org.apache.hadoop.ha.HAAdmin.failover(HAAdmin.java:282)
> at org.apache.hadoop.ha.HAAdmin.runCmd(HAAdmin.java:449)
> at org.apache.hadoop.ha.HAAdmin.run(HAAdmin.java:378)
> at org.apache.hadoop.yarn.client.cli.RMAdminCLI.run(RMAdminCLI.java:482)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.hadoop.yarn.client.cli.RMAdminCLI.main(RMAdminCLI.java:622)
> {code}
> I'm thinking the above message is confusing to users. (Users may think
> whether ZKFC is configured correctly...) The command should output error
> message to stderr instead of throwing Exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2893) AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream
[ https://issues.apache.org/jira/browse/YARN-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524452#comment-14524452 ] Hudson commented on YARN-2893: -- SUCCESS: Integrated in Hadoop-trunk-Commit #7716 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7716/]) YARN-2893. AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream. (Zhihai Xu via gera) (gera: rev f8204e241d9271497defd4d42646fb89c61cefe3) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestAppManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/amlauncher/AMLauncher.java > AMLaucher: sporadic job failures due to EOFException in readTokenStorageStream > -- > > Key: YARN-2893 > URL: https://issues.apache.org/jira/browse/YARN-2893 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Affects Versions: 2.4.0 >Reporter: Gera Shegalov >Assignee: zhihai xu > Attachments: YARN-2893.000.patch, YARN-2893.001.patch, > YARN-2893.002.patch, YARN-2893.003.patch, YARN-2893.004.patch, > YARN-2893.005.patch > > > MapReduce jobs on our clusters experience sporadic failures due to corrupt > tokens in the AM launch context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3363) add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container.
[ https://issues.apache.org/jira/browse/YARN-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524455#comment-14524455 ] Hudson commented on YARN-3363: -- SUCCESS: Integrated in Hadoop-trunk-Commit #7716 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7716/]) YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) (rkanter: rev ac7d152901e29b1f444507fe4e421eb6e1402b5a) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerStartMonitoringEvent.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/TestContainerMetrics.java > add localization and container launch time to ContainerMetrics at NM to show > these timing information for each active container. > > > Key: YARN-3363 > URL: https://issues.apache.org/jira/browse/YARN-3363 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Reporter: zhihai xu >Assignee: zhihai xu > Labels: metrics, supportability > Fix For: 2.8.0 > > Attachments: YARN-3363.000.patch, YARN-3363.001.patch > > > add localization and container launch time to ContainerMetrics at NM to show > these timing information for each active container. > Currently ContainerMetrics has container's actual memory usage(YARN-2984), > actual CPU usage(YARN-3122), resource and pid(YARN-3022). It will be better > to have localization and container launch time in ContainerMetrics for each > active container. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3534) Collect node resource utilization
[ https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Inigo Goiri updated YARN-3534: -- Attachment: YARN-3534-8.patch Fixing code style issues (nwo the output of the checker is meaningful so life is much easier). The broken tests weren't related to my changes. Any proposal for unit tests? The ones in ContainerMonitorImpl don't really apply. > Collect node resource utilization > - > > Key: YARN-3534 > URL: https://issues.apache.org/jira/browse/YARN-3534 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, resourcemanager >Affects Versions: 2.7.0 >Reporter: Inigo Goiri >Assignee: Inigo Goiri > Attachments: YARN-3534-1.patch, YARN-3534-2.patch, YARN-3534-3.patch, > YARN-3534-3.patch, YARN-3534-4.patch, YARN-3534-5.patch, YARN-3534-6.patch, > YARN-3534-7.patch, YARN-3534-8.patch > > Original Estimate: 336h > Remaining Estimate: 336h > > YARN should be aware of the resource utilization of the nodes when scheduling > containers. For this, this task will implement the NodeResourceMonitor and > send this information to the Resource Manager in the heartbeat. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3534) Collect node resource utilization
[
https://issues.apache.org/jira/browse/YARN-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524418#comment-14524418
]
Hadoop QA commented on YARN-3534:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 42s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 33s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 36s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 21s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 22s | The applied patch generated 2
new checkstyle issues (total was 211, now 213). |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 24s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 0m 23s | Tests passed in
hadoop-yarn-api. |
| {color:red}-1{color} | yarn tests | 5m 48s | Tests failed in
hadoop-yarn-server-nodemanager. |
| | | 44m 33s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12729860/YARN-3534-7.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6f541ed |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7589/artifact/patchprocess/diffcheckstylehadoop-yarn-api.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7589/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7589/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7589/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7589/console |
This message was automatically generated.
> Collect node resource utilization
> -
>
> Key: YARN-3534
> URL: https://issues.apache.org/jira/browse/YARN-3534
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
>Affects Versions: 2.7.0
>Reporter: Inigo Goiri
>Assignee: Inigo Goiri
> Attachments: YARN-3534-1.patch, YARN-3534-2.patch, YARN-3534-3.patch,
> YARN-3534-3.patch, YARN-3534-4.patch, YARN-3534-5.patch, YARN-3534-6.patch,
> YARN-3534-7.patch
>
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> YARN should be aware of the resource utilization of the nodes when scheduling
> containers. For this, this task will implement the NodeResourceMonitor and
> send this information to the Resource Manager in the heartbeat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3359) Recover collector list in RM failed over
[ https://issues.apache.org/jira/browse/YARN-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524405#comment-14524405 ] Junping Du commented on YARN-3359: -- Hi [~sidharta-s], Yes. We need this in phase 3 of new timeline service (now it is in phase 2). > Recover collector list in RM failed over > > > Key: YARN-3359 > URL: https://issues.apache.org/jira/browse/YARN-3359 > Project: Hadoop YARN > Issue Type: Sub-task > Components: resourcemanager >Reporter: Junping Du >Assignee: Junping Du > > Per discussion in YARN-3039, split the recover work from RMStateStore in a > separated JIRA. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3385) Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion.
[
https://issues.apache.org/jira/browse/YARN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524399#comment-14524399
]
zhihai xu commented on YARN-3385:
-
Agreed, If we have YARN-2716, this problem may be solved with it. thanks
[~jianhe]!
It may take sometime to stabilize YARN-2716, In the interim, it will be useful
to fix this issue.
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion.
> ---
>
> Key: YARN-3385
> URL: https://issues.apache.org/jira/browse/YARN-3385
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Critical
> Attachments: YARN-3385.000.patch, YARN-3385.001.patch
>
>
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion(Op.delete).
> The race condition is similar as YARN-3023.
> since the race condition exists for ZK node creation, it should also exist
> for ZK node deletion.
> We see this issue with the following stack trace:
> {code}
> 2015-03-17 19:18:58,958 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:945)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:973)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:992)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeApplicationStateInternal(ZKRMStateStore.java:647)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:691)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:766)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:761)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> 2015-03-17 19:18:58,959 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1743) Decorate event transitions and the event-types with their behaviour
[
https://issues.apache.org/jira/browse/YARN-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524400#comment-14524400
]
Hadoop QA commented on YARN-1743:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 53s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 43s | There were no new javac warning
messages. |
| {color:red}-1{color} | javadoc | 9m 49s | The applied patch generated 66
additional warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 14s | The applied patch generated 6
new checkstyle issues (total was , now 6). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 25s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 1m 57s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 5m 51s | Tests passed in
hadoop-yarn-server-nodemanager. |
| | | 46m 33s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12694668/YARN-1743-3.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6f541ed |
| javadoc |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/artifact/patchprocess/diffJavadocWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/artifact/patchprocess/diffcheckstylehadoop-yarn-common.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7588/console |
This message was automatically generated.
> Decorate event transitions and the event-types with their behaviour
> ---
>
> Key: YARN-1743
> URL: https://issues.apache.org/jira/browse/YARN-1743
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Jeff Zhang
> Labels: documentation
> Attachments: NodeManager.gv, NodeManager.pdf, YARN-1743-2.patch,
> YARN-1743-3.patch, YARN-1743.patch
>
>
> Helps to annotate the transitions with (start-state, end-state) pair and the
> events with (source, destination) pair.
> Not just readability, we may also use them to generate the event diagrams
> across components.
> Not a blocker for 0.23, but let's see.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2492) (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests
[ https://issues.apache.org/jira/browse/YARN-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524396#comment-14524396 ] Junping Du commented on YARN-2492: -- Shall we close one JIRA of YARN-796 or YARN-2492? Given we only need one umbrella. > (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests > > > Key: YARN-2492 > URL: https://issues.apache.org/jira/browse/YARN-2492 > Project: Hadoop YARN > Issue Type: Task > Components: api, client, resourcemanager >Reporter: Wangda Tan > > Since YARN-796 is a sub JIRA of YARN-397, this JIRA is used to create and > track sub tasks and attach split patches for YARN-796. > *Let's still keep over-all discussions on YARN-796.* -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2489) ResouceOption's overcommitTimeout should be respected during resource update on NM
[ https://issues.apache.org/jira/browse/YARN-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2489: - Target Version/s: (was: 2.6.0) > ResouceOption's overcommitTimeout should be respected during resource update > on NM > -- > > Key: YARN-2489 > URL: https://issues.apache.org/jira/browse/YARN-2489 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager, scheduler >Reporter: Junping Du >Assignee: Junping Du > > The ResourceOption to update NM's resource has two properties: Resource and > OvercommitTimeout. The later property is used to guarantee resource is > withdrawn after timeout is hit if resource is reduced to a value and current > resource consumption exceeds the new value. It currently use default value -1 > which means no timeout, and we should make this property work when updating > NM resource. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2482) DockerContainerExecutor configuration
[ https://issues.apache.org/jira/browse/YARN-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2482: - Issue Type: Sub-task (was: New Feature) Parent: YARN-2466 > DockerContainerExecutor configuration > - > > Key: YARN-2482 > URL: https://issues.apache.org/jira/browse/YARN-2482 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abin Shahab > Labels: security > > Currently DockerContainerExecutor can be configured from yarn-site.xml, and > users can add arbtrary arguments to the container launch command. This should > be fixed so that the cluster and other jobs are protected from malicious > string injections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2921) MockRM#waitForState methods can be too slow and flaky
[
https://issues.apache.org/jira/browse/YARN-2921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524387#comment-14524387
]
Hadoop QA commented on YARN-2921:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 5m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 5 new or modified test files. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | release audit | 0m 19s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 49s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 31s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 50m 51s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 68m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12696827/YARN-2921.004.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / d3d019c |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7586/console |
This message was automatically generated.
> MockRM#waitForState methods can be too slow and flaky
> -
>
> Key: YARN-2921
> URL: https://issues.apache.org/jira/browse/YARN-2921
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: test
>Affects Versions: 2.6.0
>Reporter: Karthik Kambatla
>Assignee: Tsuyoshi Ozawa
> Attachments: YARN-2921.001.patch, YARN-2921.002.patch,
> YARN-2921.003.patch, YARN-2921.004.patch
>
>
> MockRM#waitForState methods currently sleep for too long (2 seconds and 1
> second). This leads to slow tests and sometimes failures if the
> App/AppAttempt moves to another state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2482) DockerContainerExecutor configuration
[ https://issues.apache.org/jira/browse/YARN-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524386#comment-14524386 ] Junping Du commented on YARN-2482: -- Move it to under YARN-2466 > DockerContainerExecutor configuration > - > > Key: YARN-2482 > URL: https://issues.apache.org/jira/browse/YARN-2482 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Abin Shahab > Labels: security > > Currently DockerContainerExecutor can be configured from yarn-site.xml, and > users can add arbtrary arguments to the container launch command. This should > be fixed so that the cluster and other jobs are protected from malicious > string injections. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2479) DockerContainerExecutor must support handling of distributed cache
[ https://issues.apache.org/jira/browse/YARN-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524385#comment-14524385 ] Junping Du commented on YARN-2479: -- Move it to under YARN-2466 umbrella. > DockerContainerExecutor must support handling of distributed cache > -- > > Key: YARN-2479 > URL: https://issues.apache.org/jira/browse/YARN-2479 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Abin Shahab > Labels: security > > Interaction between Docker containers and distributed cache has not yet been > worked out. There should be a way to securely access distributed cache > without compromising the isolation Docker provides. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2479) DockerContainerExecutor must support handling of distributed cache
[ https://issues.apache.org/jira/browse/YARN-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2479: - Issue Type: Sub-task (was: New Feature) Parent: YARN-2466 > DockerContainerExecutor must support handling of distributed cache > -- > > Key: YARN-2479 > URL: https://issues.apache.org/jira/browse/YARN-2479 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abin Shahab > Labels: security > > Interaction between Docker containers and distributed cache has not yet been > worked out. There should be a way to securely access distributed cache > without compromising the isolation Docker provides. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2478) Nested containers should be supported
[ https://issues.apache.org/jira/browse/YARN-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524383#comment-14524383 ] Junping Du commented on YARN-2478: -- Move it to under YARN-2466 > Nested containers should be supported > - > > Key: YARN-2478 > URL: https://issues.apache.org/jira/browse/YARN-2478 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abin Shahab > > Currently DockerContainerExecutor only supports one level of containers. > However, YARN's responsibility is to handle resource isolation, and nested > containers would allow YARN to delegate handling software isolation to the > jobs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2478) Nested containers should be supported
[ https://issues.apache.org/jira/browse/YARN-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2478: - Issue Type: Sub-task (was: New Feature) Parent: YARN-2466 > Nested containers should be supported > - > > Key: YARN-2478 > URL: https://issues.apache.org/jira/browse/YARN-2478 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Abin Shahab > > Currently DockerContainerExecutor only supports one level of containers. > However, YARN's responsibility is to handle resource isolation, and nested > containers would allow YARN to delegate handling software isolation to the > jobs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2429) LCE should blacklist based upon group
[ https://issues.apache.org/jira/browse/YARN-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524376#comment-14524376 ] Allen Wittenauer commented on YARN-2429: Yes. to both. > LCE should blacklist based upon group > - > > Key: YARN-2429 > URL: https://issues.apache.org/jira/browse/YARN-2429 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Allen Wittenauer > > It should be possible to list a group to ban, not just individual users. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (YARN-2483) TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails due to incorrect AppAttempt state
[
https://issues.apache.org/jira/browse/YARN-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du resolved YARN-2483.
--
Resolution: Duplicate
Target Version/s: (was: 2.6.0)
Resolve this JIRA as duplicated.
> TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails due to
> incorrect AppAttempt state
>
>
> Key: YARN-2483
> URL: https://issues.apache.org/jira/browse/YARN-2483
> Project: Hadoop YARN
> Issue Type: Test
>Reporter: Ted Yu
>
> From https://builds.apache.org/job/Hadoop-Yarn-trunk/665/console :
> {code}
> testShouldNotCountFailureToMaxAttemptRetry(org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart)
> Time elapsed: 49.686 sec <<< FAILURE!
> java.lang.AssertionError: AppAttempt state is not correct (timedout)
> expected: but was:
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at
> org.apache.hadoop.yarn.server.resourcemanager.MockAM.waitForState(MockAM.java:84)
> at
> org.apache.hadoop.yarn.server.resourcemanager.MockRM.sendAMLaunched(MockRM.java:417)
> at
> org.apache.hadoop.yarn.server.resourcemanager.MockRM.launchAM(MockRM.java:582)
> at
> org.apache.hadoop.yarn.server.resourcemanager.MockRM.launchAndRegisterAM(MockRM.java:589)
> at
> org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForNewAMToLaunchAndRegister(MockRM.java:182)
> at
> org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart.testShouldNotCountFailureToMaxAttemptRetry(TestAMRestart.java:402)
> {code}
> TestApplicationMasterLauncher#testallocateBeforeAMRegistration fails with
> similar cause.
> These tests failed in build #664 as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-20) More information for "yarn.resourcemanager.webapp.address" in yarn-default.xml
[
https://issues.apache.org/jira/browse/YARN-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524373#comment-14524373
]
Hadoop QA commented on YARN-20:
---
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 41s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 1s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 7m 31s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 32s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | yarn tests | 1m 56s | Tests passed in
hadoop-yarn-common. |
| | | 36m 13s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12725385/YARN-20.1.patch |
| Optional Tests | javadoc javac unit |
| git revision | trunk / 6f541ed |
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7587/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7587/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7587/console |
This message was automatically generated.
> More information for "yarn.resourcemanager.webapp.address" in yarn-default.xml
> --
>
> Key: YARN-20
> URL: https://issues.apache.org/jira/browse/YARN-20
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: documentation, resourcemanager
>Affects Versions: 2.0.0-alpha
>Reporter: Nemon Lou
>Priority: Trivial
> Attachments: YARN-20.1.patch, YARN-20.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The parameter yarn.resourcemanager.webapp.address in yarn-default.xml is
> in "host:port" format,which is noted in the cluster set up guide
> (http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/ClusterSetup.html).
> When i read though the code,i find "host" format is also supported. In
> "host" format,the port will be random.
> So we may add more documentation in yarn-default.xml for easy understood.
> I will submit a patch if it's helpful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3385) Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion.
[
https://issues.apache.org/jira/browse/YARN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jian He updated YARN-3385:
--
Target Version/s: 2.8.0
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion.
> ---
>
> Key: YARN-3385
> URL: https://issues.apache.org/jira/browse/YARN-3385
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Critical
> Attachments: YARN-3385.000.patch, YARN-3385.001.patch
>
>
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion(Op.delete).
> The race condition is similar as YARN-3023.
> since the race condition exists for ZK node creation, it should also exist
> for ZK node deletion.
> We see this issue with the following stack trace:
> {code}
> 2015-03-17 19:18:58,958 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:945)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:973)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:992)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeApplicationStateInternal(ZKRMStateStore.java:647)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:691)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:766)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:761)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> 2015-03-17 19:18:58,959 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1418) Add Tracing to YARN
[ https://issues.apache.org/jira/browse/YARN-1418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524370#comment-14524370 ] Junping Du commented on YARN-1418: -- Hi [~hitliuyi] and [~iwasakims], do we have some proposal for this feature? > Add Tracing to YARN > --- > > Key: YARN-1418 > URL: https://issues.apache.org/jira/browse/YARN-1418 > Project: Hadoop YARN > Issue Type: Improvement > Components: api, nodemanager, resourcemanager >Reporter: Masatake Iwasaki >Assignee: Yi Liu > > Adding tracing using HTrace in the same way as HBASE-6449 and HDFS-5274. > The most part of changes needed for basis such as RPC seems to be almost > ready in HDFS-5274. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3385) Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion.
[
https://issues.apache.org/jira/browse/YARN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524365#comment-14524365
]
Jian He commented on YARN-3385:
---
thanks [~zxu] ! I'll review this.
The zk.delete seems not idempotent.
I think in general, if we have YARN-2716, this problem can be resolved along
with that. do you think so ?
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion.
> ---
>
> Key: YARN-3385
> URL: https://issues.apache.org/jira/browse/YARN-3385
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Critical
> Attachments: YARN-3385.000.patch, YARN-3385.001.patch
>
>
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion(Op.delete).
> The race condition is similar as YARN-3023.
> since the race condition exists for ZK node creation, it should also exist
> for ZK node deletion.
> We see this issue with the following stack trace:
> {code}
> 2015-03-17 19:18:58,958 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:945)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:973)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:992)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeApplicationStateInternal(ZKRMStateStore.java:647)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:691)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:766)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:761)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> 2015-03-17 19:18:58,959 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2469) Merge duplicated tests in Fifo/Capacity/Fair Scheduler into some common test
[ https://issues.apache.org/jira/browse/YARN-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2469: - Priority: Minor (was: Major) > Merge duplicated tests in Fifo/Capacity/Fair Scheduler into some common test > > > Key: YARN-2469 > URL: https://issues.apache.org/jira/browse/YARN-2469 > Project: Hadoop YARN > Issue Type: Test > Components: scheduler >Reporter: Junping Du >Priority: Minor > > From discussions in YARN-1506, there are duplicated test cases like: > testBlackListNode, testResourceOverCommit, etc. for different schedulers. We > need some common test code to cover the same test case for different > scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2470) A high value for yarn.nodemanager.delete.debug-delay-sec causes Nodemanager to crash. Slider needs this value to be high. Setting a very high value throws an exception a
[ https://issues.apache.org/jira/browse/YARN-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524356#comment-14524356 ] Junping Du commented on YARN-2470: -- Agree with [~chris.douglas]. This shouldn't be a problem as this is expected as common behavior for other int value. Close it as won't fix. > A high value for yarn.nodemanager.delete.debug-delay-sec causes Nodemanager > to crash. Slider needs this value to be high. Setting a very high value > throws an exception and nodemanager does not start > -- > > Key: YARN-2470 > URL: https://issues.apache.org/jira/browse/YARN-2470 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.4.1 >Reporter: Shivaji Dutta >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524357#comment-14524357
]
Hadoop QA commented on YARN-2892:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 45s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 51s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 45s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 52m 6s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 26s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler |
| | hadoop.yarn.server.resourcemanager.TestAppManager |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684732/YARN-2892.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d3d019c |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/console |
This message was automatically generated.
> Unable to get AMRMToken in unmanaged AM when using a secure cluster
> ---
>
> Key: YARN-2892
> URL: https://issues.apache.org/jira/browse/YARN-2892
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: Sevada Abraamyan
>Assignee: Sevada Abraamyan
> Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
>
>
> An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
> When the RM creates the ApplicationReport and sends it back to the client it
> makes a simple security check whether it should include the AMRMToken in the
> report (See createAndGetApplicationReport in RMAppImpl).This security check
> verifies that the user who submitted the original application is the same
> user who is requesting the ApplicationReport. If they are indeed the same
> user then it includes the AMRMToken, otherwise it does not include it.
> The problem arises from the fact that when an application is submitted, the
> RM saves the short username of the user who created the application (See
> submitApplication in ClientRmService). Afterwards when the ApplicationReport
> is requested, the system tries to match the full username of the requester
> against the previously stored short username.
> In a secure cluster using Kerberos this check fails because the principle is
> stripped from the username when we request a short username. So for example
> the short username might be "Foo" whereas the full username is
> "f...@company.com"
> Note: A very similar problem has been previously reported
> ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (YARN-2470) A high value for yarn.nodemanager.delete.debug-delay-sec causes Nodemanager to crash. Slider needs this value to be high. Setting a very high value throws an exception an
[ https://issues.apache.org/jira/browse/YARN-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved YARN-2470. -- Resolution: Won't Fix > A high value for yarn.nodemanager.delete.debug-delay-sec causes Nodemanager > to crash. Slider needs this value to be high. Setting a very high value > throws an exception and nodemanager does not start > -- > > Key: YARN-2470 > URL: https://issues.apache.org/jira/browse/YARN-2470 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.4.1 >Reporter: Shivaji Dutta >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3385) Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion.
[
https://issues.apache.org/jira/browse/YARN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524352#comment-14524352
]
zhihai xu commented on YARN-3385:
-
Thanks [~sidharta-s], I uploaded a new patch YARN-3385.001.patch based on the
latest code base.
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion.
> ---
>
> Key: YARN-3385
> URL: https://issues.apache.org/jira/browse/YARN-3385
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Critical
> Attachments: YARN-3385.000.patch, YARN-3385.001.patch
>
>
> Race condition: KeeperException$NoNodeException will cause RM shutdown during
> ZK node deletion(Op.delete).
> The race condition is similar as YARN-3023.
> since the race condition exists for ZK node creation, it should also exist
> for ZK node deletion.
> We see this issue with the following stack trace:
> {code}
> 2015-03-17 19:18:58,958 FATAL
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a
> org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type
> STATE_STORE_OP_FAILED. Cause:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
> at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:945)
> at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:857)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:973)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:992)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:854)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.removeApplicationStateInternal(ZKRMStateStore.java:647)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.handleStoreEvent(RMStateStore.java:691)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:766)
> at
> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler.handle(RMStateStore.java:761)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
> at java.lang.Thread.run(Thread.java:745)
> 2015-03-17 19:18:58,959 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-2469) Merge duplicated tests in Fifo/Capacity/Fair Scheduler into some common test
[ https://issues.apache.org/jira/browse/YARN-2469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2469: - Target Version/s: (was: 2.6.0) > Merge duplicated tests in Fifo/Capacity/Fair Scheduler into some common test > > > Key: YARN-2469 > URL: https://issues.apache.org/jira/browse/YARN-2469 > Project: Hadoop YARN > Issue Type: Test > Components: scheduler >Reporter: Junping Du > > From discussions in YARN-1506, there are duplicated test cases like: > testBlackListNode, testResourceOverCommit, etc. for different schedulers. We > need some common test code to cover the same test case for different > scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
