[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[
https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890486#comment-13890486
]
Hadoop QA commented on YARN-1461:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626838/yarn-1461-11.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3005//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3005//console
This message is automatically generated.
RM API and RM changes to handle tags for running jobs
-
Key: YARN-1461
URL: https://issues.apache.org/jira/browse/YARN-1461
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: yarn-1461-1.patch, yarn-1461-10.patch,
yarn-1461-11.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch,
yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch,
yarn-1461-8.patch, yarn-1461-9.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[
https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890493#comment-13890493
]
Hadoop QA commented on YARN-1461:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626840/yarn-1461-11.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3007//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3007//console
This message is automatically generated.
RM API and RM changes to handle tags for running jobs
-
Key: YARN-1461
URL: https://issues.apache.org/jira/browse/YARN-1461
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: yarn-1461-1.patch, yarn-1461-10.patch,
yarn-1461-11.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch,
yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch,
yarn-1461-8.patch, yarn-1461-9.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1673) Valid yarn kill application prints out help message.
[ https://issues.apache.org/jira/browse/YARN-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890566#comment-13890566 ] Hudson commented on YARN-1673: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java Valid yarn kill application prints out help message. Key: YARN-1673 URL: https://issues.apache.org/jira/browse/YARN-1673 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.4.0 Reporter: Tassapol Athiapinya Assignee: Mayank Bansal Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1673-1.patch, YARN-1673.patch, YARN-1673.txt yarn application -kill application ID used to work previously. In 2.4.0 it prints out help message and does not kill the application. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1684) Fix history server heap size in yarn script
[
https://issues.apache.org/jira/browse/YARN-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890567#comment-13890567
]
Hudson commented on YARN-1684:
--
SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/471/])
YARN-1684. Fixed history server heap size in yarn script. Contributed by Billie
Rinaldi. (zjshen:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564193)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn.cmd
Fix history server heap size in yarn script
---
Key: YARN-1684
URL: https://issues.apache.org/jira/browse/YARN-1684
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: 2.4.0
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Fix For: 2.4.0
Attachments: YARN-1684.2.patch, YARN-1684.patch
The yarn script currently has the following:
{noformat}
if [ $YARN_RESOURCEMANAGER_HEAPSIZE != ]; then
JAVA_HEAP_MAX=-Xmx$YARN_HISTORYSERVER_HEAPSIZEm
fi
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1667) Make admin refreshSuperUserGroupsConfiguration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890568#comment-13890568 ] Hudson commented on YARN-1667: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) Fix YARN's CHANGES.txt issue caused during the commit for YARN-1667. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564185) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564100) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ProxyUsers.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshSuperUserGroupsConfiguration work across RM failover -- Key: YARN-1667 URL: https://issues.apache.org/jira/browse/YARN-1667 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1667.1.patch, YARN-1667.2.patch, YARN-1667.3.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1639) YARM RM HA requires different configs on different RM hosts
[ https://issues.apache.org/jira/browse/YARN-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890574#comment-13890574 ] Hudson commented on YARN-1639: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564032) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/HAUtil.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java YARM RM HA requires different configs on different RM hosts --- Key: YARN-1639 URL: https://issues.apache.org/jira/browse/YARN-1639 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Arpit Gupta Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1639.1.patch, YARN-1639.2.patch, YARN-1639.3.patch, YARN-1639.4.patch, YARN-1639.5.patch We need to set yarn.resourcemanager.ha.id to rm1 or rm2 based on which rm you want to first or second. This means we have different configs on different RM nodes. This is unlike HDFS HA where the same configs are pushed to both NN's and it would be better to have the same setup for RM as this would make installation and managing easier. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-967) [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data
[ https://issues.apache.org/jira/browse/YARN-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890572#comment-13890572 ] Hudson commented on YARN-967: - SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data --- Key: YARN-967 URL: https://issues.apache.org/jira/browse/YARN-967 Project: Hadoop YARN Issue Type: Sub-task Reporter: Devaraj K Assignee: Mayank Bansal Fix For: 2.4.0 Attachments: YARN-967-1.patch, YARN-967-10.patch, YARN-967-11.patch, YARN-967-12.patch, YARN-967-13.patch, YARN-967-14.patch, YARN-967-2.patch, YARN-967-3.patch, YARN-967-4.patch, YARN-967-5.patch, YARN-967-6.patch, YARN-967-7.patch, YARN-967-8.patch, YARN-967-9.patch, YARN-967.15.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890571#comment-13890571 ] Hudson commented on YARN-1498: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1498 addendum to fix findbugs warning (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564018) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java Common scheduler changes for moving apps between queues --- Key: YARN-1498 URL: https://issues.apache.org/jira/browse/YARN-1498 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, YARN-1498.patch, YARN-1498.patch This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. In particular, it makes sure that QueueMetrics are properly updated when an app changes queue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890570#comment-13890570 ] Hudson commented on YARN-1611: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564002) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProviderFactory.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/FileSystemBasedConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/LocalConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refresh of capacity scheduler configuration work across RM failover -- Key: YARN-1611 URL: https://issues.apache.org/jira/browse/YARN-1611 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1611.1.patch, YARN-1611.10.patch, YARN-1611.2.patch, YARN-1611.2.patch, YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch, YARN-1611.9.patch, YARN-1611.9.patch Currently, If we do refresh* for a standby RM, it will failover to the current active RM, and do the refresh* based on the local configuration file of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1285) Inconsistency of default yarn.acl.enable value
[ https://issues.apache.org/jira/browse/YARN-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890576#comment-13890576 ] Hudson commented on YARN-1285: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1285. Changed the default value of yarn.acl.enable in yarn-default.xml to be consistent with what exists (false) in the code and documentation. Contributed by Kenji Kikushima. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564190) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml Inconsistency of default yarn.acl.enable value Key: YARN-1285 URL: https://issues.apache.org/jira/browse/YARN-1285 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Zhijie Shen Assignee: Kenji Kikushima Fix For: 2.4.0 Attachments: YARN-1285.patch In yarn-default.xml, yarn.acl.enable is true while in YarnConfiguration, DEFAULT_YARN_ACL_ENABLE is false. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1668) Make admin refreshAdminAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890577#comment-13890577 ] Hudson commented on YARN-1668: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #471 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/471/]) YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564043) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshAdminAcls work across RM failover --- Key: YARN-1668 URL: https://issues.apache.org/jira/browse/YARN-1668 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1668.1.patch, YARN-1668.2.patch, YARN-1668.3.patch Change the handling of admin-acls to be available across RM failover by making using of a remote configuration-provider -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1673) Valid yarn kill application prints out help message.
[ https://issues.apache.org/jira/browse/YARN-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890656#comment-13890656 ] Hudson commented on YARN-1673: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java Valid yarn kill application prints out help message. Key: YARN-1673 URL: https://issues.apache.org/jira/browse/YARN-1673 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.4.0 Reporter: Tassapol Athiapinya Assignee: Mayank Bansal Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1673-1.patch, YARN-1673.patch, YARN-1673.txt yarn application -kill application ID used to work previously. In 2.4.0 it prints out help message and does not kill the application. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1639) YARM RM HA requires different configs on different RM hosts
[ https://issues.apache.org/jira/browse/YARN-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890664#comment-13890664 ] Hudson commented on YARN-1639: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564032) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/HAUtil.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java YARM RM HA requires different configs on different RM hosts --- Key: YARN-1639 URL: https://issues.apache.org/jira/browse/YARN-1639 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Arpit Gupta Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1639.1.patch, YARN-1639.2.patch, YARN-1639.3.patch, YARN-1639.4.patch, YARN-1639.5.patch We need to set yarn.resourcemanager.ha.id to rm1 or rm2 based on which rm you want to first or second. This means we have different configs on different RM nodes. This is unlike HDFS HA where the same configs are pushed to both NN's and it would be better to have the same setup for RM as this would make installation and managing easier. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1668) Make admin refreshAdminAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890667#comment-13890667 ] Hudson commented on YARN-1668: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564043) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshAdminAcls work across RM failover --- Key: YARN-1668 URL: https://issues.apache.org/jira/browse/YARN-1668 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1668.1.patch, YARN-1668.2.patch, YARN-1668.3.patch Change the handling of admin-acls to be available across RM failover by making using of a remote configuration-provider -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1667) Make admin refreshSuperUserGroupsConfiguration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890658#comment-13890658 ] Hudson commented on YARN-1667: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) Fix YARN's CHANGES.txt issue caused during the commit for YARN-1667. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564185) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564100) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ProxyUsers.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshSuperUserGroupsConfiguration work across RM failover -- Key: YARN-1667 URL: https://issues.apache.org/jira/browse/YARN-1667 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1667.1.patch, YARN-1667.2.patch, YARN-1667.3.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1659) Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information
[ https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890663#comment-13890663 ] Hudson commented on YARN-1659: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564037) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSPutErrors.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineReader.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineWriter.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/NameValuePair.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/package-info.java Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information - Key: YARN-1659 URL: https://issues.apache.org/jira/browse/YARN-1659 Project: Hadoop YARN Issue Type: Sub-task Reporter: Billie Rinaldi Assignee: Billie Rinaldi Fix For: 2.4.0 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659-7.patch, YARN-1659.2.patch These will be used by ApplicationTimelineStore interface. The web services will convert the store-facing obects to the user-facing objects. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-967) [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data
[ https://issues.apache.org/jira/browse/YARN-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890662#comment-13890662 ] Hudson commented on YARN-967: - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data --- Key: YARN-967 URL: https://issues.apache.org/jira/browse/YARN-967 Project: Hadoop YARN Issue Type: Sub-task Reporter: Devaraj K Assignee: Mayank Bansal Fix For: 2.4.0 Attachments: YARN-967-1.patch, YARN-967-10.patch, YARN-967-11.patch, YARN-967-12.patch, YARN-967-13.patch, YARN-967-14.patch, YARN-967-2.patch, YARN-967-3.patch, YARN-967-4.patch, YARN-967-5.patch, YARN-967-6.patch, YARN-967-7.patch, YARN-967-8.patch, YARN-967-9.patch, YARN-967.15.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1684) Fix history server heap size in yarn script
[
https://issues.apache.org/jira/browse/YARN-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890657#comment-13890657
]
Hudson commented on YARN-1684:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/])
YARN-1684. Fixed history server heap size in yarn script. Contributed by Billie
Rinaldi. (zjshen:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564193)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn.cmd
Fix history server heap size in yarn script
---
Key: YARN-1684
URL: https://issues.apache.org/jira/browse/YARN-1684
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: 2.4.0
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Fix For: 2.4.0
Attachments: YARN-1684.2.patch, YARN-1684.patch
The yarn script currently has the following:
{noformat}
if [ $YARN_RESOURCEMANAGER_HEAPSIZE != ]; then
JAVA_HEAP_MAX=-Xmx$YARN_HISTORYSERVER_HEAPSIZEm
fi
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890660#comment-13890660 ] Hudson commented on YARN-1611: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564002) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProviderFactory.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/FileSystemBasedConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/LocalConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refresh of capacity scheduler configuration work across RM failover -- Key: YARN-1611 URL: https://issues.apache.org/jira/browse/YARN-1611 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1611.1.patch, YARN-1611.10.patch, YARN-1611.2.patch, YARN-1611.2.patch, YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch, YARN-1611.9.patch, YARN-1611.9.patch Currently, If we do refresh* for a standby RM, it will failover to the current active RM, and do the refresh* based on the local configuration file of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890661#comment-13890661 ] Hudson commented on YARN-1498: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1688 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1688/]) YARN-1498 addendum to fix findbugs warning (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564018) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java Common scheduler changes for moving apps between queues --- Key: YARN-1498 URL: https://issues.apache.org/jira/browse/YARN-1498 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, YARN-1498.patch, YARN-1498.patch This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. In particular, it makes sure that QueueMetrics are properly updated when an app changes queue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1684) Fix history server heap size in yarn script
[
https://issues.apache.org/jira/browse/YARN-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890680#comment-13890680
]
Hudson commented on YARN-1684:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/])
YARN-1684. Fixed history server heap size in yarn script. Contributed by Billie
Rinaldi. (zjshen:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564193)
* /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn.cmd
Fix history server heap size in yarn script
---
Key: YARN-1684
URL: https://issues.apache.org/jira/browse/YARN-1684
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: 2.4.0
Reporter: Billie Rinaldi
Assignee: Billie Rinaldi
Fix For: 2.4.0
Attachments: YARN-1684.2.patch, YARN-1684.patch
The yarn script currently has the following:
{noformat}
if [ $YARN_RESOURCEMANAGER_HEAPSIZE != ]; then
JAVA_HEAP_MAX=-Xmx$YARN_HISTORYSERVER_HEAPSIZEm
fi
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1673) Valid yarn kill application prints out help message.
[ https://issues.apache.org/jira/browse/YARN-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890679#comment-13890679 ] Hudson commented on YARN-1673: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java Valid yarn kill application prints out help message. Key: YARN-1673 URL: https://issues.apache.org/jira/browse/YARN-1673 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.4.0 Reporter: Tassapol Athiapinya Assignee: Mayank Bansal Priority: Blocker Fix For: 2.4.0 Attachments: YARN-1673-1.patch, YARN-1673.patch, YARN-1673.txt yarn application -kill application ID used to work previously. In 2.4.0 it prints out help message and does not kill the application. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1285) Inconsistency of default yarn.acl.enable value
[ https://issues.apache.org/jira/browse/YARN-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890689#comment-13890689 ] Hudson commented on YARN-1285: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1285. Changed the default value of yarn.acl.enable in yarn-default.xml to be consistent with what exists (false) in the code and documentation. Contributed by Kenji Kikushima. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564190) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml Inconsistency of default yarn.acl.enable value Key: YARN-1285 URL: https://issues.apache.org/jira/browse/YARN-1285 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.2.0 Reporter: Zhijie Shen Assignee: Kenji Kikushima Fix For: 2.4.0 Attachments: YARN-1285.patch In yarn-default.xml, yarn.acl.enable is true while in YarnConfiguration, DEFAULT_YARN_ACL_ENABLE is false. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1667) Make admin refreshSuperUserGroupsConfiguration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890681#comment-13890681 ] Hudson commented on YARN-1667: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) Fix YARN's CHANGES.txt issue caused during the commit for YARN-1667. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564185) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt YARN-1667. Modified RM HA handling of super users (with proxying ability) to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564100) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ProxyUsers.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshSuperUserGroupsConfiguration work across RM failover -- Key: YARN-1667 URL: https://issues.apache.org/jira/browse/YARN-1667 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1667.1.patch, YARN-1667.2.patch, YARN-1667.3.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-967) [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data
[ https://issues.apache.org/jira/browse/YARN-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890685#comment-13890685 ] Hudson commented on YARN-967: - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1673. Fix option parsing in YARN's application CLI after it is broken by YARN-967. Contributed by Mayank Bansal. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564188) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/ApplicationCLI.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data --- Key: YARN-967 URL: https://issues.apache.org/jira/browse/YARN-967 Project: Hadoop YARN Issue Type: Sub-task Reporter: Devaraj K Assignee: Mayank Bansal Fix For: 2.4.0 Attachments: YARN-967-1.patch, YARN-967-10.patch, YARN-967-11.patch, YARN-967-12.patch, YARN-967-13.patch, YARN-967-14.patch, YARN-967-2.patch, YARN-967-3.patch, YARN-967-4.patch, YARN-967-5.patch, YARN-967-6.patch, YARN-967-7.patch, YARN-967-8.patch, YARN-967-9.patch, YARN-967.15.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1498) Common scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890684#comment-13890684 ] Hudson commented on YARN-1498: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1498 addendum to fix findbugs warning (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564018) * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApplicationAttempt.java Common scheduler changes for moving apps between queues --- Key: YARN-1498 URL: https://issues.apache.org/jira/browse/YARN-1498 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1498-1.patch, YARN-1498-addendum.patch, YARN-1498.patch, YARN-1498.patch This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. In particular, it makes sure that QueueMetrics are properly updated when an app changes queue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1611) Make admin refresh of capacity scheduler configuration work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890683#comment-13890683 ] Hudson commented on YARN-1611: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1611. Introduced the concept of a configuration provider which can be used by ResourceManager to read configuration locally or from remote systems so as to help RM failover. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564002) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/ConfigurationProviderFactory.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/FileSystemBasedConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/LocalConfigurationProvider.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refresh of capacity scheduler configuration work across RM failover -- Key: YARN-1611 URL: https://issues.apache.org/jira/browse/YARN-1611 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1611.1.patch, YARN-1611.10.patch, YARN-1611.2.patch, YARN-1611.2.patch, YARN-1611.3.patch, YARN-1611.3.patch, YARN-1611.4.patch, YARN-1611.5.patch, YARN-1611.6.patch, YARN-1611.7.patch, YARN-1611.8.patch, YARN-1611.9.patch, YARN-1611.9.patch Currently, If we do refresh* for a standby RM, it will failover to the current active RM, and do the refresh* based on the local configuration file of the active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1639) YARM RM HA requires different configs on different RM hosts
[ https://issues.apache.org/jira/browse/YARN-1639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890687#comment-13890687 ] Hudson commented on YARN-1639: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1639. Modified RM HA configuration handling to have a way of not requiring separate configuration files for each RM. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564032) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/HAUtil.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHA.java YARM RM HA requires different configs on different RM hosts --- Key: YARN-1639 URL: https://issues.apache.org/jira/browse/YARN-1639 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Reporter: Arpit Gupta Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1639.1.patch, YARN-1639.2.patch, YARN-1639.3.patch, YARN-1639.4.patch, YARN-1639.5.patch We need to set yarn.resourcemanager.ha.id to rm1 or rm2 based on which rm you want to first or second. This means we have different configs on different RM nodes. This is unlike HDFS HA where the same configs are pushed to both NN's and it would be better to have the same setup for RM as this would make installation and managing easier. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1668) Make admin refreshAdminAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890690#comment-13890690 ] Hudson commented on YARN-1668: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1668. Modified RM HA handling of admin-acls to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564043) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshAdminAcls work across RM failover --- Key: YARN-1668 URL: https://issues.apache.org/jira/browse/YARN-1668 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1668.1.patch, YARN-1668.2.patch, YARN-1668.3.patch Change the handling of admin-acls to be available across RM failover by making using of a remote configuration-provider -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1659) Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information
[ https://issues.apache.org/jira/browse/YARN-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890686#comment-13890686 ] Hudson commented on YARN-1659: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1663 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1663/]) YARN-1659. Defined the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information. Contributed by Billie Rinaldi. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564037) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvent.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEvents.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSPutErrors.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/apptimeline/TestApplicationTimelineRecords.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineReader.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineWriter.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/NameValuePair.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/package-info.java Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information - Key: YARN-1659 URL: https://issues.apache.org/jira/browse/YARN-1659 Project: Hadoop YARN Issue Type: Sub-task Reporter: Billie Rinaldi Assignee: Billie Rinaldi Fix For: 2.4.0 Attachments: YARN-1659-1.patch, YARN-1659-3.patch, YARN-1659-4.patch, YARN-1659-5.patch, YARN-1659-6.patch, YARN-1659-7.patch, YARN-1659.2.patch These will be used by ApplicationTimelineStore interface. The web services will convert the store-facing obects to the user-facing objects. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1590) _HOST doesn't expand properly for RM, NM, ProxyServer and JHS
[ https://issues.apache.org/jira/browse/YARN-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated YARN-1590: Attachment: YARN-1590.4.patch New patch that addressed Vinod's comments. _HOST doesn't expand properly for RM, NM, ProxyServer and JHS - Key: YARN-1590 URL: https://issues.apache.org/jira/browse/YARN-1590 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 3.0.0, 2.2.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: YARN-1590.1.patch, YARN-1590.2.patch, YARN-1590.3.patch, YARN-1590.4.patch _HOST is not properly substituted when we use VIP address. Currently it always used the host name of the machine and disregard the VIP address. It is true mainly for RM, NM, WebProxy, and JHS rpc service. Looks like it is working fine for webservice authentication. On the other hand, the same thing is working fine for NN and SNN in RPC as well as webservice. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1444) RM crashes when node resource request sent without corresponding rack request
[ https://issues.apache.org/jira/browse/YARN-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891060#comment-13891060 ] Arun C Murthy commented on YARN-1444: - [~gp.leftnoteasy] Did you get a chance to look at this? For now, I'll move this to 2.4 since it's a long-standing issue. RM crashes when node resource request sent without corresponding rack request - Key: YARN-1444 URL: https://issues.apache.org/jira/browse/YARN-1444 Project: Hadoop YARN Issue Type: Bug Components: client, resourcemanager Reporter: Robert Grandl Assignee: Wangda Tan Priority: Blocker Attachments: yarn-1444.ver1.patch I have tried to force reducers to execute on certain nodes. What I did is I changed for reduce tasks, the RMContainerRequestor#addResourceRequest(req.priority, ResourceRequest.ANY, req.capability) to RMContainerRequestor#addResourceRequest(req.priority, HOST_NAME, req.capability). However, this change lead to RM crashes when reducers needs to be assigned with the following exception: FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling event type NODE_UPDATE to the scheduler java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:841) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:640) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:554) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:695) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:739) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:86) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:549) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1444) RM crashes when node resource request sent without corresponding rack request
[ https://issues.apache.org/jira/browse/YARN-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated YARN-1444: Target Version/s: 2.4.0 (was: 2.3.0) RM crashes when node resource request sent without corresponding rack request - Key: YARN-1444 URL: https://issues.apache.org/jira/browse/YARN-1444 Project: Hadoop YARN Issue Type: Bug Components: client, resourcemanager Reporter: Robert Grandl Assignee: Wangda Tan Priority: Blocker Attachments: yarn-1444.ver1.patch I have tried to force reducers to execute on certain nodes. What I did is I changed for reduce tasks, the RMContainerRequestor#addResourceRequest(req.priority, ResourceRequest.ANY, req.capability) to RMContainerRequestor#addResourceRequest(req.priority, HOST_NAME, req.capability). However, this change lead to RM crashes when reducers needs to be assigned with the following exception: FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling event type NODE_UPDATE to the scheduler java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:841) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:640) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:554) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:695) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:739) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:86) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:549) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1590) _HOST doesn't expand properly for RM, NM, ProxyServer and JHS
[ https://issues.apache.org/jira/browse/YARN-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891065#comment-13891065 ] Mohammad Kamrul Islam commented on YARN-1590: - bq. It is true mainly for RM, NM,, WebProxy, and JHS rpc service. Looks like it is working fine for webservice authentication. It means RM UI (for example) works fine with _HOST. However, any RM rpc communication with NM (e.g.) doesn't work. We tested that this patch resolved the RM--NM communications failure. _HOST doesn't expand properly for RM, NM, ProxyServer and JHS - Key: YARN-1590 URL: https://issues.apache.org/jira/browse/YARN-1590 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 3.0.0, 2.2.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: YARN-1590.1.patch, YARN-1590.2.patch, YARN-1590.3.patch, YARN-1590.4.patch _HOST is not properly substituted when we use VIP address. Currently it always used the host name of the machine and disregard the VIP address. It is true mainly for RM, NM, WebProxy, and JHS rpc service. Looks like it is working fine for webservice authentication. On the other hand, the same thing is working fine for NN and SNN in RPC as well as webservice. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-1635: - Attachment: YARN-1635.5.patch Updated patch fixing errors. Implement a Leveldb based ApplicationTimelineStore -- Key: YARN-1635 URL: https://issues.apache.org/jira/browse/YARN-1635 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Billie Rinaldi Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch, YARN-1635.4.patch, YARN-1635.5.patch As per the design doc, we need a levelDB + local-filesystem based implementation to start with and for small deployments. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891084#comment-13891084 ] Arun C Murthy commented on YARN-1206: - I haven't seen much progress here, [~jianhe] I'm inclined to move this to 2.4? Thoughts? Thanks. Container logs link is broken on RM web UI after application finished - Key: YARN-1206 URL: https://issues.apache.org/jira/browse/YARN-1206 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Priority: Blocker With log aggregation disabled, when container is running, its logs link works properly, but after the application is finished, the link shows 'Container does not exist.' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1577) Unmanaged AM is broken because of YARN-1493
[ https://issues.apache.org/jira/browse/YARN-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891083#comment-13891083 ] Arun C Murthy commented on YARN-1577: - I haven't seen much progress here, [~jianhe] should I move this to 2.4? Thanks. Unmanaged AM is broken because of YARN-1493 --- Key: YARN-1577 URL: https://issues.apache.org/jira/browse/YARN-1577 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.3.0 Reporter: Jian He Assignee: Jian He Priority: Blocker Today unmanaged AM client is waiting for app state to be Accepted to launch the AM. This is broken since we changed in YARN-1493 to start the attempt after the application is Accepted. We may need to introduce an attempt state report that client can rely on to query the attempt state and choose to launch the unmanaged AM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891093#comment-13891093 ] Karthik Kambatla commented on YARN-1497: Looks good to me. Can you look into the test failure? Let us get this in after YARN-1499. Expose moving apps between queues on the command line - Key: YARN-1497 URL: https://issues.apache.org/jira/browse/YARN-1497 Project: Hadoop YARN Issue Type: Sub-task Components: client Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1497.patch, YARN-1497.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891129#comment-13891129 ] Karthik Kambatla commented on YARN-1461: bq. The GetApplicationsRequest API should take either/or or both tags/scope as input? Scope goes with all other filters - types, states, tags, name, queue, start and finish times. Should we add newInstance() methods for all combinations. That would probably just be too many, which people might not actually use. Also, it would add conflicts - say username and queue both are Strings. My personal view is that we should have a single newInstance() method and users should set all other filters on their own. RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[
https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891139#comment-13891139
]
Hadoop QA commented on YARN-1635:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626951/YARN-1635.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3009//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3009//console
This message is automatically generated.
Implement a Leveldb based ApplicationTimelineStore
--
Key: YARN-1635
URL: https://issues.apache.org/jira/browse/YARN-1635
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Billie Rinaldi
Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch,
YARN-1635.4.patch, YARN-1635.5.patch
As per the design doc, we need a levelDB + local-filesystem based
implementation to start with and for small deployments.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1590) _HOST doesn't expand properly for RM, NM, ProxyServer and JHS
[
https://issues.apache.org/jira/browse/YARN-1590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891140#comment-13891140
]
Hadoop QA commented on YARN-1590:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626945/YARN-1590.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3008//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3008//console
This message is automatically generated.
_HOST doesn't expand properly for RM, NM, ProxyServer and JHS
-
Key: YARN-1590
URL: https://issues.apache.org/jira/browse/YARN-1590
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Affects Versions: 3.0.0, 2.2.0
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Attachments: YARN-1590.1.patch, YARN-1590.2.patch, YARN-1590.3.patch,
YARN-1590.4.patch
_HOST is not properly substituted when we use VIP address. Currently it
always used the host name of the machine and disregard the VIP address. It is
true mainly for RM, NM, WebProxy, and JHS rpc service. Looks like it is
working fine for webservice authentication.
On the other hand, the same thing is working fine for NN and SNN in RPC as
well as webservice.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1669) Make admin refreshServiceAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1669: Attachment: YARN-1669.5.patch Make admin refreshServiceAcls work across RM failover - Key: YARN-1669 URL: https://issues.apache.org/jira/browse/YARN-1669 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-1669.1.patch, YARN-1669.2.patch, YARN-1669.3.patch, YARN-1669.4.patch, YARN-1669.5.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-1635: - Attachment: YARN-1635.6.patch Implement a Leveldb based ApplicationTimelineStore -- Key: YARN-1635 URL: https://issues.apache.org/jira/browse/YARN-1635 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Billie Rinaldi Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch, YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch As per the design doc, we need a levelDB + local-filesystem based implementation to start with and for small deployments. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1634: -- Attachment: YARN-1634.2.patch Upload a new patch witch is rebased according to the latest trunk. Define an in-memory implementation of ApplicationTimelineStore -- Key: YARN-1634 URL: https://issues.apache.org/jira/browse/YARN-1634 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1634.1.patch, YARN-1634.2.patch As per the design doc, the store needs to pluggable. We need a base interface, and an in-memory implementation for testing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[
https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891249#comment-13891249
]
Hadoop QA commented on YARN-1634:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626978/YARN-1634.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3012//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3012//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3012//console
This message is automatically generated.
Define an in-memory implementation of ApplicationTimelineStore
--
Key: YARN-1634
URL: https://issues.apache.org/jira/browse/YARN-1634
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
Attachments: YARN-1634.1.patch, YARN-1634.2.patch
As per the design doc, the store needs to pluggable. We need a base
interface, and an in-memory implementation for testing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[
https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891252#comment-13891252
]
Hadoop QA commented on YARN-1635:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626976/YARN-1635.6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3011//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3011//console
This message is automatically generated.
Implement a Leveldb based ApplicationTimelineStore
--
Key: YARN-1635
URL: https://issues.apache.org/jira/browse/YARN-1635
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Billie Rinaldi
Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch,
YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch
As per the design doc, we need a levelDB + local-filesystem based
implementation to start with and for small deployments.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1499) Fair Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1499: - Attachment: YARN-1499-2.patch Updated patch addresses Karthik's comments and adds some documentation to FairScheduler.apt.vm Fair Scheduler changes for moving apps between queues - Key: YARN-1499 URL: https://issues.apache.org/jira/browse/YARN-1499 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1499-1.patch, YARN-1499-2.patch, YARN-1499.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1497: - Attachment: YARN-1497-1.patch Expose moving apps between queues on the command line - Key: YARN-1497 URL: https://issues.apache.org/jira/browse/YARN-1497 Project: Hadoop YARN Issue Type: Sub-task Components: client Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891332#comment-13891332 ] Sandy Ryza commented on YARN-1497: -- NMClient failure is not realted. Uploaded patch to fix TestYarnCLI failure. Expose moving apps between queues on the command line - Key: YARN-1497 URL: https://issues.apache.org/jira/browse/YARN-1497 Project: Hadoop YARN Issue Type: Sub-task Components: client Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1669) Make admin refreshServiceAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891335#comment-13891335 ] Vinod Kumar Vavilapalli commented on YARN-1669: --- +1, looks good. Checking this in. Make admin refreshServiceAcls work across RM failover - Key: YARN-1669 URL: https://issues.apache.org/jira/browse/YARN-1669 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Attachments: YARN-1669.1.patch, YARN-1669.2.patch, YARN-1669.3.patch, YARN-1669.4.patch, YARN-1669.5.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1634: -- Attachment: YARN-1634.3.patch Upload a new patch: 1. Refactor the test code 2. Fix RAT warning The javadoc warnings should be unrelated. Define an in-memory implementation of ApplicationTimelineStore -- Key: YARN-1634 URL: https://issues.apache.org/jira/browse/YARN-1634 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1634.1.patch, YARN-1634.2.patch, YARN-1634.3.patch As per the design doc, the store needs to pluggable. We need a base interface, and an in-memory implementation for testing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1669) Make admin refreshServiceAcls work across RM failover
[ https://issues.apache.org/jira/browse/YARN-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891464#comment-13891464 ] Hudson commented on YARN-1669: -- FAILURE: Integrated in Hadoop-trunk-Commit #5104 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5104/]) YARN-1669. Modified RM HA handling of protocol level service-ACLS to be available across RM failover by making using of a remote configuration-provider. Contributed by Xuan Gong. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564549) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/authorize/ServiceAuthorizationManager.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java Make admin refreshServiceAcls work across RM failover - Key: YARN-1669 URL: https://issues.apache.org/jira/browse/YARN-1669 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Xuan Gong Fix For: 2.4.0 Attachments: YARN-1669.1.patch, YARN-1669.2.patch, YARN-1669.3.patch, YARN-1669.4.patch, YARN-1669.5.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891473#comment-13891473 ] Karthik Kambatla commented on YARN-1461: Spoke to Vinod offline. We want to add a single newInstance() method that takes all possible filters (users, queues, types, tags, scope etc.). RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[
https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891484#comment-13891484
]
Hadoop QA commented on YARN-1634:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627030/YARN-1634.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3015//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3015//console
This message is automatically generated.
Define an in-memory implementation of ApplicationTimelineStore
--
Key: YARN-1634
URL: https://issues.apache.org/jira/browse/YARN-1634
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
Attachments: YARN-1634.1.patch, YARN-1634.2.patch, YARN-1634.3.patch
As per the design doc, the store needs to pluggable. We need a base
interface, and an in-memory implementation for testing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1461: --- Attachment: yarn-1461-12.patch Patch that adds the newInstance() method. Running javadoc locally didn't show any javadoc errors. RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1444) RM crashes when node resource request sent without corresponding rack request
[ https://issues.apache.org/jira/browse/YARN-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891499#comment-13891499 ] Wangda Tan commented on YARN-1444: -- Hi [~acmurthy], I think the patch I attached before should work, I hope get some reviews about this. Sorry for mis-understanding caused by my last comment. RM crashes when node resource request sent without corresponding rack request - Key: YARN-1444 URL: https://issues.apache.org/jira/browse/YARN-1444 Project: Hadoop YARN Issue Type: Bug Components: client, resourcemanager Reporter: Robert Grandl Assignee: Wangda Tan Priority: Blocker Attachments: yarn-1444.ver1.patch I have tried to force reducers to execute on certain nodes. What I did is I changed for reduce tasks, the RMContainerRequestor#addResourceRequest(req.priority, ResourceRequest.ANY, req.capability) to RMContainerRequestor#addResourceRequest(req.priority, HOST_NAME, req.capability). However, this change lead to RM crashes when reducers needs to be assigned with the following exception: FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling event type NODE_UPDATE to the scheduler java.lang.NullPointerException at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:841) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:640) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:554) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:695) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:739) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:86) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:549) at java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891503#comment-13891503
]
Hadoop QA commented on YARN-1499:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626995/YARN-1499-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3014//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3014//console
This message is automatically generated.
Fair Scheduler changes for moving apps between queues
-
Key: YARN-1499
URL: https://issues.apache.org/jira/browse/YARN-1499
Project: Hadoop YARN
Issue Type: Sub-task
Components: scheduler
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1499-1.patch, YARN-1499-2.patch, YARN-1499.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891506#comment-13891506 ] Vinod Kumar Vavilapalli commented on YARN-1634: --- Thanks, that'll be all. +1, Checking this in. Define an in-memory implementation of ApplicationTimelineStore -- Key: YARN-1634 URL: https://issues.apache.org/jira/browse/YARN-1634 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1634.1.patch, YARN-1634.2.patch, YARN-1634.3.patch As per the design doc, the store needs to pluggable. We need a base interface, and an in-memory implementation for testing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1660) add the ability to set yarn.resourcemanager.hostname.rm-id instead of setting all the various host:port properties for RM
[
https://issues.apache.org/jira/browse/YARN-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891551#comment-13891551
]
Karthik Kambatla commented on YARN-1660:
Just skimmed through the patch. Few comments:
# We should may be move getRMDefaultPortNumber to YarnConfiguration
# Didn't quite understand why we need the following change:
{code}
static String getConfKeyForRMInstance(String prefix, Configuration conf) {
-return YarnConfiguration.RM_SERVICES_ADDRESS_CONF_KEYS.contains(prefix)
-? addSuffix(prefix, getRMHAId(conf))
-: prefix;
+if (! YarnConfiguration.RM_SERVICES_ADDRESS_CONF_KEYS.contains(prefix)) {
+ return prefix;
+} else {
+ String RMId = getRMHAId(conf);
+ checkAndSetRMRPCAddress(prefix, RMId, conf);
+ return addSuffix(prefix, RMId);
+}
{code}
# I would use two different variables for the confKey corresponding to the
RPC-address conf, and the confKey for hostname.
{code}
+ private static void checkAndSetRMRPCAddress(String prefix, String RMId,
+ Configuration conf) {
+String confKey = null;
+try {
+ confKey = addSuffix(prefix, RMId);
+ if (conf.getTrimmed(confKey) == null) {
+confKey = addSuffix(YarnConfiguration.RM_HOSTNAME, RMId);
+String confVal = conf.getTrimmed(confKey);
+if (confVal == null) {
+ throwBadConfigurationException(getNeedToSetValueMessage(confKey
+ + or + addSuffix(prefix, RMId)));
+} else {
+ conf.set(addSuffix(prefix, RMId), confVal + :
+ + getRMDefaultPortNumber(prefix));
+}
+ }
{code}
add the ability to set yarn.resourcemanager.hostname.rm-id instead of setting
all the various host:port properties for RM
-
Key: YARN-1660
URL: https://issues.apache.org/jira/browse/YARN-1660
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Reporter: Arpit Gupta
Assignee: Xuan Gong
Attachments: YARN-1660.1.patch
Currently the user has to specify all the various host:port properties for
RM. We should follow the pattern that we do for non HA setup where we can
specify yarn.resourcemanager.hostname.rm-id and the defaults are used for all
other affected properties.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+eventies
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1636: -- Attachment: YARN-1636.2.patch Upload a new patch to rebase against the latest trunk. Implement timeline related web-services inside AHS for storing and retrieving entities+eventies --- Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1634) Define an in-memory implementation of ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891556#comment-13891556 ] Hudson commented on YARN-1634: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5105 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5105/]) YARN-1634. Added a testable in-memory implementation of ApplicationTimelineStore. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564583) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/apptimeline/ATSEntity.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/EntityId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/MemoryApplicationTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/ApplicationTimelineStoreTestUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/apptimeline/TestMemoryApplicationTimelineStore.java Define an in-memory implementation of ApplicationTimelineStore -- Key: YARN-1634 URL: https://issues.apache.org/jira/browse/YARN-1634 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Fix For: 2.4.0 Attachments: YARN-1634.1.patch, YARN-1634.2.patch, YARN-1634.3.patch As per the design doc, the store needs to pluggable. We need a base interface, and an in-memory implementation for testing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+eventies
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891555#comment-13891555 ] Zhijie Shen commented on YARN-1636: --- Test cases are also included Implement timeline related web-services inside AHS for storing and retrieving entities+eventies --- Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+eventes
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-1636: -- Summary: Implement timeline related web-services inside AHS for storing and retrieving entities+eventes (was: Implement timeline related web-services inside AHS for storing and retrieving entities+eventies) Implement timeline related web-services inside AHS for storing and retrieving entities+eventes -- Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-1636: -- Summary: Implement timeline related web-services inside AHS for storing and retrieving entities+events (was: Implement timeline related web-services inside AHS for storing and retrieving entities+eventes) Implement timeline related web-services inside AHS for storing and retrieving entities+events - Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1531) Update yarn command document
[
https://issues.apache.org/jira/browse/YARN-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891570#comment-13891570
]
Karthik Kambatla commented on YARN-1531:
Thanks Akira. Sorry for the delay in getting around to this. Patch looks mostly
good. I would leave out the following two lines, as yarn rmadmin -failover
isn't really supported currently.
{code}
+ [-failover [--forcefence] [--forceactive] serviceId
serviceId]
...
+| -failover [--forcefence] [--forceactive] \serviceId\ \serviceId\ |
+| | Failover from the first service to the second. Unconditionally fence
+| | services if the --forcefence option is used. Try to failover to the target
+| | service even if it is not ready if the --forceactive option is used.
{code}
Update yarn command document
Key: YARN-1531
URL: https://issues.apache.org/jira/browse/YARN-1531
Project: Hadoop YARN
Issue Type: Bug
Components: documentation
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Labels: documentaion
Attachments: YARN-1531.2.patch, YARN-1531.patch
There are some options which are not written to Yarn Command document.
For example, yarn rmadmin command options are as follows:
{code}
Usage: yarn rmadmin
-refreshQueues
-refreshNodes
-refreshSuperUserGroupsConfiguration
-refreshUserToGroupsMappings
-refreshAdminAcls
-refreshServiceAcl
-getGroups [username]
-help [cmd]
-transitionToActive serviceId
-transitionToStandby serviceId
-failover [--forcefence] [--forceactive] serviceId serviceId
-getServiceState serviceId
-checkHealth serviceId
{code}
But some of the new options such as -getGroups, -transitionToActive, and
-transitionToStandby are not documented.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[
https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891575#comment-13891575
]
Hadoop QA commented on YARN-1497:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627000/YARN-1497-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client:
org.apache.hadoop.yarn.client.api.impl.TestNMClient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3013//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3013//console
This message is automatically generated.
Expose moving apps between queues on the command line
-
Key: YARN-1497
URL: https://issues.apache.org/jira/browse/YARN-1497
Project: Hadoop YARN
Issue Type: Sub-task
Components: client
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1531) Update yarn command document
[
https://issues.apache.org/jira/browse/YARN-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated YARN-1531:
Attachment: YARN-1531.3.patch
Thank you for the review, [~kkambatl]!
Attaching an updated patch.
Update yarn command document
Key: YARN-1531
URL: https://issues.apache.org/jira/browse/YARN-1531
Project: Hadoop YARN
Issue Type: Bug
Components: documentation
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Labels: documentaion
Attachments: YARN-1531.2.patch, YARN-1531.3.patch, YARN-1531.patch
There are some options which are not written to Yarn Command document.
For example, yarn rmadmin command options are as follows:
{code}
Usage: yarn rmadmin
-refreshQueues
-refreshNodes
-refreshSuperUserGroupsConfiguration
-refreshUserToGroupsMappings
-refreshAdminAcls
-refreshServiceAcl
-getGroups [username]
-help [cmd]
-transitionToActive serviceId
-transitionToStandby serviceId
-failover [--forcefence] [--forceactive] serviceId serviceId
-getServiceState serviceId
-checkHealth serviceId
{code}
But some of the new options such as -getGroups, -transitionToActive, and
-transitionToStandby are not documented.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1497) Expose moving apps between queues on the command line
[ https://issues.apache.org/jira/browse/YARN-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891581#comment-13891581 ] Sandy Ryza commented on YARN-1497: -- The javadoc warnings appear to be unrelated Expose moving apps between queues on the command line - Key: YARN-1497 URL: https://issues.apache.org/jira/browse/YARN-1497 Project: Hadoop YARN Issue Type: Sub-task Components: client Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1497-1.patch, YARN-1497.patch, YARN-1497.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891578#comment-13891578 ] Sandy Ryza commented on YARN-1499: -- The javadoc warnings appear to be unrelated Fair Scheduler changes for moving apps between queues - Key: YARN-1499 URL: https://issues.apache.org/jira/browse/YARN-1499 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1499-1.patch, YARN-1499-2.patch, YARN-1499.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[
https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891604#comment-13891604
]
Hadoop QA commented on YARN-1636:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627044/YARN-1636.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice:
org.apache.hadoop.yarn.server.applicationhistoryservice.TestApplicationHistoryServer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3017//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3017//console
This message is automatically generated.
Implement timeline related web-services inside AHS for storing and retrieving
entities+events
-
Key: YARN-1636
URL: https://issues.apache.org/jira/browse/YARN-1636
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
Attachments: YARN-1636.1.patch, YARN-1636.2.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[
https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891610#comment-13891610
]
Hadoop QA commented on YARN-1461:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627038/yarn-1461-12.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3016//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3016//console
This message is automatically generated.
RM API and RM changes to handle tags for running jobs
-
Key: YARN-1461
URL: https://issues.apache.org/jira/browse/YARN-1461
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: yarn-1461-1.patch, yarn-1461-10.patch,
yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-2.patch, yarn-1461-3.patch,
yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch,
yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1531) Update yarn command document
[
https://issues.apache.org/jira/browse/YARN-1531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891625#comment-13891625
]
Hadoop QA commented on YARN-1531:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627047/YARN-1531.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3018//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3018//console
This message is automatically generated.
Update yarn command document
Key: YARN-1531
URL: https://issues.apache.org/jira/browse/YARN-1531
Project: Hadoop YARN
Issue Type: Bug
Components: documentation
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Labels: documentaion
Attachments: YARN-1531.2.patch, YARN-1531.3.patch, YARN-1531.patch
There are some options which are not written to Yarn Command document.
For example, yarn rmadmin command options are as follows:
{code}
Usage: yarn rmadmin
-refreshQueues
-refreshNodes
-refreshSuperUserGroupsConfiguration
-refreshUserToGroupsMappings
-refreshAdminAcls
-refreshServiceAcl
-getGroups [username]
-help [cmd]
-transitionToActive serviceId
-transitionToStandby serviceId
-failover [--forcefence] [--forceactive] serviceId serviceId
-getServiceState serviceId
-checkHealth serviceId
{code}
But some of the new options such as -getGroups, -transitionToActive, and
-transitionToStandby are not documented.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891651#comment-13891651
]
Karthik Kambatla commented on YARN-1499:
Looks good to me, but for one nits. Okay with fixing it at commit time.
# The variable name is still around.
{code}
+FSQueue leastCommonAncestor = findLowestCommonAncestorQueue(oldQueue,
{code}
Fair Scheduler changes for moving apps between queues
-
Key: YARN-1499
URL: https://issues.apache.org/jira/browse/YARN-1499
Project: Hadoop YARN
Issue Type: Sub-task
Components: scheduler
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1499-1.patch, YARN-1499-2.patch, YARN-1499.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1499) Fair Scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891658#comment-13891658 ] Karthik Kambatla commented on YARN-1499: +1 after the nit. Fair Scheduler changes for moving apps between queues - Key: YARN-1499 URL: https://issues.apache.org/jira/browse/YARN-1499 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1499-1.patch, YARN-1499-2.patch, YARN-1499.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1461: --- Attachment: yarn-1461-13.patch Fix javadoc warning. RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-13.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891672#comment-13891672 ] Jian He commented on YARN-1206: --- I remember this was working on branch-2, but failed on trunk, it should be fine Container logs link is broken on RM web UI after application finished - Key: YARN-1206 URL: https://issues.apache.org/jira/browse/YARN-1206 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Priority: Blocker With log aggregation disabled, when container is running, its logs link works properly, but after the application is finished, the link shows 'Container does not exist.' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1577) Unmanaged AM is broken because of YARN-1493
[ https://issues.apache.org/jira/browse/YARN-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891674#comment-13891674 ] Jian He commented on YARN-1577: --- This can be quickly fixed by just exposing the attempt state to the user, we may just fix it ? Unmanaged AM is broken because of YARN-1493 --- Key: YARN-1577 URL: https://issues.apache.org/jira/browse/YARN-1577 Project: Hadoop YARN Issue Type: Sub-task Affects Versions: 2.3.0 Reporter: Jian He Assignee: Jian He Priority: Blocker Today unmanaged AM client is waiting for app state to be Accepted to launch the AM. This is broken since we changed in YARN-1493 to start the attempt after the application is Accepted. We may need to introduce an attempt state report that client can rely on to query the attempt state and choose to launch the unmanaged AM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1636: -- Attachment: YARN-1636.3.patch Upload a new patch that fixes the test failure Implement timeline related web-services inside AHS for storing and retrieving entities+events - Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch, YARN-1636.3.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891694#comment-13891694 ] Vinod Kumar Vavilapalli commented on YARN-1636: --- Patch looks fine overall. Couple of minor things - ApplicationHistoryServer.createApplicationTimeStore() - createApplicationTimelineStore. - Changes in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/pom.xml seem unnecessary. Implement timeline related web-services inside AHS for storing and retrieving entities+events - Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch, YARN-1636.3.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[
https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891696#comment-13891696
]
Hadoop QA commented on YARN-1461:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627063/yarn-1461-13.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3019//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3019//console
This message is automatically generated.
RM API and RM changes to handle tags for running jobs
-
Key: YARN-1461
URL: https://issues.apache.org/jira/browse/YARN-1461
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: yarn-1461-1.patch, yarn-1461-10.patch,
yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-13.patch,
yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch,
yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch,
yarn-1461-9.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1459) Handle supergroups, usergroups and ACLs across RMs during failover
[ https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1459: Attachment: YARN-1459.2.patch Handle supergroups, usergroups and ACLs across RMs during failover -- Key: YARN-1459 URL: https://issues.apache.org/jira/browse/YARN-1459 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Xuan Gong Attachments: YARN-1459.1.patch, YARN-1459.2.patch The supergroups, usergroups and ACL configurations are per RM and might have been changed while the RM is running. After failing over, the new Active RM should have the latest configuration from the previously Active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1459) Handle supergroups, usergroups and ACLs across RMs during failover
[ https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891697#comment-13891697 ] Xuan Gong commented on YARN-1459: - In this patch, we will make sure that we load the yarn-site.xml and core-site.xml from remoteDir if configurationProvider is not LocalConfigurationProvider before we initiate the RM. The scheduler#reinit() will make sure that it will load capacity-scheduler.xml from remoteDir (Will take care of FS in https://issues.apache.org/jira/browse/YARN-1679). If the security is enable, the AdminServicer, clientRMService, applicationMasterService and ResourceTrackerServicer#serviceInit() can make sure that they will load the hadoop-policy.xml from remoteDir. Handle supergroups, usergroups and ACLs across RMs during failover -- Key: YARN-1459 URL: https://issues.apache.org/jira/browse/YARN-1459 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Xuan Gong Attachments: YARN-1459.1.patch, YARN-1459.2.patch The supergroups, usergroups and ACL configurations are per RM and might have been changed while the RM is running. After failing over, the new Active RM should have the latest configuration from the previously Active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[
https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891698#comment-13891698
]
Hadoop QA commented on YARN-1636:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627070/YARN-1636.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3020//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3020//console
This message is automatically generated.
Implement timeline related web-services inside AHS for storing and retrieving
entities+events
-
Key: YARN-1636
URL: https://issues.apache.org/jira/browse/YARN-1636
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
Attachments: YARN-1636.1.patch, YARN-1636.2.patch, YARN-1636.3.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[ https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-1636: -- Attachment: YARN-1636.4.patch Vinod, thanks for your review. Patch updated Implement timeline related web-services inside AHS for storing and retrieving entities+events - Key: YARN-1636 URL: https://issues.apache.org/jira/browse/YARN-1636 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Zhijie Shen Attachments: YARN-1636.1.patch, YARN-1636.2.patch, YARN-1636.3.patch, YARN-1636.4.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891795#comment-13891795 ] Vinod Kumar Vavilapalli commented on YARN-1461: --- Tx Karthik. Looks fine overall. I don't like the names startRange, finishRange. But that isn't caused by your patch. We can change them separately. Also, for some reason, the setters are all marked private unnecessarily. Again unrelated, will file a ticket. RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-13.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891796#comment-13891796 ] Zhijie Shen commented on YARN-1461: --- I'm going to commit the patch. RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-13.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891803#comment-13891803 ] Zhijie Shen commented on YARN-1461: --- Committed to trunk and branch-2. Thanks, [~kkambatl]! Also, thanks [~vinodkv] for reviewing the patch! RM API and RM changes to handle tags for running jobs - Key: YARN-1461 URL: https://issues.apache.org/jira/browse/YARN-1461 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Fix For: 2.4.0 Attachments: yarn-1461-1.patch, yarn-1461-10.patch, yarn-1461-11.patch, yarn-1461-12.patch, yarn-1461-13.patch, yarn-1461-2.patch, yarn-1461-3.patch, yarn-1461-4.patch, yarn-1461-5.patch, yarn-1461-6.patch, yarn-1461-6.patch, yarn-1461-7.patch, yarn-1461-8.patch, yarn-1461-9.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891805#comment-13891805 ] Billie Rinaldi commented on YARN-1635: -- I have a new patch ready addressing most of the comments, but haven't attached it yet. This patch applies on top of YARN-1636. I've added more comments to GenericObjectMapper but haven't renamed it yet. The name comes from ObjectMapper, since it does ObjectMapper-like things without making you specify the class to deserialize into. I could see renaming it to ATSUtil or LeveldbApplicationTimelineStoreUtil. Let me know what you think after seeing the additional comments. Implement a Leveldb based ApplicationTimelineStore -- Key: YARN-1635 URL: https://issues.apache.org/jira/browse/YARN-1635 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Billie Rinaldi Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch, YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch As per the design doc, we need a levelDB + local-filesystem based implementation to start with and for small deployments. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1636) Implement timeline related web-services inside AHS for storing and retrieving entities+events
[
https://issues.apache.org/jira/browse/YARN-1636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891804#comment-13891804
]
Hadoop QA commented on YARN-1636:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627074/YARN-1636.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3021//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3021//console
This message is automatically generated.
Implement timeline related web-services inside AHS for storing and retrieving
entities+events
-
Key: YARN-1636
URL: https://issues.apache.org/jira/browse/YARN-1636
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Vinod Kumar Vavilapalli
Assignee: Zhijie Shen
Attachments: YARN-1636.1.patch, YARN-1636.2.patch, YARN-1636.3.patch,
YARN-1636.4.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1461) RM API and RM changes to handle tags for running jobs
[ https://issues.apache.org/jira/browse/YARN-1461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891807#comment-13891807 ] Hudson commented on YARN-1461: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5107 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5107/]) YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. (zjshen: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1564633) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/ApplicationsRequestScope.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/GetApplicationsRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationReport.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/GetApplicationsRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationReportPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationSubmissionContextPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/utils/BuilderUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationsmanager/MockAsm.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/MockRMApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesApps.java *
[jira] [Updated] (YARN-1635) Implement a Leveldb based ApplicationTimelineStore
[ https://issues.apache.org/jira/browse/YARN-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-1635: - Attachment: YARN-1635.7.patch Implement a Leveldb based ApplicationTimelineStore -- Key: YARN-1635 URL: https://issues.apache.org/jira/browse/YARN-1635 Project: Hadoop YARN Issue Type: Sub-task Reporter: Vinod Kumar Vavilapalli Assignee: Billie Rinaldi Attachments: YARN-1635.1.patch, YARN-1635.2.patch, YARN-1635.3.patch, YARN-1635.4.patch, YARN-1635.5.patch, YARN-1635.6.patch, YARN-1635.7.patch As per the design doc, we need a levelDB + local-filesystem based implementation to start with and for small deployments. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1483) is ${hadoop.tmp.dir} a hdfs dir or a linux local dir
[
https://issues.apache.org/jira/browse/YARN-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891819#comment-13891819
]
Akira AJISAKA commented on YARN-1483:
-
I think ${hadoop.tmp.dir} is a local dir in both hadoop 1.x and 2.x.
Closing this is as invalid.
https://wiki.apache.org/hadoop/InvalidJiraIssues
If you have a question, ask on mailing lists.
is ${hadoop.tmp.dir} a hdfs dir or a linux local dir
Key: YARN-1483
URL: https://issues.apache.org/jira/browse/YARN-1483
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: prophy Yan
in hadoop 1.1.1, the ${hadoop.tmp.dir} is a hdfs dir.
recently, i upgrade my hadoop cluster from 1.1.1 to 2.2.0, on official
yarn-default.xml configuration page. the property
'yarn.nodemanager.local-dirs' value is '${hadoop.tmp.dir}/nm-local-dir', but
i haven't seen this dir in hdfs. i see it in the linux local /tmp dir.
so is this a bug?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1577) Unmanaged AM is broken because of YARN-1493
[
https://issues.apache.org/jira/browse/YARN-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891853#comment-13891853
]
Alejandro Abdelnur commented on YARN-1577:
--
the problem I'm seeing with YARN-1493 is when trying to register the UAM with
the scheduler, the exception I'm getting follows.
Reverting YARN-1493 friends makes this problem go away.
The patches to revert, in order are:
YARN-1566
YARN-1041
YARN-1166
YARN-1490
YARN-1493
{code}
2014-02-03 10:58:40,403 ERROR UserGroupInformation - PriviledgedActionException
as:llama (auth:PROXY) via tucu (auth:SIMPLE)
cause:org.apache.hadoop.security.AccessControlException: SIMPLE authentication
is not enabled. Available:[TOKEN]
2014-02-03 10:58:40,407 WARN LlamaAMServiceImpl - Reserve() error:
com.cloudera.llama.util.LlamaException: AM_CANNOT_REGISTER - cannot register AM
'application_1391453743418_0001' for queue 'root.queue1' :
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not
enabled. Available:[TOKEN]
com.cloudera.llama.util.LlamaException: AM_CANNOT_REGISTER - cannot register AM
'application_1391453743418_0001' for queue 'root.queue1' :
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not
enabled. Available:[TOKEN]
…
Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE
authentication is not enabled. Available:[TOKEN]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
at
org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at $Proxy12.registerApplicationMaster(Unknown Source)
at
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:196)
at
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138)
at
com.cloudera.llama.am.yarn.YarnRMConnector._initYarnApp(YarnRMConnector.java:270)
at
com.cloudera.llama.am.yarn.YarnRMConnector.access$200(YarnRMConnector.java:80)
at
com.cloudera.llama.am.yarn.YarnRMConnector$3.run(YarnRMConnector.java:212)
at
com.cloudera.llama.am.yarn.YarnRMConnector$3.run(YarnRMConnector.java:209)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
com.cloudera.llama.am.yarn.YarnRMConnector.register(YarnRMConnector.java:209)
... 20 more
Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
SIMPLE authentication is not enabled. Available:[TOKEN]
at org.apache.hadoop.ipc.Client.call(Client.java:1406)
at org.apache.hadoop.ipc.Client.call(Client.java:1359)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at $Proxy11.registerApplicationMaster(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
... 37 more
{code}
Unmanaged AM is broken because of YARN-1493
---
Key: YARN-1577
URL: https://issues.apache.org/jira/browse/YARN-1577
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: 2.3.0
Reporter: Jian He
Assignee: Jian He
Priority: Blocker
Today unmanaged AM client is waiting for app state to be Accepted to launch
the AM. This is broken since we changed in YARN-1493 to start the attempt
after the application is Accepted. We may need to introduce an attempt state
report that
[jira] [Updated] (YARN-1459) Handle supergroups, usergroups and ACLs across RMs during failover
[ https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-1459: Attachment: YARN-1459.3.patch fix test cases failure Handle supergroups, usergroups and ACLs across RMs during failover -- Key: YARN-1459 URL: https://issues.apache.org/jira/browse/YARN-1459 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Xuan Gong Attachments: YARN-1459.1.patch, YARN-1459.2.patch, YARN-1459.3.patch The supergroups, usergroups and ACL configurations are per RM and might have been changed while the RM is running. After failing over, the new Active RM should have the latest configuration from the previously Active RM. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1413) [YARN-321] AHS WebUI should server aggregated logs as well
[ https://issues.apache.org/jira/browse/YARN-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891884#comment-13891884 ] Mayank Bansal commented on YARN-1413: - [~zjshen] Thanks for the comments. As discussed opening another jira for the followup items and closing this. Thanks, Mayank [YARN-321] AHS WebUI should server aggregated logs as well -- Key: YARN-1413 URL: https://issues.apache.org/jira/browse/YARN-1413 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Mayank Bansal Fix For: YARN-321 Attachments: YARN-1413-1.patch, YARN-1413-2.patch, YARN-1413-3.patch, YARN-1413-4.patch, YARN-1413-5.patch, YARN-1413-6.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1685) [YARN-321] Logs link can be null so avoid NPE
Mayank Bansal created YARN-1685: --- Summary: [YARN-321] Logs link can be null so avoid NPE Key: YARN-1685 URL: https://issues.apache.org/jira/browse/YARN-1685 Project: Hadoop YARN Issue Type: Sub-task Reporter: Mayank Bansal Assignee: Mayank Bansal Fix For: YARN-321 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Resolved] (YARN-1413) [YARN-321] AHS WebUI should server aggregated logs as well
[ https://issues.apache.org/jira/browse/YARN-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal resolved YARN-1413. - Resolution: Fixed [YARN-321] AHS WebUI should server aggregated logs as well -- Key: YARN-1413 URL: https://issues.apache.org/jira/browse/YARN-1413 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Mayank Bansal Fix For: YARN-321 Attachments: YARN-1413-1.patch, YARN-1413-2.patch, YARN-1413-3.patch, YARN-1413-4.patch, YARN-1413-5.patch, YARN-1413-6.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1685) [YARN-321] Logs link can be null so avoid NPE
[ https://issues.apache.org/jira/browse/YARN-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal updated YARN-1685: Description: https://issues.apache.org/jira/browse/YARN-1413?focusedCommentId=13866844page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13866844 [YARN-321] Logs link can be null so avoid NPE - Key: YARN-1685 URL: https://issues.apache.org/jira/browse/YARN-1685 Project: Hadoop YARN Issue Type: Sub-task Reporter: Mayank Bansal Assignee: Mayank Bansal Fix For: YARN-321 https://issues.apache.org/jira/browse/YARN-1413?focusedCommentId=13866844page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13866844 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1459) Handle supergroups, usergroups and ACLs across RMs during failover
[
https://issues.apache.org/jira/browse/YARN-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891899#comment-13891899
]
Hadoop QA commented on YARN-1459:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627088/YARN-1459.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3023//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/3023//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3023//console
This message is automatically generated.
Handle supergroups, usergroups and ACLs across RMs during failover
--
Key: YARN-1459
URL: https://issues.apache.org/jira/browse/YARN-1459
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Karthik Kambatla
Assignee: Xuan Gong
Attachments: YARN-1459.1.patch, YARN-1459.2.patch, YARN-1459.3.patch
The supergroups, usergroups and ACL configurations are per RM and might have
been changed while the RM is running. After failing over, the new Active RM
should have the latest configuration from the previously Active RM.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
