[jira] [Created] (YARN-2808) yarn client tool can not list app_attempt's container info correctly
Gordon Wang created YARN-2808:
-
Summary: yarn client tool can not list app_attempt's container
info correctly
Key: YARN-2808
URL: https://issues.apache.org/jira/browse/YARN-2808
Project: Hadoop YARN
Issue Type: Bug
Components: client
Reporter: Gordon Wang
When enabling timeline server, yarn client can not list the container info for
a application attempt correctly.
Here is the reproduce step.
# enabling yarn timeline server
# submit a MR job
# after the job is finished. use yarn client to list the container info of the
app attempt.
Then, since the RM has cached the application's attempt info, the output show
{noformat}
[hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list
appattempt_1415168250217_0001_01
14/11/05 01:19:15 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/11/05 01:19:15 INFO impl.TimelineClientImpl: Timeline service address:
http://0.0.0.0:8188/ws/v1/timeline/
14/11/05 01:19:16 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
14/11/05 01:19:16 INFO client.AHSProxy: Connecting to Application History
server at /0.0.0.0:10200
Total number of containers :0
Container-IdStart Time Finish Time
StateHost
LOG-URL
{noformat}
But if the rm is restarted, client can fetch the container info from timeline
server correctly.
{noformat}
[hadoop@localhost hadoop-3.0.0-SNAPSHOT]$ ./bin/yarn container -list
appattempt_1415168250217_0001_01
14/11/05 01:21:06 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/11/05 01:21:06 INFO impl.TimelineClientImpl: Timeline service address:
http://0.0.0.0:8188/ws/v1/timeline/
14/11/05 01:21:06 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
14/11/05 01:21:06 INFO client.AHSProxy: Connecting to Application History
server at /0.0.0.0:10200
Total number of containers :4
Container-IdStart Time Finish Time
StateHost
LOG-URL
container_1415168250217_0001_01_01 1415168318376
1415168349896COMPLETElocalhost.localdomain:47024
http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_01/container_1415168250217_0001_01_01/hadoop
container_1415168250217_0001_01_02 1415168326399
1415168334858COMPLETElocalhost.localdomain:47024
http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_02/container_1415168250217_0001_01_02/hadoop
container_1415168250217_0001_01_03 1415168326400
1415168335277COMPLETElocalhost.localdomain:47024
http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_03/container_1415168250217_0001_01_03/hadoop
container_1415168250217_0001_01_04 1415168335825
1415168343873COMPLETElocalhost.localdomain:47024
http://0.0.0.0:8188/applicationhistory/logs/localhost.localdomain:47024/container_1415168250217_0001_01_04/container_1415168250217_0001_01_04/hadoop
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (YARN-1951) Change root queue default ACL to NOBODY_ACL in yarn scheduler
[ https://issues.apache.org/jira/browse/YARN-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gordon Wang resolved YARN-1951. --- Resolution: Won't Fix Per discussion with [~sandyr] in YARN-1941. The cost of breaking existing setups outweighs the benefits of changing this. Let's keep the root queue default ACL. > Change root queue default ACL to NOBODY_ACL in yarn scheduler > - > > Key: YARN-1951 > URL: https://issues.apache.org/jira/browse/YARN-1951 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler >Affects Versions: 2.3.0 >Reporter: Gordon Wang > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (YARN-1951) Change root queue default ACL to NOBODY_ACL in yarn scheduler
Gordon Wang created YARN-1951: - Summary: Change root queue default ACL to NOBODY_ACL in yarn scheduler Key: YARN-1951 URL: https://issues.apache.org/jira/browse/YARN-1951 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Affects Versions: 2.3.0 Reporter: Gordon Wang -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (YARN-1950) Do not check ACL in Yarn scheduler when Yarn ACL is disabled
Gordon Wang created YARN-1950: - Summary: Do not check ACL in Yarn scheduler when Yarn ACL is disabled Key: YARN-1950 URL: https://issues.apache.org/jira/browse/YARN-1950 Project: Hadoop YARN Issue Type: Bug Components: scheduler Affects Versions: 2.3.0 Reporter: Gordon Wang -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (YARN-1941) Yarn scheduler ACL improvement
Gordon Wang created YARN-1941: - Summary: Yarn scheduler ACL improvement Key: YARN-1941 URL: https://issues.apache.org/jira/browse/YARN-1941 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 2.3.0 Reporter: Gordon Wang Defect: 1. Currently, in Yarn Capacity Scheduler and Yarn Fair Scheduler, the queue ACL is always checked when submitting a app to scheduler, regardless of the property "yarn.acl.enable". But for killing an app, the ACL is checked when yarn.acl.enable is set. The behaviour is not consistent. 2. default ACL for root queue is EVERYBODY_ACL(*), while default ACL for other queues is NODODY_ACL( ). From users' view, this is error prone and not easy to understand the ACL policy of Yarn scheduler. root queue should not be so special compared with other parent queues. For example, if I want to set capacity scheduler ACL, the ACL of root has to be set explicitly. Otherwise, everyone can submit APP to yarn scheduler. Because root queue ACL is EVERYBODY_ACL. This is hard for user to administrate yarn scheduler. So, I propose to improve the ACL of yarn scheduler in the following aspects. 1. only enable scheduler queue ACL when yarn.acl.enable is set to true. 2. set the default ACL of root queue as NOBODY_ACL( ). Make all the parent queues' ACL consistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
