[jira] [Created] (YARN-11606) Upgrade fst to 2.57
D M Murali Krishna Reddy created YARN-11606: --- Summary: Upgrade fst to 2.57 Key: YARN-11606 URL: https://issues.apache.org/jira/browse/YARN-11606 Project: Hadoop YARN Issue Type: Bug Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy de.ruedigermoeller:fst has a dependency of json-io:2.5.1 which is vulnerable with CVE-2023-34610, Higher versions of fst do not have any dependency with json-io. [https://nvd.nist.gov/vuln/detail/CVE-2023-34610] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11389) Upgrade spring-core to 5.3.20 in wro4j-maven-plugin
[ https://issues.apache.org/jira/browse/YARN-11389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-11389: Component/s: build (was: buid) > Upgrade spring-core to 5.3.20 in wro4j-maven-plugin > > > Key: YARN-11389 > URL: https://issues.apache.org/jira/browse/YARN-11389 > Project: Hadoop YARN > Issue Type: Improvement > Components: build, yarn-ui-v2 >Affects Versions: 3.4.0 >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Labels: transitive-cve > > Currently during yarn-ui build we are using vulnerable version of > spring-core-3.1.1.RELEASE.jar which has serveral critical and high > vulnerablilites, we need to upgrade to a version 5.3.20+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11389) Upgrade spring-core to 5.3.20 in wro4j-maven-plugin
D M Murali Krishna Reddy created YARN-11389: --- Summary: Upgrade spring-core to 5.3.20 in wro4j-maven-plugin Key: YARN-11389 URL: https://issues.apache.org/jira/browse/YARN-11389 Project: Hadoop YARN Issue Type: Improvement Components: yarn-ui-v2 Affects Versions: 3.4.0 Reporter: D M Murali Krishna Reddy Currently during yarn-ui build we are using vulnerable version of spring-core-3.1.1.RELEASE.jar which has serveral critical and high vulnerablilites, we need to upgrade to a version 5.3.20+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11389) Upgrade spring-core to 5.3.20 in wro4j-maven-plugin
[ https://issues.apache.org/jira/browse/YARN-11389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-11389: --- Assignee: D M Murali Krishna Reddy > Upgrade spring-core to 5.3.20 in wro4j-maven-plugin > > > Key: YARN-11389 > URL: https://issues.apache.org/jira/browse/YARN-11389 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn-ui-v2 >Affects Versions: 3.4.0 >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > > Currently during yarn-ui build we are using vulnerable version of > spring-core-3.1.1.RELEASE.jar which has serveral critical and high > vulnerablilites, we need to upgrade to a version 5.3.20+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11303) Upgrade jquery ui to 1.13.2
D M Murali Krishna Reddy created YARN-11303: --- Summary: Upgrade jquery ui to 1.13.2 Key: YARN-11303 URL: https://issues.apache.org/jira/browse/YARN-11303 Project: Hadoop YARN Issue Type: Improvement Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy The current jquery-ui version used(1.13.1) in the trunk has the following vulnerability [CVE-2022-31160|https://nvd.nist.gov/vuln/detail/CVE-2022-31160] so we need to upgrade to at least 1.13.2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11092) Upgrade jquery ui to 1.13.1
[ https://issues.apache.org/jira/browse/YARN-11092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528335#comment-17528335 ] D M Murali Krishna Reddy commented on YARN-11092: - [~groot] , you can take up this task. > Upgrade jquery ui to 1.13.1 > --- > > Key: YARN-11092 > URL: https://issues.apache.org/jira/browse/YARN-11092 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > > The current jquery-ui version used(1.12.1) in the trunk has the following > vulnerabilities CVE-2021-41182, CVE-2021-41183, CVE-2021-41184, so we need to > upgrade to at least 1.13.0. > > Also currently for the UI2 we are using the shims repo which is not being > maintained as per the discussion > [https://github.com/components/jqueryui/issues/70] , so if possible we should > move to the main jquery repo [https://github.com/jquery/jquery-ui] -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11092) Upgrade jquery ui to 1.13.1
D M Murali Krishna Reddy created YARN-11092: --- Summary: Upgrade jquery ui to 1.13.1 Key: YARN-11092 URL: https://issues.apache.org/jira/browse/YARN-11092 Project: Hadoop YARN Issue Type: Improvement Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy The current jquery-ui version used(1.12.1) in the trunk has the following vulnerabilities CVE-2021-41182, CVE-2021-41183, CVE-2021-41184, so we need to upgrade to at least 1.13.0. Also currently for the UI2 we are using the shims repo which is not being maintained as per the discussion [https://github.com/components/jqueryui/issues/70] , so if possible we should move to the main jquery repo [https://github.com/jquery/jquery-ui] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7982) Do ACLs check while retrieving entity-types per application
[
https://issues.apache.org/jira/browse/YARN-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447080#comment-17447080
]
D M Murali Krishna Reddy commented on YARN-7982:
[~prabhujoseph] the above [^YARN-7982-004.patch] applies cleanly to the 3.2
branch, you can merge it directly. Thanks.
> Do ACLs check while retrieving entity-types per application
> ---
>
> Key: YARN-7982
> URL: https://issues.apache.org/jira/browse/YARN-7982
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-7982-001.patch, YARN-7982-002.patch,
> YARN-7982-003.patch, YARN-7982-004.patch
>
>
> REST end point {{/apps/$appid/entity-types}} retrieves all the entity-types
> for given application. This need to be guarded with ACL check
> {code}
> [yarn@yarn-ats-3 ~]$ curl
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1552297011473_0002?user.name=ambari-qa1";
> {"exception":"ForbiddenException","message":"java.lang.Exception: User
> ambari-qa1 is not allowed to read TimelineService V2
> data.","javaClassName":"org.apache.hadoop.yarn.webapp.ForbiddenException"}
> [yarn@yarn-ats-3 ~]$ curl
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1552297011473_0002/entity-types?user.name=ambari-qa1";
> ["YARN_APPLICATION_ATTEMPT","YARN_CONTAINER"]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7982) Do ACLs check while retrieving entity-types per application
[
https://issues.apache.org/jira/browse/YARN-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426981#comment-17426981
]
D M Murali Krishna Reddy commented on YARN-7982:
[~prabhujoseph], [~abmodi] Can we backport this to 3.2 branch as well.
Thanks.
> Do ACLs check while retrieving entity-types per application
> ---
>
> Key: YARN-7982
> URL: https://issues.apache.org/jira/browse/YARN-7982
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Rohith Sharma K S
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-7982-001.patch, YARN-7982-002.patch,
> YARN-7982-003.patch, YARN-7982-004.patch
>
>
> REST end point {{/apps/$appid/entity-types}} retrieves all the entity-types
> for given application. This need to be guarded with ACL check
> {code}
> [yarn@yarn-ats-3 ~]$ curl
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1552297011473_0002?user.name=ambari-qa1";
> {"exception":"ForbiddenException","message":"java.lang.Exception: User
> ambari-qa1 is not allowed to read TimelineService V2
> data.","javaClassName":"org.apache.hadoop.yarn.webapp.ForbiddenException"}
> [yarn@yarn-ats-3 ~]$ curl
> "http://yarn-ats-3:8198/ws/v2/timeline/apps/application_1552297011473_0002/entity-types?user.name=ambari-qa1";
> ["YARN_APPLICATION_ATTEMPT","YARN_CONTAINER"]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10875) CLI queue usage command only reflects default partition usage
[ https://issues.apache.org/jira/browse/YARN-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10875: --- Assignee: D M Murali Krishna Reddy > CLI queue usage command only reflects default partition usage > - > > Key: YARN-10875 > URL: https://issues.apache.org/jira/browse/YARN-10875 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rajshree Mishra >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: queueA_scheduler.png, queueA_usage.png > > > Test step: > # Hadoop cluster with nodelabels -> default, label1 > # Job is submitted to queueA using resources of accessible nodelabel label1 > # Check queue usage for queueA using CLI command "yarn queue -status queueA" > Output: Current capacity is displayed as 00% > Expected: queueA is being utilized under label1 resource pool, and status > command should reflect the same. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10876) CLI queue usage should indicate absolute usage
[ https://issues.apache.org/jira/browse/YARN-10876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10876: --- Assignee: D M Murali Krishna Reddy > CLI queue usage should indicate absolute usage > -- > > Key: YARN-10876 > URL: https://issues.apache.org/jira/browse/YARN-10876 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Rajshree Mishra >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: schedulerUsage.png, usageCLI.png > > > For large cluster with multiple users, WebUI proves to be very slow. > Users use the CLI to check the usage information, however the output displays > percentages above 100. > Users wants to know the available resources to judge if more jobs can be > submitted and these percentages don't give a clear picture about this > information. > CLI output should be made more user friendly to provide information about > used and available resources in a queue, as user may not know total resource > of a large cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10825) Yarn Service containers not getting killed after NM shutdown
[ https://issues.apache.org/jira/browse/YARN-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10825: Component/s: yarn-native-services > Yarn Service containers not getting killed after NM shutdown > > > Key: YARN-10825 > URL: https://issues.apache.org/jira/browse/YARN-10825 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn, yarn-native-services >Affects Versions: 3.1.1 >Reporter: Sushanta Sen >Assignee: D M Murali Krishna Reddy >Priority: Major > > When yarn.nodemanager.recovery.supervised is enabled and NM is shutdown, the > new containers are getting launched after the RM sends the node lost event to > AM, but the existing containers on the lost node are not getting killed. The > issue has occurred only for yarn service. For Normal jobs the behavior is > working fine. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10825) Yarn Service containers not getting killed after NM shutdown
[ https://issues.apache.org/jira/browse/YARN-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17367223#comment-17367223 ] D M Murali Krishna Reddy commented on YARN-10825: - As per my analysis when yarn.nodemanager.recovery.supervised is enabled, I could see that for Mapreduce jobs, once the NM is shutdown, after some time RM assumes the Node lost and then with UPDATED_NODES_TRANSITION, AM removes all the taskAttempts of the containers launched on Lost node and launches the next taskattempt. Once the old containers sends *status update*, the AM assumes it as illegal task and returns feedback with taskFound as false in TaskAttemptListenerImpl. In Task.java container gets killed by itself. But in yarn services I couldnt find any communication directly from container to AM like *status update* in MR jobs. So, I think the AM is not able to communicate to container directly to get the container killed. I think the only communication is from AM to RM and then from RM to NM to container, which is not possible as the NM itself is down. [~billie], [~eyang], [~prabhujoseph] Can you have look over this issue. Thanks! > Yarn Service containers not getting killed after NM shutdown > > > Key: YARN-10825 > URL: https://issues.apache.org/jira/browse/YARN-10825 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.1.1 >Reporter: Sushanta Sen >Assignee: D M Murali Krishna Reddy >Priority: Major > > When yarn.nodemanager.recovery.supervised is enabled and NM is shutdown, the > new containers are getting launched after the RM sends the node lost event to > AM, but the existing containers on the lost node are not getting killed. The > issue has occurred only for yarn service. For Normal jobs the behavior is > working fine. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10825) Yarn Service containers not getting killed after NM shutdown
[ https://issues.apache.org/jira/browse/YARN-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10825: --- Assignee: D M Murali Krishna Reddy > Yarn Service containers not getting killed after NM shutdown > > > Key: YARN-10825 > URL: https://issues.apache.org/jira/browse/YARN-10825 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Affects Versions: 3.1.1 >Reporter: Sushanta Sen >Assignee: D M Murali Krishna Reddy >Priority: Major > > When yarn.nodemanager.recovery.supervised is enabled and NM is shutdown, the > new containers are getting launched after the RM sends the node lost event to > AM, but the existing containers on the lost node are not getting killed. The > issue has occurred only for yarn service. For Normal jobs the behavior is > working fine. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: YARN-10767.004.patch > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch, YARN-10767.002.patch, > YARN-10767.003.patch, YARN-10767.004.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17363087#comment-17363087 ] D M Murali Krishna Reddy commented on YARN-10767: - [~Jim_Brennan], I have fixed the spotbugs issue in the v3 patch. Can you have a look? > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch, YARN-10767.002.patch, > YARN-10767.003.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: YARN-10767.003.patch > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch, YARN-10767.002.patch, > YARN-10767.003.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360826#comment-17360826 ] D M Murali Krishna Reddy commented on YARN-10767: - [~Jim_Brennan], [~BilwaST], [~brahma] Can you review the v2 patch. Thanks. > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch, YARN-10767.002.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: YARN-10767.002.patch > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch, YARN-10767.002.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357070#comment-17357070 ] D M Murali Krishna Reddy commented on YARN-10767: - Thanks [~BilwaST], [~Jim_Brennan] for the review 1. I will handle the null check in the v2 patch. 2. Yes, findActiveRMHAId is going to contact all the RM's but there is no retry policy in this case, they try to connect only once to each RM, and return the active one as soon as they are able to connect to any one of the RM. Yes I have verified the fix on a HA cluster. 3. I also believe it is not necessary to loop through all the RM's once we find out the active RM. I just wanted to improve the existing functionality without impacting the existing one, So I have just changed the executing order of RM's so that we try to connect to the active RM first. 4. I am also not sure why the method name is execOnActiveRM but currently it executes on all the RM's in a loop. > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.006.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch, YARN-10745.005.patch, > YARN-10745.006.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17346178#comment-17346178 ] D M Murali Krishna Reddy commented on YARN-10745: - [~ebadger] Can you review the patch, so that it can be merged to 3.3.1 Thanks > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch, YARN-10745.005.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343010#comment-17343010 ] D M Murali Krishna Reddy commented on YARN-10767: - [~BilwaST] [~zhuqi] can you review the above patch. Thanks. > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: YARN-10767.001.patch > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: (was: YARN-10767.001.patch) > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
[ https://issues.apache.org/jira/browse/YARN-10767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10767: Attachment: YARN-10767.001.patch > Yarn Logs Command retrying on Standby RM for 30 times > - > > Key: YARN-10767 > URL: https://issues.apache.org/jira/browse/YARN-10767 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10767.001.patch > > > When ResourceManager HA is enabled and the first RM is unavailable, on > executing "bin/yarn logs -applicationId -am 1", we get > ConnectionException for connecting to the first RM, the ConnectionException > Occurs for 30 times before it tries to connect to the second RM. > > This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10767) Yarn Logs Command retrying on Standby RM for 30 times
D M Murali Krishna Reddy created YARN-10767: --- Summary: Yarn Logs Command retrying on Standby RM for 30 times Key: YARN-10767 URL: https://issues.apache.org/jira/browse/YARN-10767 Project: Hadoop YARN Issue Type: Bug Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy When ResourceManager HA is enabled and the first RM is unavailable, on executing "bin/yarn logs -applicationId -am 1", we get ConnectionException for connecting to the first RM, the ConnectionException Occurs for 30 times before it tries to connect to the second RM. This can be optimized by trying to fetch the logs from the Active RM. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10725) Backport YARN-10120 to branch-3.3
[ https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10725: Attachment: YARN-10725-branch-3.3.v3.patch > Backport YARN-10120 to branch-3.3 > - > > Key: YARN-10725 > URL: https://issues.apache.org/jira/browse/YARN-10725 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10120-branch-3.3.patch, > YARN-10725-branch-3.3.patch, YARN-10725-branch-3.3.v2.patch, > YARN-10725-branch-3.3.v3.patch, image-2021-04-05-16-48-57-034.png, > image-2021-04-05-16-50-55-238.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10725) Backport YARN-10120 to branch-3.3
[ https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10725: Attachment: (was: YARN-10725-branch-3.3.v3.patch) > Backport YARN-10120 to branch-3.3 > - > > Key: YARN-10725 > URL: https://issues.apache.org/jira/browse/YARN-10725 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10120-branch-3.3.patch, > YARN-10725-branch-3.3.patch, YARN-10725-branch-3.3.v2.patch, > image-2021-04-05-16-48-57-034.png, image-2021-04-05-16-50-55-238.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10725) Backport YARN-10120 to branch-3.3
[ https://issues.apache.org/jira/browse/YARN-10725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10725: Attachment: YARN-10725-branch-3.3.v3.patch > Backport YARN-10120 to branch-3.3 > - > > Key: YARN-10725 > URL: https://issues.apache.org/jira/browse/YARN-10725 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10120-branch-3.3.patch, > YARN-10725-branch-3.3.patch, YARN-10725-branch-3.3.v2.patch, > YARN-10725-branch-3.3.v3.patch, image-2021-04-05-16-48-57-034.png, > image-2021-04-05-16-50-55-238.png > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.005.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch, YARN-10745.005.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[
https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340555#comment-17340555
]
D M Murali Krishna Reddy commented on YARN-10745:
-
Thanks [~ebadger] for the review comments.
Regarding the wrong indentation, In the initial patches I have followed the
correct level of indentation but the hadoopQA checkstyle was showing error, So
I have changed the indentation to fix the checkstyle warnings. I will change
the indentation level as per your review.
Is {{clusterNodeReports}} guaranteeed to be non-null here?
Yes, as per my understanding the clusterNodeReports will never be null. If it
was null we would be getting a NPE in the below for loop anyway. Also, I think
findbugs would catch this type of potential NPE, So I don't think it is a
problem.
{code:java}
-// NodeManager is the last service to start, so NodeId is available.
+// NodeStatusUpdater is the last service to start, so NodeId is available.
{code}
Regarding the above change, I have misunderstood the old comment and changed
it. Will be reverting it.
{code:java}
+ LOG.info("Callback succeeded for initializing request processing " +
+ "pipeline for an AM ");
{code}
I haven't debugged AMRMProxy a lot, but going through the code found it might
be useful to have this log. If you feel it is not required and doesn't add any
value, I can remove it.
{code:java}
-LOG.info("hostsReader include:{" +
-StringUtils.join(",", hostsReader.getHosts()) +
-"} exclude:{" +
-StringUtils.join(",", hostsReader.getExcludedHosts()) + "}");
-
+if (!hostsReader.getHosts().isEmpty() ||
+!hostsReader.getExcludedHosts().isEmpty()) {
+ LOG.info("hostsReader include:{" +
+ StringUtils.join(",", hostsReader.getHosts()) +
+ "} exclude:{" +
+ StringUtils.join(",", hostsReader.getExcludedHosts()) + "}");
+}
{code}
I have added this change as per the suggestion of [~BilwaST], I will remove
this change in the 005 patch.
> Change Log level from info to debug for few logs and remove unnecessary
> debuglog checks
> ---
>
> Key: YARN-10745
> URL: https://issues.apache.org/jira/browse/YARN-10745
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-10745.001.patch, YARN-10745.002.patch,
> YARN-10745.003.patch, YARN-10745.004.patch
>
>
> Change the info log level to debug for few logs so that the load on the
> logger decreases in large cluster and improves the performance.
> Remove the unnecessary isDebugEnabled() checks for printing strings without
> any string concatenation
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.004.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch, YARN-10745.004.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.003.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch, > YARN-10745.003.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.002.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch, YARN-10745.002.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17338974#comment-17338974
]
D M Murali Krishna Reddy commented on YARN-10648:
-
Hi [~snemeth], [~ebadger] Can you have a look at the issue and review the patch.
Thanks
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-10648.001.patch
>
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
[ https://issues.apache.org/jira/browse/YARN-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10745: Attachment: YARN-10745.001.patch > Change Log level from info to debug for few logs and remove unnecessary > debuglog checks > --- > > Key: YARN-10745 > URL: https://issues.apache.org/jira/browse/YARN-10745 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10745.001.patch > > > Change the info log level to debug for few logs so that the load on the > logger decreases in large cluster and improves the performance. > Remove the unnecessary isDebugEnabled() checks for printing strings without > any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10749) Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647
[ https://issues.apache.org/jira/browse/YARN-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17330348#comment-17330348 ] D M Murali Krishna Reddy commented on YARN-10749: - Thanks [~zhuqi] for the prompt review. I have attached a v2 patch addressing the checkstyle issue. [~caozhiqiang], [~ebadger] Can you have a look and review the patch. > Can't remove all node labels after add node label without nodemanager port, > broken by YARN-10647 > > > Key: YARN-10749 > URL: https://issues.apache.org/jira/browse/YARN-10749 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10749.001.patch, YARN-10749.002.patch > > > The fix done in YARN-10501, doesn't work after YARN-10647. > To reproduce follow the same steps in YARN-10501 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10749) Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647
[ https://issues.apache.org/jira/browse/YARN-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10749: Attachment: YARN-10749.002.patch > Can't remove all node labels after add node label without nodemanager port, > broken by YARN-10647 > > > Key: YARN-10749 > URL: https://issues.apache.org/jira/browse/YARN-10749 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10749.001.patch, YARN-10749.002.patch > > > The fix done in YARN-10501, doesn't work after YARN-10647. > To reproduce follow the same steps in YARN-10501 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10749) Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647
[ https://issues.apache.org/jira/browse/YARN-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10749: --- Assignee: D M Murali Krishna Reddy > Can't remove all node labels after add node label without nodemanager port, > broken by YARN-10647 > > > Key: YARN-10749 > URL: https://issues.apache.org/jira/browse/YARN-10749 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10749.001.patch > > > The fix done in YARN-10501, doesn't work after YARN-10647. > To reproduce follow the same steps in YARN-10501 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10749) Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647
[ https://issues.apache.org/jira/browse/YARN-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10749: Attachment: YARN-10749.001.patch > Can't remove all node labels after add node label without nodemanager port, > broken by YARN-10647 > > > Key: YARN-10749 > URL: https://issues.apache.org/jira/browse/YARN-10749 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10749.001.patch > > > The fix done in YARN-10501, doesn't work after YARN-10647. > To reproduce follow the same steps in YARN-10501 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10749) Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647
D M Murali Krishna Reddy created YARN-10749: --- Summary: Can't remove all node labels after add node label without nodemanager port, broken by YARN-10647 Key: YARN-10749 URL: https://issues.apache.org/jira/browse/YARN-10749 Project: Hadoop YARN Issue Type: Bug Reporter: D M Murali Krishna Reddy The fix done in YARN-10501, doesn't work after YARN-10647. To reproduce follow the same steps in YARN-10501 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10745) Change Log level from info to debug for few logs and remove unnecessary debuglog checks
D M Murali Krishna Reddy created YARN-10745: --- Summary: Change Log level from info to debug for few logs and remove unnecessary debuglog checks Key: YARN-10745 URL: https://issues.apache.org/jira/browse/YARN-10745 Project: Hadoop YARN Issue Type: Improvement Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy Change the info log level to debug for few logs so that the load on the logger decreases in large cluster and improves the performance. Remove the unnecessary isDebugEnabled() checks for printing strings without any string concatenation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10125) In Federation, kill application from client does not kill Unmanaged AM's and containers launched by Unmanaged AM
[ https://issues.apache.org/jira/browse/YARN-10125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10125: Attachment: YARN-10125.002.patch > In Federation, kill application from client does not kill Unmanaged AM's and > containers launched by Unmanaged AM > > > Key: YARN-10125 > URL: https://issues.apache.org/jira/browse/YARN-10125 > Project: Hadoop YARN > Issue Type: Bug > Components: client, federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10125.001.patch, YARN-10125.002.patch > > > In Federation, killing an application from client using "bin/yarn application > -kill ", kills the containers only of the home subcluster, > the Unmanaged AM and the containers launched in other subcluster are not > being killed causing blocking of resources. > The containers get killed after the task gets completed and The unmanaged AM > gets killed after 10 minutes of killing the application, killing any > remaining running containers in that subcluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10671) Fix Typo in TestSchedulingRequestContainerAllocation
[
https://issues.apache.org/jira/browse/YARN-10671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10671:
Attachment: YARN-10671.001.patch
> Fix Typo in TestSchedulingRequestContainerAllocation
>
>
> Key: YARN-10671
> URL: https://issues.apache.org/jira/browse/YARN-10671
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-10671.001.patch
>
>
> Remove extra colon in
> TestSchedulingRequestContainerAllocation.testInterAppCompositeConstraints ,
> the test case is passing currently but, It will fail once YARN-4283 gets
> merged.
> {code:java}
> MockNM nm1 = rm.registerNode("192.168.0.1:1234:", 100*GB, 100);{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10671) Fix Typo in TestSchedulingRequestContainerAllocation
D M Murali Krishna Reddy created YARN-10671:
---
Summary: Fix Typo in TestSchedulingRequestContainerAllocation
Key: YARN-10671
URL: https://issues.apache.org/jira/browse/YARN-10671
Project: Hadoop YARN
Issue Type: Bug
Reporter: D M Murali Krishna Reddy
Assignee: D M Murali Krishna Reddy
Remove extra colon in
TestSchedulingRequestContainerAllocation.testInterAppCompositeConstraints , the
test case is passing currently but, It will fail once YARN-4283 gets merged.
{code:java}
MockNM nm1 = rm.registerNode("192.168.0.1:1234:", 100*GB, 100);{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288991#comment-17288991
]
D M Murali Krishna Reddy edited comment on YARN-10648 at 2/23/21, 10:20 AM:
Hi [~snemeth] [~brahmareddy] Can you have a look over the issue and review the
patch.
Thanks
was (Author: dmmkr):
Hi [~snemeth] [~brahma] Can you have a look over the issue and review the
patch.
Thanks
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-10648.001.patch
>
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288991#comment-17288991
]
D M Murali Krishna Reddy commented on YARN-10648:
-
Hi [~snemeth] [~brahma] Can you have a look over the issue and review the
patch.
Thanks
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-10648.001.patch
>
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10648:
Attachment: YARN-10648.001.patch
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-10648.001.patch
>
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10648) NM local logs are not cleared after uploading to hdfs
[
https://issues.apache.org/jira/browse/YARN-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10648:
Description:
YARN-8273 has induced the following issues.
# The {color:#00}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
was:
YARN-8273 has induced the following issues.
# The {color:#00}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
> NM local logs are not cleared after uploading to hdfs
> -
>
> Key: YARN-10648
> URL: https://issues.apache.org/jira/browse/YARN-10648
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.2.0
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Major
>
> YARN-8273 has induced the following issues.
> # The {color:#00}delService.delete(deletionTask){color} has been removed
> from the for loop, and added at the end in finally block. Inside the for loop
> we are creating FileDeletionTask for each container, but not storing it, due
> to this, only the last container log files will be present in the
> deletionTask and only those files will be removed. Ideally all the container
> log files which are uploaded must be deleted.
> # The LogAggregationDFSException is caught in the closeswriter, but when we
> configure LogAggregationTFileController as logAggregationFileController,
> this.logAggregationFileController.closeWriter() itself calls closeWriter,
> which throws LogAggregationDFSException if any, and the exception is not
> saved. Again when we try to do closeWriter we dont get any exception and, we
> are not throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8273) Log aggregation does not warn if HDFS quota in target directory is exceeded
[
https://issues.apache.org/jira/browse/YARN-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288890#comment-17288890
]
D M Murali Krishna Reddy commented on YARN-8273:
This Jira has induced the following issues.
# The {color:#00}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() call itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
YARN-10648 is raised for fixing these issues
> Log aggregation does not warn if HDFS quota in target directory is exceeded
> ---
>
> Key: YARN-8273
> URL: https://issues.apache.org/jira/browse/YARN-8273
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.1.0
>Reporter: Gergo Repas
>Assignee: Gergo Repas
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-8273.000.patch, YARN-8273.001.patch,
> YARN-8273.002.patch, YARN-8273.003.patch, YARN-8273.004.patch,
> YARN-8273.005.patch, YARN-8273.006.patch
>
>
> It appears that if an HDFS space quota is set on a target directory for log
> aggregation and the quota is already exceeded when log aggregation is
> attempted, zero-byte log files will be written to the HDFS directory, however
> NodeManager logs do not reflect a failure to write the files successfully
> (i.e. there are no ERROR or WARN messages to this effect).
> An improvement may be worth investigating to alert users to this scenario, as
> otherwise logs for a YARN application may be missing both on HDFS and locally
> (after local log cleanup is done) and the user may not otherwise be informed.
> Steps to reproduce:
> * Set a small HDFS space quota on /tmp/logs/username/logs (e.g. 2MB)
> * Write files to HDFS such that /tmp/logs/username/logs is almost 2MB full
> * Run a Spark or MR job in the cluster
> * Observe that zero byte files are written to HDFS after job completion
> * Observe that YARN container logs are also not present on the NM hosts (or
> are deleted after yarn.nodemanager.delete.debug-delay-sec)
> * Observe that no ERROR or WARN messages appear to be logged in the NM role
> log
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10648) NM local logs are not cleared after uploading to hdfs
D M Murali Krishna Reddy created YARN-10648:
---
Summary: NM local logs are not cleared after uploading to hdfs
Key: YARN-10648
URL: https://issues.apache.org/jira/browse/YARN-10648
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation
Affects Versions: 3.2.0
Reporter: D M Murali Krishna Reddy
Assignee: D M Murali Krishna Reddy
YARN-8273 has induced the following issues.
# The {color:#00}delService.delete(deletionTask){color} has been removed
from the for loop, and added at the end in finally block. Inside the for loop
we are creating FileDeletionTask for each container, but not storing it, due to
this, only the last container log files will be present in the deletionTask and
only those files will be removed. Ideally all the container log files which are
uploaded must be deleted.
# The LogAggregationDFSException is caught in the closeswriter, but when we
configure LogAggregationTFileController as logAggregationFileController,
this.logAggregationFileController.closeWriter() itself calls closeWriter,
which throws LogAggregationDFSException if any, and the exception is not saved.
Again when we try to do closeWriter we dont get any exception and, we are not
throwing the LogAggregationDFSException in this scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10466) Fix NullPointerException in yarn-services Component.java
[ https://issues.apache.org/jira/browse/YARN-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1722#comment-1722 ] D M Murali Krishna Reddy commented on YARN-10466: - Thanks [~brahmareddy] for the review, I have changed the log level and updated the patch. > Fix NullPointerException in yarn-services Component.java > - > > Key: YARN-10466 > URL: https://issues.apache.org/jira/browse/YARN-10466 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10466.001.patch, YARN-10466.002.patch > > > Due to changes in > [YARN-10219|https://issues.apache.org/jira/browse/YARN-10219] where the > constraint is initialised as null, there might be few scenarios in which NPE > can be thrown in requestContainers method. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10466) Fix NullPointerException in yarn-services Component.java
[ https://issues.apache.org/jira/browse/YARN-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10466: Attachment: YARN-10466.002.patch > Fix NullPointerException in yarn-services Component.java > - > > Key: YARN-10466 > URL: https://issues.apache.org/jira/browse/YARN-10466 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10466.001.patch, YARN-10466.002.patch > > > Due to changes in > [YARN-10219|https://issues.apache.org/jira/browse/YARN-10219] where the > constraint is initialised as null, there might be few scenarios in which NPE > can be thrown in requestContainers method. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9883) Reshape SchedulerHealth class
[
https://issues.apache.org/jira/browse/YARN-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17240511#comment-17240511
]
D M Murali Krishna Reddy commented on YARN-9883:
Thanks [~adam.antal] for the review. I have handled the javadoc warnings in
YARN-9883.002.patch
> Reshape SchedulerHealth class
> -
>
> Key: YARN-9883
> URL: https://issues.apache.org/jira/browse/YARN-9883
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-9883.001.patch, YARN-9883.002.patch
>
>
> The {{SchedulerHealth}} class has some flaws, for example:
> - It has no javadoc at all
> - All its objects are package-private: they should be private
> - The internal maps should be (Concurrent) EnumMaps instead of HashMaps: they
> are more efficient in storing Enums
> - schedulerHealthDetails only stores the last operation, its name should
> reflect that (just like lastSchedulerRunDetails)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9883) Reshape SchedulerHealth class
[
https://issues.apache.org/jira/browse/YARN-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-9883:
---
Attachment: YARN-9883.002.patch
> Reshape SchedulerHealth class
> -
>
> Key: YARN-9883
> URL: https://issues.apache.org/jira/browse/YARN-9883
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-9883.001.patch, YARN-9883.002.patch
>
>
> The {{SchedulerHealth}} class has some flaws, for example:
> - It has no javadoc at all
> - All its objects are package-private: they should be private
> - The internal maps should be (Concurrent) EnumMaps instead of HashMaps: they
> are more efficient in storing Enums
> - schedulerHealthDetails only stores the last operation, its name should
> reflect that (just like lastSchedulerRunDetails)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10487) Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10487: --- Assignee: D M Murali Krishna Reddy > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation > --- > > Key: YARN-10487 > URL: https://issues.apache.org/jira/browse/YARN-10487 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10487.001.patch > > > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9883) Reshape SchedulerHealth class
[
https://issues.apache.org/jira/browse/YARN-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235959#comment-17235959
]
D M Murali Krishna Reddy commented on YARN-9883:
[~BilwaST] I have uploaded the patch, can you review the patch
> Reshape SchedulerHealth class
> -
>
> Key: YARN-9883
> URL: https://issues.apache.org/jira/browse/YARN-9883
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-9883.001.patch
>
>
> The {{SchedulerHealth}} class has some flaws, for example:
> - It has no javadoc at all
> - All its objects are package-private: they should be private
> - The internal maps should be (Concurrent) EnumMaps instead of HashMaps: they
> are more efficient in storing Enums
> - schedulerHealthDetails only stores the last operation, its name should
> reflect that (just like lastSchedulerRunDetails)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9883) Reshape SchedulerHealth class
[
https://issues.apache.org/jira/browse/YARN-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-9883:
---
Attachment: YARN-9883.001.patch
> Reshape SchedulerHealth class
> -
>
> Key: YARN-9883
> URL: https://issues.apache.org/jira/browse/YARN-9883
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-9883.001.patch
>
>
> The {{SchedulerHealth}} class has some flaws, for example:
> - It has no javadoc at all
> - All its objects are package-private: they should be private
> - The internal maps should be (Concurrent) EnumMaps instead of HashMaps: they
> are more efficient in storing Enums
> - schedulerHealthDetails only stores the last operation, its name should
> reflect that (just like lastSchedulerRunDetails)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9883) Reshape SchedulerHealth class
[
https://issues.apache.org/jira/browse/YARN-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy reassigned YARN-9883:
--
Assignee: D M Murali Krishna Reddy (was: Bilwa S T)
> Reshape SchedulerHealth class
> -
>
> Key: YARN-9883
> URL: https://issues.apache.org/jira/browse/YARN-9883
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, yarn
>Affects Versions: 3.3.0
>Reporter: Adam Antal
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
>
> The {{SchedulerHealth}} class has some flaws, for example:
> - It has no javadoc at all
> - All its objects are package-private: they should be private
> - The internal maps should be (Concurrent) EnumMaps instead of HashMaps: they
> are more efficient in storing Enums
> - schedulerHealthDetails only stores the last operation, its name should
> reflect that (just like lastSchedulerRunDetails)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10474) [JDK 12] TestAsyncDispatcher fails
[
https://issues.apache.org/jira/browse/YARN-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy reassigned YARN-10474:
---
Assignee: (was: D M Murali Krishna Reddy)
> [JDK 12] TestAsyncDispatcher fails
> --
>
> Key: YARN-10474
> URL: https://issues.apache.org/jira/browse/YARN-10474
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Priority: Major
>
> Similar to HDFS-15580. Updating a final variable via reflection is not
> allowed in Java 12+.
> {noformat}
> [INFO] Running org.apache.hadoop.yarn.event.TestAsyncDispatcher
> [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.953
> s <<< FAILURE! - in org.apache.hadoop.yarn.event.TestAsyncDispatcher
> [ERROR]
> testPrintDispatcherEventDetails(org.apache.hadoop.yarn.event.TestAsyncDispatcher)
> Time elapsed: 0.114 s <<< ERROR!
> java.lang.NoSuchFieldException: modifiers
> at java.base/java.lang.Class.getDeclaredField(Class.java:2569)
> at
> org.apache.hadoop.yarn.event.TestAsyncDispatcher.testPrintDispatcherEventDetails(TestAsyncDispatcher.java:152)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:832)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10487) Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10487: Attachment: YARN-10487.001.patch > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation > --- > > Key: YARN-10487 > URL: https://issues.apache.org/jira/browse/YARN-10487 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10487.001.patch > > > Support getQueueUserAcls, listReservations, getApplicationAttempts, > getContainerReport, getContainers, getResourceTypeInfo API's for Federation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10487) Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation
D M Murali Krishna Reddy created YARN-10487: --- Summary: Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation Key: YARN-10487 URL: https://issues.apache.org/jira/browse/YARN-10487 Project: Hadoop YARN Issue Type: Sub-task Reporter: D M Murali Krishna Reddy Support getQueueUserAcls, listReservations, getApplicationAttempts, getContainerReport, getContainers, getResourceTypeInfo API's for Federation -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8173) [Router] Implement missing FederationClientInterceptor#getApplications()
[
https://issues.apache.org/jira/browse/YARN-8173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-8173:
---
Attachment: YARN-8173.008.patch
> [Router] Implement missing FederationClientInterceptor#getApplications()
>
>
> Key: YARN-8173
> URL: https://issues.apache.org/jira/browse/YARN-8173
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-8173.001.patch, YARN-8173.002.patch,
> YARN-8173.003.patch, YARN-8173.004.patch, YARN-8173.005.patch,
> YARN-8173.006.patch, YARN-8173.007.patch, YARN-8173.008.patch
>
>
> oozie dependent method Implement
> {code:java}
> getApplications()
> getDeglationToken()
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-8173) [Router] Implement missing FederationClientInterceptor#getApplications()
[
https://issues.apache.org/jira/browse/YARN-8173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy reassigned YARN-8173:
--
Assignee: D M Murali Krishna Reddy (was: Yiran Wu)
> [Router] Implement missing FederationClientInterceptor#getApplications()
>
>
> Key: YARN-8173
> URL: https://issues.apache.org/jira/browse/YARN-8173
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: D M Murali Krishna Reddy
>Priority: Major
> Attachments: YARN-8173.001.patch, YARN-8173.002.patch,
> YARN-8173.003.patch, YARN-8173.004.patch, YARN-8173.005.patch,
> YARN-8173.006.patch, YARN-8173.007.patch
>
>
> oozie dependent method Implement
> {code:java}
> getApplications()
> getDeglationToken()
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10125) In Federation, kill application from client does not kill Unmanaged AM's and containers launched by Unmanaged AM
[ https://issues.apache.org/jira/browse/YARN-10125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10125: Attachment: YARN-10125.001.patch > In Federation, kill application from client does not kill Unmanaged AM's and > containers launched by Unmanaged AM > > > Key: YARN-10125 > URL: https://issues.apache.org/jira/browse/YARN-10125 > Project: Hadoop YARN > Issue Type: Bug > Components: client, federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10125.001.patch > > > In Federation, killing an application from client using "bin/yarn application > -kill ", kills the containers only of the home subcluster, > the Unmanaged AM and the containers launched in other subcluster are not > being killed causing blocking of resources. > The containers get killed after the task gets completed and The unmanaged AM > gets killed after 10 minutes of killing the application, killing any > remaining running containers in that subcluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10474) [JDK 12] TestAsyncDispatcher fails
[
https://issues.apache.org/jira/browse/YARN-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy reassigned YARN-10474:
---
Assignee: D M Murali Krishna Reddy
> [JDK 12] TestAsyncDispatcher fails
> --
>
> Key: YARN-10474
> URL: https://issues.apache.org/jira/browse/YARN-10474
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test
>Reporter: Akira Ajisaka
>Assignee: D M Murali Krishna Reddy
>Priority: Major
>
> Similar to HDFS-15580. Updating a final variable via reflection is not
> allowed in Java 12+.
> {noformat}
> [INFO] Running org.apache.hadoop.yarn.event.TestAsyncDispatcher
> [ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.953
> s <<< FAILURE! - in org.apache.hadoop.yarn.event.TestAsyncDispatcher
> [ERROR]
> testPrintDispatcherEventDetails(org.apache.hadoop.yarn.event.TestAsyncDispatcher)
> Time elapsed: 0.114 s <<< ERROR!
> java.lang.NoSuchFieldException: modifiers
> at java.base/java.lang.Class.getDeclaredField(Class.java:2569)
> at
> org.apache.hadoop.yarn.event.TestAsyncDispatcher.testPrintDispatcherEventDetails(TestAsyncDispatcher.java:152)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:564)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:832)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10466) Fix NullPointerException in yarn-services Component.java
[ https://issues.apache.org/jira/browse/YARN-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10466: Attachment: YARN-10466.001.patch > Fix NullPointerException in yarn-services Component.java > - > > Key: YARN-10466 > URL: https://issues.apache.org/jira/browse/YARN-10466 > Project: Hadoop YARN > Issue Type: Bug >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10466.001.patch > > > Due to changes in > [YARN-10219|https://issues.apache.org/jira/browse/YARN-10219] where the > constraint is initialised as null, there might be few scenarios in which NPE > can be thrown in requestContainers method. > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8173) [Router] Implement missing FederationClientInterceptor#getApplications()
[
https://issues.apache.org/jira/browse/YARN-8173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17217056#comment-17217056
]
D M Murali Krishna Reddy commented on YARN-8173:
[~yiran], I would like to work on this, if you are not currently working on
this task.
> [Router] Implement missing FederationClientInterceptor#getApplications()
>
>
> Key: YARN-8173
> URL: https://issues.apache.org/jira/browse/YARN-8173
> Project: Hadoop YARN
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Yiran Wu
>Assignee: Yiran Wu
>Priority: Major
> Attachments: YARN-8173.001.patch, YARN-8173.002.patch,
> YARN-8173.003.patch, YARN-8173.004.patch, YARN-8173.005.patch,
> YARN-8173.006.patch, YARN-8173.007.patch
>
>
> oozie dependent method Implement
> {code:java}
> getApplications()
> getDeglationToken()
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10466) Fix NullPointerException in yarn-services Component.java
D M Murali Krishna Reddy created YARN-10466: --- Summary: Fix NullPointerException in yarn-services Component.java Key: YARN-10466 URL: https://issues.apache.org/jira/browse/YARN-10466 Project: Hadoop YARN Issue Type: Bug Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy Due to changes in [YARN-10219|https://issues.apache.org/jira/browse/YARN-10219] where the constraint is initialised as null, there might be few scenarios in which NPE can be thrown in requestContainers method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10465) Support getClusterNodes, getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation
[ https://issues.apache.org/jira/browse/YARN-10465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10465: Attachment: YARN-10465.001.patch > Support getClusterNodes, getNodeToLabels, getLabelsToNodes, > getClusterNodeLabels API's for Federation > - > > Key: YARN-10465 > URL: https://issues.apache.org/jira/browse/YARN-10465 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10465.001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10465) Support getClusterNodes, getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation
D M Murali Krishna Reddy created YARN-10465: --- Summary: Support getClusterNodes, getNodeToLabels, getLabelsToNodes, getClusterNodeLabels API's for Federation Key: YARN-10465 URL: https://issues.apache.org/jira/browse/YARN-10465 Project: Hadoop YARN Issue Type: Sub-task Components: federation Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10441) Add support for hadoop.http.rmwebapp.scheduler.page.class
[ https://issues.apache.org/jira/browse/YARN-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10441: Attachment: YARN-10441.002.patch > Add support for hadoop.http.rmwebapp.scheduler.page.class > - > > Key: YARN-10441 > URL: https://issues.apache.org/jira/browse/YARN-10441 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10441.001.patch, YARN-10441.002.patch > > > In https://issues.apache.org/jira/browse/YARN-10361 the existing > configuration of hadoop.http.rmwebapp.scheduler.page.class is updated to > yarn.http.rmwebapp.scheduler.page.class, which causes incompatibility with > old versions, It is better to make the old configuration deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10441) Add support for hadoop.http.rmwebapp.scheduler.page.class
[ https://issues.apache.org/jira/browse/YARN-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10441: Attachment: YARN-10441.001.patch > Add support for hadoop.http.rmwebapp.scheduler.page.class > - > > Key: YARN-10441 > URL: https://issues.apache.org/jira/browse/YARN-10441 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10441.001.patch > > > In https://issues.apache.org/jira/browse/YARN-10361 the existing > configuration of hadoop.http.rmwebapp.scheduler.page.class is updated to > yarn.http.rmwebapp.scheduler.page.class, which causes incompatibility with > old versions, It is better to make the old configuration deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10439) Yarn Service AM listens on all IP's on the machine
[
https://issues.apache.org/jira/browse/YARN-10439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10439:
Attachment: YARN-10439.002.patch
> Yarn Service AM listens on all IP's on the machine
> --
>
> Key: YARN-10439
> URL: https://issues.apache.org/jira/browse/YARN-10439
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security, yarn-native-services
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-10439.001.patch, YARN-10439.002.patch
>
>
> In ClientAMService.java, rpc server is created without passing hostname, due
> to which the client listens on 0.0.0.0, which is a bad practise.
>
> {{InetSocketAddress address = {color:#cc7832}new
> {color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};{color}}}
> {{{color:#9876aa}server {color}=
> rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
> {color}address{color:#cc7832}, {color}conf{color:#cc7832},{color}
> {color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
> {color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
>
> Also, a new configuration must be added similar to
> "yarn.app.mapreduce.am.job.client.port-range", so that client can configure
> port range for yarn service AM to bind.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10441) Add support for hadoop.http.rmwebapp.scheduler.page.class
[ https://issues.apache.org/jira/browse/YARN-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10441: --- Assignee: D M Murali Krishna Reddy > Add support for hadoop.http.rmwebapp.scheduler.page.class > - > > Key: YARN-10441 > URL: https://issues.apache.org/jira/browse/YARN-10441 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > > In https://issues.apache.org/jira/browse/YARN-10361 the existing > configuration of hadoop.http.rmwebapp.scheduler.page.class is updated to > yarn.http.rmwebapp.scheduler.page.class, which causes incompatibility with > old versions, It is better to make the old configuration deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10441) Add support for hadoop.http.rmwebapp.scheduler.page.class
D M Murali Krishna Reddy created YARN-10441: --- Summary: Add support for hadoop.http.rmwebapp.scheduler.page.class Key: YARN-10441 URL: https://issues.apache.org/jira/browse/YARN-10441 Project: Hadoop YARN Issue Type: Bug Components: scheduler Reporter: D M Murali Krishna Reddy In https://issues.apache.org/jira/browse/YARN-10361 the existing configuration of hadoop.http.rmwebapp.scheduler.page.class is updated to yarn.http.rmwebapp.scheduler.page.class, which causes incompatibility with old versions, It is better to make the old configuration deprecated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10439) Yarn Service AM listens on all IP's on the machine
[
https://issues.apache.org/jira/browse/YARN-10439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10439:
Attachment: YARN-10439.001.patch
> Yarn Service AM listens on all IP's on the machine
> --
>
> Key: YARN-10439
> URL: https://issues.apache.org/jira/browse/YARN-10439
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security, yarn-native-services
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
> Attachments: YARN-10439.001.patch
>
>
> In ClientAMService.java, rpc server is created without passing hostname, due
> to which the client listens on 0.0.0.0, which is a bad practise.
>
> {{InetSocketAddress address = {color:#cc7832}new
> {color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};{color}}}
> {{{color:#9876aa}server {color}=
> rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
> {color}address{color:#cc7832}, {color}conf{color:#cc7832},{color}
> {color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
> {color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
>
> Also, a new configuration must be added similar to
> "yarn.app.mapreduce.am.job.client.port-range", so that client can configure
> port range for yarn service AM to bind.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10439) Yarn Service AM listens on all IP's on the machine
D M Murali Krishna Reddy created YARN-10439:
---
Summary: Yarn Service AM listens on all IP's on the machine
Key: YARN-10439
URL: https://issues.apache.org/jira/browse/YARN-10439
Project: Hadoop YARN
Issue Type: Bug
Components: security, yarn-native-services
Reporter: D M Murali Krishna Reddy
Assignee: D M Murali Krishna Reddy
In ClientAMService.java, rpc server is created without passing hostname, due to
which the client listens on 0.0.0.0, which is a bad practise.
{{InetSocketAddress address = {color:#cc7832}new
{color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};
{color}{color:#9876aa}server {color}=
rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
{color}address{color:#cc7832}, {color}conf{color:#cc7832},
{color}
{color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
{color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
Also, a new configuration must be added similar to
"yarn.app.mapreduce.am.job.client.port-range", so that client can configure
port range for yarn service AM to bind.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10439) Yarn Service AM listens on all IP's on the machine
[
https://issues.apache.org/jira/browse/YARN-10439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10439:
Description:
In ClientAMService.java, rpc server is created without passing hostname, due to
which the client listens on 0.0.0.0, which is a bad practise.
{{InetSocketAddress address = {color:#cc7832}new
{color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};{color}}}
{{{color:#9876aa}server {color}=
rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
{color}address{color:#cc7832}, {color}conf{color:#cc7832},{color}
{color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
{color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
Also, a new configuration must be added similar to
"yarn.app.mapreduce.am.job.client.port-range", so that client can configure
port range for yarn service AM to bind.
was:
In ClientAMService.java, rpc server is created without passing hostname, due to
which the client listens on 0.0.0.0, which is a bad practise.
{{InetSocketAddress address = {color:#cc7832}new
{color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};
{color}{color:#9876aa}server {color}=
rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
{color}address{color:#cc7832}, {color}conf{color:#cc7832},
{color}
{color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
{color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
Also, a new configuration must be added similar to
"yarn.app.mapreduce.am.job.client.port-range", so that client can configure
port range for yarn service AM to bind.
> Yarn Service AM listens on all IP's on the machine
> --
>
> Key: YARN-10439
> URL: https://issues.apache.org/jira/browse/YARN-10439
> Project: Hadoop YARN
> Issue Type: Bug
> Components: security, yarn-native-services
>Reporter: D M Murali Krishna Reddy
>Assignee: D M Murali Krishna Reddy
>Priority: Minor
>
> In ClientAMService.java, rpc server is created without passing hostname, due
> to which the client listens on 0.0.0.0, which is a bad practise.
>
> {{InetSocketAddress address = {color:#cc7832}new
> {color}InetSocketAddress({color:#6897bb}0{color}){color:#cc7832};{color}}}
> {{{color:#9876aa}server {color}=
> rpc.getServer(ClientAMProtocol.{color:#cc7832}class, this,
> {color}address{color:#cc7832}, {color}conf{color:#cc7832},{color}
> {color:#9876aa}context{color}.{color:#9876aa}secretManager{color}{color:#cc7832},
> {color}{color:#6897bb}1{color}){color:#cc7832};{color}}}
>
> Also, a new configuration must be added similar to
> "yarn.app.mapreduce.am.job.client.port-range", so that client can configure
> port range for yarn service AM to bind.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10437) Destroy yarn service if any YarnException occurs during submitApp
[ https://issues.apache.org/jira/browse/YARN-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17196809#comment-17196809 ] D M Murali Krishna Reddy commented on YARN-10437: - [~hemanthboyina] Added test case > Destroy yarn service if any YarnException occurs during submitApp > - > > Key: YARN-10437 > URL: https://issues.apache.org/jira/browse/YARN-10437 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10437.001.patch, YARN-10437.002.patch > > > If a user submits a yarn service with configuration such that it causes an > exception during application submission, the files related to the service are > not cleared from hdfs automatically. Ideally the files stored to hdfs cannot > be used in future to start or stop the service as the configuration itself is > invalid. So, we should destroy the service and remove the residual files in > hdfs, if any YarnException is thrown. > For example if the user submits a service with configuring with "memory" more > than the maximum resource, the service fails but the files in hdfs are not > cleared. But these files should be cleared. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10437) Destroy yarn service if any YarnException occurs during submitApp
[ https://issues.apache.org/jira/browse/YARN-10437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10437: Attachment: YARN-10437.002.patch > Destroy yarn service if any YarnException occurs during submitApp > - > > Key: YARN-10437 > URL: https://issues.apache.org/jira/browse/YARN-10437 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn-native-services >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Minor > Attachments: YARN-10437.001.patch, YARN-10437.002.patch > > > If a user submits a yarn service with configuration such that it causes an > exception during application submission, the files related to the service are > not cleared from hdfs automatically. Ideally the files stored to hdfs cannot > be used in future to start or stop the service as the configuration itself is > invalid. So, we should destroy the service and remove the residual files in > hdfs, if any YarnException is thrown. > For example if the user submits a service with configuring with "memory" more > than the maximum resource, the service fails but the files in hdfs are not > cleared. But these files should be cleared. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10437) Destroy yarn service if any YarnException occurs during submitApp
D M Murali Krishna Reddy created YARN-10437: --- Summary: Destroy yarn service if any YarnException occurs during submitApp Key: YARN-10437 URL: https://issues.apache.org/jira/browse/YARN-10437 Project: Hadoop YARN Issue Type: Bug Components: yarn-native-services Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy If a user submits a yarn service with configuration such that it causes an exception during application submission, the files related to the service are not cleared from hdfs automatically. Ideally the files stored to hdfs cannot be used in future to start or stop the service as the configuration itself is invalid. So, we should destroy the service and remove the residual files in hdfs, if any YarnException is thrown. For example if the user submits a service with configuring with "memory" more than the maximum resource, the service fails but the files in hdfs are not cleared. But these files should be cleared. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10246) Enable YARN Router to have a dedicated Zookeeper
[ https://issues.apache.org/jira/browse/YARN-10246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10246: Attachment: YARN-10246.002.patch > Enable YARN Router to have a dedicated Zookeeper > > > Key: YARN-10246 > URL: https://issues.apache.org/jira/browse/YARN-10246 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10246.001.patch, YARN-10246.002.patch > > > Currently, we have a single parameter hadoop.zk.address for Router and > Resourcemanager, Due to this we need have FederationStateStore and > RMStateStore on the same Zookeeper instance. > With the above topology there can be a load on ZooKeeper, since all > subcluster RMs will write to single ZooKeeper. > So, If we Introduce a new configuration such as hadoop.federation.zk.address > we can have FederationStateStore on a dedicated Zookeeper. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10246) Enable Yarn Router to have a dedicated Zookeeper
[ https://issues.apache.org/jira/browse/YARN-10246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10246: Attachment: YARN-10246.001.patch > Enable Yarn Router to have a dedicated Zookeeper > > > Key: YARN-10246 > URL: https://issues.apache.org/jira/browse/YARN-10246 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10246.001.patch > > > Currently, we have a single parameter hadoop.zk.address for Router and > Resourcemanager, Due to this we need have FederationStateStore and > RMStateStore on the same Zookeeper instance. > With the above topology there can be a load on ZooKeeper, since all > subcluster RMs will write to single ZooKeeper. > So, If we Introduce a new configuration such as hadoop.federation.zk.address > we can have FederationStateStore on a dedicated Zookeeper. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10246) Enable Yarn Router to have a dedicated Zookeeper
[ https://issues.apache.org/jira/browse/YARN-10246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10246: Attachment: (was: YARN-10246.001.patch) > Enable Yarn Router to have a dedicated Zookeeper > > > Key: YARN-10246 > URL: https://issues.apache.org/jira/browse/YARN-10246 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10246.001.patch > > > Currently, we have a single parameter hadoop.zk.address for Router and > Resourcemanager, Due to this we need have FederationStateStore and > RMStateStore on the same Zookeeper instance. > With the above topology there can be a load on ZooKeeper, since all > subcluster RMs will write to single ZooKeeper. > So, If we Introduce a new configuration such as hadoop.federation.zk.address > we can have FederationStateStore on a dedicated Zookeeper. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10246) Enable Yarn Router to have a dedicated Zookeeper
[ https://issues.apache.org/jira/browse/YARN-10246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10246: Attachment: YARN-10246.001.patch > Enable Yarn Router to have a dedicated Zookeeper > > > Key: YARN-10246 > URL: https://issues.apache.org/jira/browse/YARN-10246 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10246.001.patch > > > Currently, we have a single parameter hadoop.zk.address for Router and > Resourcemanager, Due to this we need have FederationStateStore and > RMStateStore on the same Zookeeper instance. > With the above topology there can be a load on ZooKeeper, since all > subcluster RMs will write to single ZooKeeper. > So, If we Introduce a new configuration such as hadoop.federation.zk.address > we can have FederationStateStore on a dedicated Zookeeper. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10246) Enable Yarn Router to have a dedicated Zookeeper
D M Murali Krishna Reddy created YARN-10246: --- Summary: Enable Yarn Router to have a dedicated Zookeeper Key: YARN-10246 URL: https://issues.apache.org/jira/browse/YARN-10246 Project: Hadoop YARN Issue Type: Improvement Components: federation, router Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy Currently, we have a single parameter hadoop.zk.address for Router and Resourcemanager, Due to this we need have FederationStateStore and RMStateStore on the same Zookeeper instance. With the above topology there can be a load on ZooKeeper, since all subcluster RMs will write to single ZooKeeper. So, If we Introduce a new configuration such as hadoop.federation.zk.address we can have FederationStateStore on a dedicated Zookeeper. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.004.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch, YARN-10141.004.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.003.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch, > YARN-10141.003.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
[ https://issues.apache.org/jira/browse/YARN-10141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10141: Attachment: YARN-10141.002.patch > Interceptor in FederationInterceptorREST doesnt update on RM switchover > --- > > Key: YARN-10141 > URL: https://issues.apache.org/jira/browse/YARN-10141 > Project: Hadoop YARN > Issue Type: Bug > Components: federation, restapi >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > Attachments: YARN-10141.001.patch, YARN-10141.002.patch > > > In Federation Setup, In the event of a RM switchover in a subcluster the > interceptor for that subcluster in FederationInterceptorREST doesnt get > updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the > subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10111) In Federation cluster Distributed Shell Application submission fails as YarnClient#getQueueInfo is not implemented
[ https://issues.apache.org/jira/browse/YARN-10111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy updated YARN-10111: Parent: YARN-10144 Issue Type: Sub-task (was: Bug) > In Federation cluster Distributed Shell Application submission fails as > YarnClient#getQueueInfo is not implemented > -- > > Key: YARN-10111 > URL: https://issues.apache.org/jira/browse/YARN-10111 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Sushanta Sen >Assignee: Bilwa S T >Priority: Blocker > > In Federation cluster Distributed Shell Application submission fails as > YarnClient#getQueueInfo is not implemented. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10121) In Federation executing yarn queue status command throws an exception
[
https://issues.apache.org/jira/browse/YARN-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10121:
Parent: YARN-10144
Issue Type: Sub-task (was: Bug)
> In Federation executing yarn queue status command throws an exception
> -
>
> Key: YARN-10121
> URL: https://issues.apache.org/jira/browse/YARN-10121
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, yarn
>Reporter: Sushanta Sen
>Assignee: Bilwa S T
>Priority: Major
>
> yarn queue status is failing, prompting an error
> “org.apache.commons.lang.NotImplementedException: Code is not implemented”.
> {noformat}
> ./yarn queue -status default
> Exception in thread "main" org.apache.commons.lang.NotImplementedException:
> Code is not implemented
> at
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.getQueueInfo(FederationClientInterceptor.java:715)
> at
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.getQueueInfo(RouterClientRMService.java:246)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getQueueInfo(ApplicationClientProtocolPBServiceImpl.java:328)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:591)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:530)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:928)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:863)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2793)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getQueueInfo(ApplicationClientProtocolPBClientImpl.java:341)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy8.getQueueInfo(Unknown Source)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getQueueInfo(YarnClientImpl.java:650)
> at
> org.apache.hadoop.yarn.client.cli.QueueCLI.listQueue(QueueCLI.java:111)
> at org.apache.hadoop.yarn.client.cli.QueueCLI.run(QueueCLI.java:78)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.yarn.client.cli.QueueCLI.main(QueueCLI.java:50)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10132) For Federation,yarn applicationattempt fail command throws an exception
[
https://issues.apache.org/jira/browse/YARN-10132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10132:
Parent: YARN-10144
Issue Type: Sub-task (was: Bug)
> For Federation,yarn applicationattempt fail command throws an exception
> ---
>
> Key: YARN-10132
> URL: https://issues.apache.org/jira/browse/YARN-10132
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Sushanta Sen
>Assignee: Bilwa S T
>Priority: Major
>
> yarn applicationattempt fail command is failing with exception
> “org.apache.commons.lang.NotImplementedException: Code is not implemented”.
> {noformat}
> ./yarn applicationattempt -fail appattempt_1581497870689_0001_01
> Failing attempt appattempt_1581497870689_0001_01 of application
> application_1581497870689_0001
> 2020-02-12 20:48:48,530 INFO impl.YarnClientImpl: Failing application attempt
> appattempt_1581497870689_0001_01
> Exception in thread "main" org.apache.commons.lang.NotImplementedException:
> Code is not implemented
> at
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.failApplicationAttempt(FederationClientInterceptor.java:980)
> at
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.failApplicationAttempt(RouterClientRMService.java:388)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.failApplicationAttempt(ApplicationClientProtocolPBServiceImpl.java:210)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:581)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:530)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:928)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:863)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2793)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.failApplicationAttempt(ApplicationClientProtocolPBClientImpl.java:223)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy8.failApplicationAttempt(Unknown Source)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.failApplicationAttempt(YarnClientImpl.java:447)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.failApplicationAttempt(ApplicationCLI.java:985)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:455)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:119)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---
[jira] [Updated] (YARN-10122) In Federation,executing yarn container signal command throws an exception
[
https://issues.apache.org/jira/browse/YARN-10122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
D M Murali Krishna Reddy updated YARN-10122:
Parent: YARN-10144
Issue Type: Sub-task (was: Bug)
> In Federation,executing yarn container signal command throws an exception
> -
>
> Key: YARN-10122
> URL: https://issues.apache.org/jira/browse/YARN-10122
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, yarn
>Reporter: Sushanta Sen
>Assignee: Bilwa S T
>Priority: Major
>
> Executing yarn container signal command failed, prompting an error
> “org.apache.commons.lang.NotImplementedException: Code is not implemented”.
> {noformat}
> ./yarn container -signal container_e79_1581316978887_0001_01_10
> Signalling container container_e79_1581316978887_0001_01_10
> 2020-02-10 14:51:18,045 INFO impl.YarnClientImpl: Signalling container
> container_e79_1581316978887_0001_01_10 with command OUTPUT_THREAD_DUMP
> Exception in thread "main" org.apache.commons.lang.NotImplementedException:
> Code is not implemented
> at
> org.apache.hadoop.yarn.server.router.clientrm.FederationClientInterceptor.signalToContainer(FederationClientInterceptor.java:993)
> at
> org.apache.hadoop.yarn.server.router.clientrm.RouterClientRMService.signalToContainer(RouterClientRMService.java:403)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.signalToContainer(ApplicationClientProtocolPBServiceImpl.java:629)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:629)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:530)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:928)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:863)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2793)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateRuntimeException(RPCUtil.java:85)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:122)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.signalToContainer(ApplicationClientProtocolPBClientImpl.java:620)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy8.signalToContainer(Unknown Source)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.signalToContainer(YarnClientImpl.java:949)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.signalToContainer(ApplicationCLI.java:717)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:478)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:119)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (YARN-10141) Interceptor in FederationInterceptorREST doesnt update on RM switchover
D M Murali Krishna Reddy created YARN-10141: --- Summary: Interceptor in FederationInterceptorREST doesnt update on RM switchover Key: YARN-10141 URL: https://issues.apache.org/jira/browse/YARN-10141 Project: Hadoop YARN Issue Type: Bug Components: federation, restapi Reporter: D M Murali Krishna Reddy Assignee: D M Murali Krishna Reddy In Federation Setup, In the event of a RM switchover in a subcluster the interceptor for that subcluster in FederationInterceptorREST doesnt get updated. Due to this, Cluster Nodes REST API doesnt return the nodes from the subcluster in which the RM switchover has occured. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10125) In Federation, kill application from client does not kill Unmanaged AM's and containers launched by Unmanaged AM
[ https://issues.apache.org/jira/browse/YARN-10125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] D M Murali Krishna Reddy reassigned YARN-10125: --- Assignee: D M Murali Krishna Reddy (was: Bilwa S T) > In Federation, kill application from client does not kill Unmanaged AM's and > containers launched by Unmanaged AM > > > Key: YARN-10125 > URL: https://issues.apache.org/jira/browse/YARN-10125 > Project: Hadoop YARN > Issue Type: Bug > Components: client, federation, router >Reporter: D M Murali Krishna Reddy >Assignee: D M Murali Krishna Reddy >Priority: Major > > In Federation, killing an application from client using "bin/yarn application > -kill ", kills the containers only of the home subcluster, > the Unmanaged AM and the containers launched in other subcluster are not > being killed causing blocking of resources. > The containers get killed after the task gets completed and The unmanaged AM > gets killed after 10 minutes of killing the application, killing any > remaining running containers in that subcluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10125) In Federation, kill application from client does not kill Unmanaged AM's and containers launched by Unmanaged AM
D M Murali Krishna Reddy created YARN-10125: --- Summary: In Federation, kill application from client does not kill Unmanaged AM's and containers launched by Unmanaged AM Key: YARN-10125 URL: https://issues.apache.org/jira/browse/YARN-10125 Project: Hadoop YARN Issue Type: Bug Components: client, federation, router Reporter: D M Murali Krishna Reddy In Federation, killing an application from client using "bin/yarn application -kill ", kills the containers only of the home subcluster, the Unmanaged AM and the containers launched in other subcluster are not being killed causing blocking of resources. The containers get killed after the task gets completed and The unmanaged AM gets killed after 10 minutes of killing the application, killing any remaining running containers in that subcluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
