[jira] [Commented] (YARN-8793) QueuePlacementPolicy bind more information to assgining result

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621585#comment-16621585 ] Hadoop QA commented on YARN-8793: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (YARN-8793) QueuePlacementPolicy bind more information to assgining result

2018-09-20 Thread Shuai Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuai Zhang updated YARN-8793: -- Attachment: YARN-8793.007.patch > QueuePlacementPolicy bind more information to assgining result > -

[jira] [Commented] (YARN-8549) Adding a NoOp timeline writer and reader plugin classes for ATSv2

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621592#comment-16621592 ] Hadoop QA commented on YARN-8549: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (YARN-8793) QueuePlacementPolicy bind more information to assgining result

2018-09-20 Thread Shuai Zhang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shuai Zhang updated YARN-8793: -- Attachment: YARN-8793.008.patch > QueuePlacementPolicy bind more information to assgining result > -

[jira] [Created] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
Tao Yang created YARN-8804: -- Summary: resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues Key: YARN-8804 URL: https://issues.apache.org/jira/browse/YARN-8804 P

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Description: This problem is due to YARN-4280, parent queue will deduct child queue's headroom when the child

[jira] [Commented] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621611#comment-16621611 ] Antal Bálint Steinbach commented on YARN-8759: -- Thank you [~sunilg] . > Copy

[jira] [Commented] (YARN-8270) Adding JMX Metrics for Timeline Collector and Reader

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621628#comment-16621628 ] Hadoop QA commented on YARN-8270: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote

[jira] [Updated] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated YARN-8804: --- Attachment: YARN-8804.001.patch > resourceLimits may be wrongly calculated when leaf-queue is blocked in > cl

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Tao Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621631#comment-16621631 ] Tao Yang commented on YARN-8804: Attached v1 patch which will add a blockedHeadroom field

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621659#comment-16621659 ] Antal Bálint Steinbach commented on YARN-8468: -- Hi [~cheersyang] , Some chan

[jira] [Comment Edited] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621659#comment-16621659 ] Antal Bálint Steinbach edited comment on YARN-8468 at 9/20/18 8:38 AM: -

[jira] [Comment Edited] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621659#comment-16621659 ] Antal Bálint Steinbach edited comment on YARN-8468 at 9/20/18 8:39 AM: -

[jira] [Comment Edited] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621659#comment-16621659 ] Antal Bálint Steinbach edited comment on YARN-8468 at 9/20/18 8:39 AM: -

[jira] [Commented] (YARN-8759) Copy of "resource-types.xml" is not deleted if test fails, causes other test failures

2018-09-20 Thread Manikandan R (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621695#comment-16621695 ] Manikandan R commented on YARN-8759: {quote}I just set it to null, because there are s

[jira] [Commented] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621705#comment-16621705 ] Hadoop QA commented on YARN-8665: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (YARN-8793) QueuePlacementPolicy bind more information to assgining result

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621711#comment-16621711 ] Hadoop QA commented on YARN-8793: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote

[jira] [Assigned] (YARN-8783) Improve the documentation for the docker.trusted.registries configuration

2018-09-20 Thread Simon Prewo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Prewo reassigned YARN-8783: - Assignee: Simon Prewo > Improve the documentation for the docker.trusted.registries configuration

[jira] [Updated] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Attachment: YARN-8468.015.patch > Limit container sizes per queue in FairScheduler

[jira] [Updated] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Attachment: (was: YARN-8468.015.patch) > Limit container sizes per queue in Fa

[jira] [Commented] (YARN-8783) Improve the documentation for the docker.trusted.registries configuration

2018-09-20 Thread Simon Prewo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621786#comment-16621786 ] Simon Prewo commented on YARN-8783: --- [~jlowe]  [~eyang] Sure, I am happy to take care o

[jira] [Updated] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Attachment: YARN-8468.015.patch > Limit container sizes per queue in FairScheduler

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621796#comment-16621796 ] Antal Bálint Steinbach commented on YARN-8468: -- Hi [~cheersyang] , [~leftnote

[jira] [Commented] (YARN-5745) [YARN-3368] Disable war packaging under default profile for new YARN UI

2018-09-20 Thread Zoltan Siegl (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621822#comment-16621822 ] Zoltan Siegl commented on YARN-5745: Thanks for this Wangda,  LGTM +1 (non-binding)

[jira] [Commented] (YARN-5745) [YARN-3368] Disable war packaging under default profile for new YARN UI

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621823#comment-16621823 ] Hadoop QA commented on YARN-5745: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621843#comment-16621843 ] Hadoop QA commented on YARN-8468: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-4858) start-yarn and stop-yarn scripts to support timeline and sharedcachemanager

2018-09-20 Thread Gergely Pollak (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621863#comment-16621863 ] Gergely Pollak commented on YARN-4858: -- [~subru] thank you for the patch! LGTM+1 (No

[jira] [Commented] (YARN-7834) [GQ] Rebalance queue configuration for load-balancing and locality affinities

2018-09-20 Thread Gergely Pollak (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621876#comment-16621876 ] Gergely Pollak commented on YARN-7834: -- [~curino] thank you for the patch, while I'm

[jira] [Comment Edited] (YARN-7834) [GQ] Rebalance queue configuration for load-balancing and locality affinities

2018-09-20 Thread Gergely Pollak (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621876#comment-16621876 ] Gergely Pollak edited comment on YARN-7834 at 9/20/18 11:46 AM:

[jira] [Created] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Shane Kumpf (JIRA)
Shane Kumpf created YARN-8805: - Summary: Automatically convert the launch command to the exec form when using entrypoint support Key: YARN-8805 URL: https://issues.apache.org/jira/browse/YARN-8805 Project

[jira] [Commented] (YARN-7834) [GQ] Rebalance queue configuration for load-balancing and locality affinities

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621935#comment-16621935 ] Hadoop QA commented on YARN-7834: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-4858) start-yarn and stop-yarn scripts to support timeline and sharedcachemanager

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621948#comment-16621948 ] Hadoop QA commented on YARN-4858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8769) [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job

2018-09-20 Thread Sunil Govindan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622108#comment-16622108 ] Sunil Govindan commented on YARN-8769: -- +1. Committing shortly. > [Submarine] Allow

[jira] [Commented] (YARN-8800) Updated documentation of Submarine with latest examples.

2018-09-20 Thread Sunil Govindan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622110#comment-16622110 ] Sunil Govindan commented on YARN-8800: -- Hi [~leftnoteasy] Cud u pls help in this tic

[jira] [Created] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8806: -- Summary: Enable local staging directory and clean it up when submarine job is submitted Key: YARN-8806 URL: https://issues.apache.org/jira/browse/YARN-8806 Project: Hadoop YARN

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622156#comment-16622156 ] Hadoop QA commented on YARN-8806: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622159#comment-16622159 ] Zac Zhou commented on YARN-8725: [~leftnoteasy], [~tangzhankun]. Thanks a lot for your eff

[jira] [Comment Edited] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622159#comment-16622159 ] Zac Zhou edited comment on YARN-8725 at 9/20/18 2:51 PM: - [~leftno

[jira] [Commented] (YARN-8801) java doc comments in docker-util.h is confusing

2018-09-20 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622193#comment-16622193 ] Eric Yang commented on YARN-8801: - +1 > java doc comments in docker-util.h is confusing >

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.002.patch > Enable local staging directory and clean it up when submarine job is submitt

[jira] [Commented] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622208#comment-16622208 ] Eric Yang commented on YARN-8805: - This has been documented in https://github.com/apache/

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622280#comment-16622280 ] Hadoop QA commented on YARN-8806: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622281#comment-16622281 ] Jason Lowe commented on YARN-8804: -- Thanks for the report and patch! Nice analysis. A n

[jira] [Updated] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-20 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated YARN-8789: -- Attachment: YARN-8789.3.patch > Add BoundedQueue to AsyncDispatcher > --

[jira] [Commented] (YARN-8774) Memory leak when CapacityScheduler allocates from reserved container with non-default label

2018-09-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622323#comment-16622323 ] Eric Payne commented on YARN-8774: -- [~Tao Yang], The patch and UT look good to me. Howeve

[jira] [Created] (YARN-8807) FairScheduler crashes RM with oversubscription turned on if an application is killed.

2018-09-20 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-8807: Summary: FairScheduler crashes RM with oversubscription turned on if an application is killed. Key: YARN-8807 URL: https://issues.apache.org/jira/browse/YARN-8807 Project: Ha

[jira] [Created] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-20 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-8808: Summary: Use aggregate container utilization instead of node utilization to determine resources available for oversubscription Key: YARN-8808 URL: https://issues.apache.org/jira/browse/YA

[jira] [Created] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Haibo Chen (JIRA)
Haibo Chen created YARN-8809: Summary: Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released. Key: YARN-8809 URL: https://issues.apache.org/jira/browse/YARN-8809 Proje

[jira] [Commented] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622355#comment-16622355 ] Wangda Tan commented on YARN-8468: -- Thanks [~bsteinbach], I think YARN-8720 only covers

[jira] [Updated] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-20 Thread BELUGA BEHR (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated YARN-8789: -- Description: I recently came across a scenario where an MR ApplicationMaster was failing with an OOM ex

[jira] [Updated] (YARN-8468) Limit container sizes per queue in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Attachment: YARN-8468.016.patch > Limit container sizes per queue in FairScheduler

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Summary: Enable queue based maximum allocation limit and implement it in FairSched

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Updated] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8808: - Description: Resource oversubscription should be bound to the amount of the resources that can be allocate

[jira] [Updated] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8809: - Attachment: YARN-8809-YARN-1011.00.patch > Fair Scheduler does not decrement queue metrics when OPPORTUNIS

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Commented] (YARN-8801) java doc comments in docker-util.h is confusing

2018-09-20 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622383#comment-16622383 ] Hudson commented on YARN-8801: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #150

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Updated] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8808: - Attachment: YARN-8808-YARN-1011.00.patch > Use aggregate container utilization instead of node utilization

[jira] [Updated] (YARN-8468) Enable queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Updated] (YARN-8808) Use aggregate container utilization instead of node utilization to determine resources available for oversubscription

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8808: - Attachment: (was: YARN-8808-YARN-1011.00.patch) > Use aggregate container utilization instead of node

[jira] [Updated] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Summary: Enable the use of queue based maximum container allocation limit and impl

[jira] [Updated] (YARN-8468) Enable the use of queue based maximum allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Summary: Enable the use of queue based maximum allocation limit and implement it i

[jira] [Updated] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antal Bálint Steinbach updated YARN-8468: - Description: When using any scheduler, you can use "yarn.scheduler.maximum-allocat

[jira] [Commented] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-09-20 Thread JIRA
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622433#comment-16622433 ] Antal Bálint Steinbach commented on YARN-8468: -- Hi [~leftnoteasy] Sure, I up

[jira] [Commented] (YARN-8801) java doc comments in docker-util.h is confusing

2018-09-20 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622438#comment-16622438 ] Zian Chen commented on YARN-8801: - Thank you [~eyang] > java doc comments in docker-util.

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622445#comment-16622445 ] Arun Suresh commented on YARN-8809: --- Thanks for the patch [~haibochen].. >From the patch

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2018-09-20 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622450#comment-16622450 ] Arun Suresh commented on YARN-1011: --- Planning on spending more cycles on this now. Looki

[jira] [Comment Edited] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2018-09-20 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622450#comment-16622450 ] Arun Suresh edited comment on YARN-1011 at 9/20/18 6:11 PM: Pl

[jira] [Commented] (YARN-7225) Add queue and partition info to RM audit log

2018-09-20 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622458#comment-16622458 ] Eric Payne commented on YARN-7225: -- Thanks [~jhung] for raising this issue. This came up

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622460#comment-16622460 ] Wangda Tan commented on YARN-8804: -- Thanks [~Tao Yang] for the nice analysis, I'm not sur

[jira] [Commented] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-20 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622495#comment-16622495 ] Zian Chen commented on YARN-8785: - Hi [~simonprewo], would you like to work on this Jira a

[jira] [Commented] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622499#comment-16622499 ] Zian Chen commented on YARN-8805: - Yes, just checked the latest released doc, [https://ha

[jira] [Commented] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Shane Kumpf (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622503#comment-16622503 ] Shane Kumpf commented on YARN-8805: --- Thanks, [~Zian Chen]. Please feel free to take this

[jira] [Commented] (YARN-5215) Scheduling containers based on external load in the servers

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622518#comment-16622518 ] Hadoop QA commented on YARN-5215: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Young Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-8658: - Attachment: YARN-8658-branch-2.09.patch > [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInter

[jira] [Reopened] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Giovanni Matteo Fumarola (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giovanni Matteo Fumarola reopened YARN-8658: Re-opening the Jira for Branch-2 patch. > [AMRMProxy] Metrics for AMRMClientRel

[jira] [Updated] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-20 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated YARN-8665: Attachment: YARN-8665.003.patch > Yarn Service Upgrade: Support cancelling upgrade > --

[jira] [Assigned] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zian Chen reassigned YARN-8805: --- Assignee: Zian Chen > Automatically convert the launch command to the exec form when using > entrypo

[jira] [Commented] (YARN-8805) Automatically convert the launch command to the exec form when using entrypoint support

2018-09-20 Thread Zian Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622555#comment-16622555 ] Zian Chen commented on YARN-8805: - Thanks [~shaneku...@gmail.com], I'll work on the patch

[jira] [Commented] (YARN-8468) Enable the use of queue based maximum container allocation limit and implement it in FairScheduler

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622605#comment-16622605 ] Hadoop QA commented on YARN-8468: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622648#comment-16622648 ] Hadoop QA commented on YARN-8809: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622653#comment-16622653 ] Hadoop QA commented on YARN-8658: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8777) Container Executor C binary change to execute interactive docker command

2018-09-20 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622690#comment-16622690 ] Eric Badger commented on YARN-8777: --- Thanks for the updated patch, [~eyang]! {noformat}

[jira] [Commented] (YARN-8789) Add BoundedQueue to AsyncDispatcher

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622719#comment-16622719 ] Hadoop QA commented on YARN-8789: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Assigned] (YARN-8785) Error Message "Invalid docker rw mount" not helpful

2018-09-20 Thread Simon Prewo (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Prewo reassigned YARN-8785: - Assignee: Simon Prewo > Error Message "Invalid docker rw mount" not helpful > ---

[jira] [Updated] (YARN-8807) FairScheduler crashes RM with oversubscription turned on if an application is killed.

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8807: - Description: When an application, that has got opportunistic containers allocated, is killed, its contain

[jira] [Updated] (YARN-8807) FairScheduler crashes RM with oversubscription turned on if an application is killed.

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-8807: - Attachment: YARN-8807-YARN-1011.00.patch > FairScheduler crashes RM with oversubscription turned on if an

[jira] [Updated] (YARN-7599) [GPG] ApplicationCleaner in Global Policy Generator

2018-09-20 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7599: --- Attachment: YARN-7599-YARN-7402.v7.patch > [GPG] ApplicationCleaner in Global Policy Generator > -

[jira] [Updated] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Young Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Young Chen updated YARN-8658: - Attachment: YARN-8658-branch-2.10.patch > [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInter

[jira] [Commented] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Young Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622752#comment-16622752 ] Young Chen commented on YARN-8658: -- Fixed indentation issues from merging conflicts. yarn

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622754#comment-16622754 ] Haibo Chen commented on YARN-8809: -- [~asuresh] The core change is the overriding of comp

[jira] [Comment Edited] (YARN-8658) [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor

2018-09-20 Thread Young Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622752#comment-16622752 ] Young Chen edited comment on YARN-8658 at 9/20/18 9:50 PM: --- Fixe

[jira] [Commented] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-20 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622756#comment-16622756 ] Hadoop QA commented on YARN-8665: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622762#comment-16622762 ] Haibo Chen commented on YARN-8809: -- The failed unit tests are all unrelated, see YARN-738

[jira] [Commented] (YARN-8665) Yarn Service Upgrade: Support cancelling upgrade

2018-09-20 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622765#comment-16622765 ] Eric Yang commented on YARN-8665: - [~csingh] Thank you for the patch. Patch 3 works corre

[jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers

2018-09-20 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622767#comment-16622767 ] Haibo Chen commented on YARN-1011: -- I'm +1 on moving this. > [Umbrella] Schedule contain

[jira] [Commented] (YARN-8804) resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues

2018-09-20 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622771#comment-16622771 ] Jason Lowe commented on YARN-8804: -- >From the looks of YARN-8513 this appears to be a sep

[jira] [Commented] (YARN-8809) Fair Scheduler does not decrement queue metrics when OPPORTUNISTIC containers are released.

2018-09-20 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622784#comment-16622784 ] Arun Suresh commented on YARN-8809: --- Got it... So, in essence, FS has a unified code pat

  1   2   >