[jira] [Commented] (YARN-8508) On NodeManager container gets cleaned up before its pid file is created

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564256#comment-16564256 ] Wangda Tan commented on YARN-8508: -- Committed to branch-3.1.1, thanks [~csingh]! > On NodeManager

[jira] [Commented] (YARN-8546) Resource leak caused by a reserved container being released more than once under async scheduling

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564254#comment-16564254 ] Wangda Tan commented on YARN-8546: -- Committed to branch-3.1.1, thanks [~Tao Yang]/[~cheersyang] >

[jira] [Updated] (YARN-8301) Yarn Service Upgrade: Add documentation

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8301: - Fix Version/s: (was: 3.1.2) 3.1.1 > Yarn Service Upgrade: Add documentation >

[jira] [Commented] (YARN-8528) Final states in ContainerAllocation might be modified externally causing unexpected allocation results

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564253#comment-16564253 ] Wangda Tan commented on YARN-8528: -- Committed to branch-3.1.1, thanks [~cheersyang] > Final states in

[jira] [Updated] (YARN-8546) Resource leak caused by a reserved container being released more than once under async scheduling

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8546: - Fix Version/s: (was: 3.1.2) 3.1.1 > Resource leak caused by a reserved container

[jira] [Updated] (YARN-8528) Final states in ContainerAllocation might be modified externally causing unexpected allocation results

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8528: - Fix Version/s: (was: 3.1.2) 3.1.1 > Final states in ContainerAllocation might be

[jira] [Commented] (YARN-8559) Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564213#comment-16564213 ] Wangda Tan commented on YARN-8559: -- Thanks [~cheersyang], Some suggestions: 1) Instead of creating a

[jira] [Commented] (YARN-8606) Opportunistic scheduling doesnt work after failover

2018-07-31 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564089#comment-16564089 ] Wangda Tan commented on YARN-8606: -- [~bibinchundatt], is this a regression recently? If yes, which JIRA

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562483#comment-16562483 ] Wangda Tan commented on YARN-8509: -- [~eepayne], if you have some bandwidth, could u help to check this

[jira] [Updated] (YARN-8603) [UI2] Latest run application should be listed first in the RM UI

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8603: - Reporter: Sumana Sathish (was: Akhil PB) > [UI2] Latest run application should be listed first in the RM

[jira] [Commented] (YARN-8591) [ATSv2] NPE while checking for entity acl in non-secure cluster

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562368#comment-16562368 ] Wangda Tan commented on YARN-8591: -- Updated fixed version to 3.1.2 given this don't exist in branch-3.1.1

[jira] [Updated] (YARN-8508) On NodeManager container gets cleaned up before its pid file is created

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8508: - Priority: Critical (was: Major) > On NodeManager container gets cleaned up before its pid file is

[jira] [Commented] (YARN-8508) On NodeManager container gets cleaned up before its pid file is created

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562364#comment-16562364 ] Wangda Tan commented on YARN-8508: -- I think it is important to get it backported to branch-3.1.1, I'm

[jira] [Commented] (YARN-8545) YARN native service should return container if launch failed

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562358#comment-16562358 ] Wangda Tan commented on YARN-8545: -- I think it is important to get it backported to branch-3.1.1, I'm

[jira] [Commented] (YARN-8546) Resource leak caused by a reserved container being released more than once under async scheduling

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562357#comment-16562357 ] Wangda Tan commented on YARN-8546: -- I think it is important to get it backported to branch-3.1.1, I'm

[jira] [Commented] (YARN-8301) Yarn Service Upgrade: Add documentation

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562354#comment-16562354 ] Wangda Tan commented on YARN-8301: -- Updated fixed version to 3.1.2 given this don't exist in

[jira] [Updated] (YARN-8301) Yarn Service Upgrade: Add documentation

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8301: - Fix Version/s: (was: 3.1.1) 3.1.2 > Yarn Service Upgrade: Add documentation >

[jira] [Commented] (YARN-8528) Final states in ContainerAllocation might be modified externally causing unexpected allocation results

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562350#comment-16562350 ] Wangda Tan commented on YARN-8528: -- Updated fixed version to 3.1.2 given this don't exist in

[jira] [Updated] (YARN-8301) Yarn Service Upgrade: Add documentation

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8301: - Priority: Critical (was: Major) > Yarn Service Upgrade: Add documentation >

[jira] [Updated] (YARN-8528) Final states in ContainerAllocation might be modified externally causing unexpected allocation results

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8528: - Fix Version/s: (was: 3.1.1) 3.1.2 > Final states in ContainerAllocation might be

[jira] [Updated] (YARN-8508) On NodeManager container gets cleaned up before its pid file is created

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8508: - Summary: On NodeManager container gets cleaned up before its pid file is created (was: GPU does not get

[jira] [Updated] (YARN-8330) Avoid publishing reserved container to ATS from RM

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8330: - Summary: Avoid publishing reserved container to ATS from RM (was: Avoid publishing reserved container to

[jira] [Updated] (YARN-8330) Avoid publishing reserved container to ATS

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8330: - Summary: Avoid publishing reserved container to ATS (was: An extra container got launched by RM for

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562288#comment-16562288 ] Wangda Tan commented on YARN-8418: -- Thanks [~bibinchundatt], +1 to the latest patch, it gonna be ideal if

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562026#comment-16562026 ] Wangda Tan commented on YARN-8418: -- [~rohithsharma], regarding to the YARN-4984 revert, Initially I have

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-29 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561188#comment-16561188 ] Wangda Tan commented on YARN-8418: -- Thanks [~bibinchundatt] for updating the patch and

[jira] [Updated] (YARN-8563) [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

2018-07-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8563: - Summary: [Submarine] Support users to specify Python/TF package/version/dependencies for training job.

[jira] [Updated] (YARN-8561) [Submarine] Add initial implementation: training job submission and job history retrieve.

2018-07-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8561: - Summary: [Submarine] Add initial implementation: training job submission and job history retrieve. (was:

[jira] [Created] (YARN-8563) Support users to specify Python/TF package/version/dependencies for training job.

2018-07-21 Thread Wangda Tan (JIRA)
Wangda Tan created YARN-8563: Summary: Support users to specify Python/TF package/version/dependencies for training job. Key: YARN-8563 URL: https://issues.apache.org/jira/browse/YARN-8563 Project:

[jira] [Commented] (YARN-8558) NM recovery level db not cleaned up properly on container finish

2018-07-21 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551755#comment-16551755 ] Wangda Tan commented on YARN-8558: -- [~bibinchundatt], I think we should have a follow up Jira to make

[jira] [Updated] (YARN-8135) Hadoop {Submarine} Project: Simple and scalable deployment of deep learning training / serving jobs on Hadoop

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8135: - Attachment: (was: YARN-8135.poc.001.patch) > Hadoop {Submarine} Project: Simple and scalable

[jira] [Commented] (YARN-8135) Hadoop {Submarine} Project: Simple and scalable deployment of deep learning training / serving jobs on Hadoop

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551490#comment-16551490 ] Wangda Tan commented on YARN-8135: -- Discussed with many folks, thanks inputs from: [~sunilg],

[jira] [Commented] (YARN-8561) Add submarine initial implementation: training job submission and job history retrieve.

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551487#comment-16551487 ] Wangda Tan commented on YARN-8561: -- Attached initial patch to get some early feedbacks. Please refer to

[jira] [Updated] (YARN-8561) Add submarine initial implementation: training job submission and job history retrieve.

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8561: - Attachment: YARN-8561.001.patch > Add submarine initial implementation: training job submission and job

[jira] [Created] (YARN-8561) Add submarine initial implementation: training job submission and job history retrieve.

2018-07-20 Thread Wangda Tan (JIRA)
Wangda Tan created YARN-8561: Summary: Add submarine initial implementation: training job submission and job history retrieve. Key: YARN-8561 URL: https://issues.apache.org/jira/browse/YARN-8561 Project:

[jira] [Updated] (YARN-8360) Yarn service conflict between restart policy and NM configuration

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8360: - Target Version/s: 3.2.0, 3.1.2 Priority: Critical (was: Major) > Yarn service conflict

[jira] [Updated] (YARN-8544) [DS] AM registration fails when hadoop authorization is enabled

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8544: - Target Version/s: 3.2.0, 3.1.2 (was: 3.1.1) > [DS] AM registration fails when hadoop authorization is

[jira] [Commented] (YARN-8480) Add boolean option for resources

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551245#comment-16551245 ] Wangda Tan commented on YARN-8480: -- [~templedf], {quote}It also looks to me like the PCM is an

[jira] [Commented] (YARN-8330) An extra container got launched by RM for yarn-service

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551137#comment-16551137 ] Wangda Tan commented on YARN-8330: -- [~rohithsharma],  To me it is fine that we publish

[jira] [Commented] (YARN-8559) Expose scheduling configuration info in Resource Manager's /conf endpoint

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551066#comment-16551066 ] Wangda Tan commented on YARN-8559: -- [~cheersyang] / [~banditka],  Actually the scheduler-conf was not

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551057#comment-16551057 ] Wangda Tan commented on YARN-8418: -- For logics of the patch, a couple of comments:  1) Is it possible to

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-20 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551034#comment-16551034 ] Wangda Tan commented on YARN-8418: -- [~bibinchundatt],  {quote}As part of YARN-4984 we disabled the thread

[jira] [Commented] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549964#comment-16549964 ] Wangda Tan commented on YARN-8418: -- [~bibinchundatt], I'm a bit hesitated to get this patch committed

[jira] [Updated] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8418: - Target Version/s: 3.1.1 (was: 3.1.2) > App local logs could leaked if log aggregation fails to

[jira] [Updated] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8418: - Target Version/s: 3.1.2 (was: 3.1.1) > App local logs could leaked if log aggregation fails to

[jira] [Updated] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8418: - Target Version/s: 3.1.1 (was: 3.1.2) > App local logs could leaked if log aggregation fails to

[jira] [Commented] (YARN-8474) sleeper service fails to launch with "Authentication Required"

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549901#comment-16549901 ] Wangda Tan commented on YARN-8474: -- Bulk update: moved all 3.1.1 non-blocker issues, please move back if

[jira] [Commented] (YARN-8234) Improve RM system metrics publisher's performance by pushing events to timeline server in batch

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549902#comment-16549902 ] Wangda Tan commented on YARN-8234: -- Bulk update: moved all 3.1.1 non-blocker issues, please move back if

[jira] [Updated] (YARN-8474) sleeper service fails to launch with "Authentication Required"

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8474: - Target Version/s: 3.2.0, 3.1.2 (was: 3.2.0, 3.1.1) > sleeper service fails to launch with

[jira] [Updated] (YARN-8234) Improve RM system metrics publisher's performance by pushing events to timeline server in batch

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8234: - Target Version/s: 3.1.2 (was: 3.1.1, 3.1.2) > Improve RM system metrics publisher's performance by

[jira] [Updated] (YARN-8234) Improve RM system metrics publisher's performance by pushing events to timeline server in batch

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8234: - Target Version/s: 3.1.1, 3.1.2 (was: 3.1.1) > Improve RM system metrics publisher's performance by

[jira] [Updated] (YARN-8514) YARN RegistryDNS throws NPE when Kerberos tgt expires

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8514: - Target Version/s: 3.2.0, 3.1.2 (was: 3.2.0, 3.1.1) > YARN RegistryDNS throws NPE when Kerberos tgt

[jira] [Updated] (YARN-8242) YARN NM: OOM error while reading back the state store on recovery

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8242: - Target Version/s: 3.2.0, 3.1.2 (was: 3.2.0) > YARN NM: OOM error while reading back the state store on

[jira] [Commented] (YARN-8242) YARN NM: OOM error while reading back the state store on recovery

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549898#comment-16549898 ] Wangda Tan commented on YARN-8242: -- Bulk update: moved all 3.1.1 non-blocker issues, please move back if

[jira] [Updated] (YARN-8242) YARN NM: OOM error while reading back the state store on recovery

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8242: - Target Version/s: 3.2.0 (was: 3.2.0, 3.1.1) > YARN NM: OOM error while reading back the state store on

[jira] [Updated] (YARN-8015) Support inter-app placement constraints in AppPlacementAllocator

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8015: - Target Version/s: 3.2.0 (was: 3.1.2) > Support inter-app placement constraints in AppPlacementAllocator

[jira] [Commented] (YARN-8015) Support inter-app placement constraints in AppPlacementAllocator

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549897#comment-16549897 ] Wangda Tan commented on YARN-8015: -- Bulk update: moved all 3.1.1 non-blocker issues, please move back if

[jira] [Updated] (YARN-8015) Support inter-app placement constraints in AppPlacementAllocator

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8015: - Target Version/s: 3.1.2 (was: 3.1.1) > Support inter-app placement constraints in AppPlacementAllocator

[jira] [Updated] (YARN-8418) App local logs could leaked if log aggregation fails to initialize for the app

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8418: - Target Version/s: 3.1.2 (was: 3.1.1) > App local logs could leaked if log aggregation fails to

[jira] [Commented] (YARN-8544) [DS] AM registration fails when hadoop authorization is enabled

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549715#comment-16549715 ] Wangda Tan commented on YARN-8544: -- [~bibinchundatt], thanks for working on the issue.  Is this a

[jira] [Commented] (YARN-8541) RM startup failure on recovery after user deletion

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549710#comment-16549710 ] Wangda Tan commented on YARN-8541: -- Thanks [~bibinchundatt] for reporting this issue, so what is the

[jira] [Commented] (YARN-7974) Allow updating application tracking url after registration

2018-07-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549668#comment-16549668 ] Wangda Tan commented on YARN-7974: -- LGTM +1, thanks [~jhung] for the patch. If Jenkins comes with green

[jira] [Updated] (YARN-8545) YARN native service should return container if launch failed

2018-07-17 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8545: - Description: In some cases, container launch may fail but container will not be properly returned to RM. 

[jira] [Created] (YARN-8545) YARN native service should return container if launch failed

2018-07-17 Thread Wangda Tan (JIRA)
Wangda Tan created YARN-8545: Summary: YARN native service should return container if launch failed Key: YARN-8545 URL: https://issues.apache.org/jira/browse/YARN-8545 Project: Hadoop YARN

[jira] [Updated] (YARN-8361) Change App Name Placement Rule to use App Name instead of App Id for configuration

2018-07-17 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8361: - Target Version/s: 3.2.0 (was: 3.2.0, 3.1.1) > Change App Name Placement Rule to use App Name instead of

[jira] [Updated] (YARN-8361) Change App Name Placement Rule to use App Name instead of App Id for configuration

2018-07-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8361: - Fix Version/s: 3.2.0 > Change App Name Placement Rule to use App Name instead of App Id for >

[jira] [Commented] (YARN-8361) Change App Name Placement Rule to use App Name instead of App Id for configuration

2018-07-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545572#comment-16545572 ] Wangda Tan commented on YARN-8361: -- Thanks [~Zian Chen] for the patch and thanks reviews from

[jira] [Commented] (YARN-8524) Single parameter Resource / LightWeightResource constructor looks confusing

2018-07-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16545534#comment-16545534 ] Wangda Tan commented on YARN-8524: -- Thanks [~snemeth] for the patch, LGTM too. Will commit soon. >

[jira] [Commented] (YARN-8361) Change App Name Placement Rule to use App Name instead of App Id for configuration

2018-07-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543711#comment-16543711 ] Wangda Tan commented on YARN-8361: -- LGTM +1, thanks [~Zian Chen] and reviews from [~suma.shivaprasad].

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

2018-07-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543689#comment-16543689 ] Wangda Tan commented on YARN-8513: -- [~cyfdecyf], I couldn't find the error message on the latest

[jira] [Commented] (YARN-8511) When AM releases a container, RM removes allocation tags before it is released by NM

2018-07-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16543683#comment-16543683 ] Wangda Tan commented on YARN-8511: -- Thanks [~cheersyang],  The latest patch LGTM, +1. Will commit today

[jira] [Commented] (YARN-8135) Hadoop {Submarine} Project: Simple and scalable deployment of deep learning training / serving jobs on Hadoop

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542363#comment-16542363 ] Wangda Tan commented on YARN-8135: -- Added Google doc link to Design doc. > Hadoop {Submarine} Project:

[jira] [Comment Edited] (YARN-8330) An extra container got launched by RM for yarn-service

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542327#comment-16542327 ] Wangda Tan edited comment on YARN-8330 at 7/12/18 11:35 PM: Trying to remember

[jira] [Commented] (YARN-8330) An extra container got launched by RM for yarn-service

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542327#comment-16542327 ] Wangda Tan commented on YARN-8330: -- Trying to remember this issue, and post it here before forgot again:

[jira] [Assigned] (YARN-8522) Application fails with InvalidResourceRequestException

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned YARN-8522: Assignee: Zian Chen > Application fails with InvalidResourceRequestException >

[jira] [Commented] (YARN-7556) Fair scheduler configuration should allow resource types in the minResources and maxResources properties

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541999#comment-16541999 ] Wangda Tan commented on YARN-7556: -- Thanks [~snemeth] > Fair scheduler configuration should allow

[jira] [Commented] (YARN-7481) Gpu locality support for Better AI scheduling

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541998#comment-16541998 ] Wangda Tan commented on YARN-7481: -- [~qinc...@microsoft.com], is there any detailed plan of how to better

[jira] [Commented] (YARN-8511) When AM releases a container, RM removes allocation tags before it is released by NM

2018-07-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541973#comment-16541973 ] Wangda Tan commented on YARN-8511: -- Thanks [~cheersyang], for explanation. I completely missed YARN-4148.

[jira] [Commented] (YARN-8480) Add boolean option for resources

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541156#comment-16541156 ] Wangda Tan commented on YARN-8480: -- [~cheersyang],  What [~templedf] / [~snemeth] proposed is make the

[jira] [Commented] (YARN-8505) AMLimit and userAMLimit check should be skipped for unmanaged AM

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541155#comment-16541155 ] Wangda Tan commented on YARN-8505: -- [~bibinchundatt]  / [~Tao Yang] / [~cheersyang],  I would prefer to

[jira] [Commented] (YARN-8511) When AM releases a container, RM removes allocation tags before it is released by NM

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541153#comment-16541153 ] Wangda Tan commented on YARN-8511: -- [~cheersyang],  Thanks for reporting and working on this issue, this

[jira] [Commented] (YARN-8480) Add boolean option for resources

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540897#comment-16540897 ] Wangda Tan commented on YARN-8480: -- btw, [~templedf], I knew you found some troubles of support node

[jira] [Commented] (YARN-8480) Add boolean option for resources

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540895#comment-16540895 ] Wangda Tan commented on YARN-8480: -- [~templedf],  If this only changes Fair Scheduler, I'm fine with

[jira] [Commented] (YARN-7974) Allow updating application tracking url after registration

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540879#comment-16540879 ] Wangda Tan commented on YARN-7974: -- [~oliverhuh...@gmail.com], [~jhung],   Thanks for updating the

[jira] [Updated] (YARN-7481) Gpu locality support for Better AI scheduling

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-7481: - Fix Version/s: (was: 2.7.2) > Gpu locality support for Better AI scheduling >

[jira] [Commented] (YARN-8480) Add boolean option for resources

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540810#comment-16540810 ] Wangda Tan commented on YARN-8480: -- [~snemeth]/[~templedf]. To me we should move this to node

[jira] [Commented] (YARN-7481) Gpu locality support for Better AI scheduling

2018-07-11 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540501#comment-16540501 ] Wangda Tan commented on YARN-7481: -- [~qinc...@microsoft.com], I saw you were keep updating patches in the

[jira] [Commented] (YARN-8512) ATSv2 entities are not published to HBase from second attempt onwards

2018-07-10 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539027#comment-16539027 ] Wangda Tan commented on YARN-8512: -- Patch LGTM as well, thanks [~rohithsharma] for the fix. > ATSv2

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537854#comment-16537854 ] Wangda Tan commented on YARN-8509: -- And downgraded priority to major, removed 3.1.1 from target version.

[jira] [Updated] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8509: - Priority: Major (was: Critical) > Fix UserLimit calculation for preemption to balance scenario after

[jira] [Commented] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537853#comment-16537853 ] Wangda Tan commented on YARN-8509: -- [~Zian Chen], fix version is only set once the patch got committed,

[jira] [Updated] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8509: - Target Version/s: 3.2.0, 3.1.2 (was: 3.2.0, 3.1.1) > Fix UserLimit calculation for preemption to balance

[jira] [Updated] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8509: - Target Version/s: 3.2.0, 3.1.1 > Fix UserLimit calculation for preemption to balance scenario after queue

[jira] [Updated] (YARN-8509) Fix UserLimit calculation for preemption to balance scenario after queue satisfied

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8509: - Fix Version/s: (was: 3.1.1) (was: 3.2.0) > Fix UserLimit calculation for

[jira] [Commented] (YARN-8506) Make GetApplicationsRequestPBImpl thread safe

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537244#comment-16537244 ] Wangda Tan commented on YARN-8506: -- Rebased to latest trunk. (002) > Make GetApplicationsRequestPBImpl

[jira] [Updated] (YARN-8506) Make GetApplicationsRequestPBImpl thread safe

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8506: - Attachment: YARN-8506.002.patch > Make GetApplicationsRequestPBImpl thread safe >

[jira] [Updated] (YARN-8506) Make GetApplicationsRequestPBImpl thread safe

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8506: - Priority: Critical (was: Blocker) > Make GetApplicationsRequestPBImpl thread safe >

[jira] [Updated] (YARN-8506) Make GetApplicationsRequestPBImpl thread safe

2018-07-09 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8506: - Attachment: YARN-8506.001.patch > Make GetApplicationsRequestPBImpl thread safe >

[jira] [Created] (YARN-8506) Make GetApplicationsRequestPBImpl thread safe

2018-07-09 Thread Wangda Tan (JIRA)
Wangda Tan created YARN-8506: Summary: Make GetApplicationsRequestPBImpl thread safe Key: YARN-8506 URL: https://issues.apache.org/jira/browse/YARN-8506 Project: Hadoop YARN Issue Type: Task

[jira] [Commented] (YARN-7556) Fair scheduler configuration should allow resource types in the minResources and maxResources properties

2018-07-07 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535984#comment-16535984 ] Wangda Tan commented on YARN-7556: -- [~snemeth], please go ahead create the JIRA to track the issue.

<    2   3   4   5   6   7   8   9   10   11   >