[jira] [Updated] (YARN-8881) [YARN-8851] Add basic pluggable device plugin framework

2018-11-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8881: - Fix Version/s: 3.3.0 > [YARN-8851] Add basic pluggable device plugin framework >

[jira] [Updated] (YARN-8881) [YARN-8851] Add basic pluggable device plugin framework

2018-11-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8881: - Summary: [YARN-8851] Add basic pluggable device plugin framework (was: Phase 1 - Add basic pluggable

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691972#comment-16691972 ] Wangda Tan commented on YARN-8960: -- +1, committing, thanks [~yuan_zac]. > [Submarine] Can't get

[jira] [Updated] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8299: - Target Version/s: (was: 3.1.2) > Yarn Service Upgrade: Add GET APIs that returns instances matching

[jira] [Updated] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8299: - Fix Version/s: 3.1.2 > Yarn Service Upgrade: Add GET APIs that returns instances matching query > params

[jira] [Commented] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689823#comment-16689823 ] Wangda Tan commented on YARN-8299: -- Committing to branch-3.1 now .. > Yarn Service Upgrade: Add GET APIs

[jira] [Updated] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8299: - Priority: Critical (was: Major) > Yarn Service Upgrade: Add GET APIs that returns instances matching

[jira] [Commented] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689805#comment-16689805 ] Wangda Tan commented on YARN-8299: -- Reopened to backport to 3.1.2 > Yarn Service Upgrade: Add GET APIs

[jira] [Updated] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8299: - Target Version/s: 3.1.2 > Yarn Service Upgrade: Add GET APIs that returns instances matching query >

[jira] [Reopened] (YARN-8299) Yarn Service Upgrade: Add GET APIs that returns instances matching query params

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened YARN-8299: -- > Yarn Service Upgrade: Add GET APIs that returns instances matching query > params >

[jira] [Updated] (YARN-8779) Fix few discrepancies between YARN Service swagger spec and code

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8779: - Target Version/s: 3.2.0, 3.1.3 (was: 3.2.0, 3.1.2) > Fix few discrepancies between YARN Service swagger

[jira] [Updated] (YARN-8161) ServiceState FLEX should be removed

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8161: - Target Version/s: 3.2.0, 3.1.3 (was: 3.2.0, 3.1.2) > ServiceState FLEX should be removed >

[jira] [Updated] (YARN-8366) Expose debug log information when user intend to enable GPU without setting nvidia-smi path

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8366: - Target Version/s: 3.2.0, 3.1.3 (was: 3.2.0, 3.1.2) > Expose debug log information when user intend to

[jira] [Updated] (YARN-8986) publish all exposed ports to random ports when using bridge network

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8986: - Fix Version/s: (was: 3.1.2) > publish all exposed ports to random ports when using bridge network >

[jira] [Updated] (YARN-8986) publish all exposed ports to random ports when using bridge network

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8986: - Target Version/s: 3.1.3 (was: 3.1.2) > publish all exposed ports to random ports when using bridge

[jira] [Updated] (YARN-8552) [DS] Container report fails for distributed containers

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8552: - Target Version/s: 3.1.3 (was: 3.1.2) > [DS] Container report fails for distributed containers >

[jira] [Updated] (YARN-8509) Total pending resource calculation in preemption should use user-limit factor instead of minimum-user-limit-percent

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8509: - Target Version/s: 3.2.0, 3.1.3 (was: 3.2.0, 3.1.2) > Total pending resource calculation in preemption

[jira] [Updated] (YARN-8453) Additional Unit tests to verify queue limit and max-limit with multiple resource types

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8453: - Target Version/s: 3.0.4, 3.1.3 (was: 3.0.4, 3.1.2) > Additional Unit tests to verify queue limit and

[jira] [Updated] (YARN-8052) Move overwriting of service definition during flex to service master

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8052: - Target Version/s: 3.1.3 (was: 3.1.2) > Move overwriting of service definition during flex to service

[jira] [Updated] (YARN-8417) Should skip passing HDFS_HOME, HADOOP_CONF_DIR, JAVA_HOME, etc. to Docker container.

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8417: - Target Version/s: 3.1.3 (was: 3.1.2) > Should skip passing HDFS_HOME, HADOOP_CONF_DIR, JAVA_HOME, etc.

[jira] [Updated] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8657: - Target Version/s: 3.2.1, 3.1.3 (was: 3.1.2, 3.2.1) > User limit calculation should be

[jira] [Updated] (YARN-8234) Improve RM system metrics publisher's performance by pushing events to timeline server in batch

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8234: - Target Version/s: 3.1.3 (was: 3.1.2) > Improve RM system metrics publisher's performance by pushing

[jira] [Updated] (YARN-8257) Native service should automatically adding escapes for environment/launch cmd before sending to YARN

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8257: - Target Version/s: 3.1.3 (was: 3.1.2) > Native service should automatically adding escapes for

[jira] [Commented] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689755#comment-16689755 ] Wangda Tan commented on YARN-9030: -- [~suma.shivaprasad], it seems the logic of verifyAndCreateRemoteDir

[jira] [Commented] (YARN-8881) Phase 1 - Add basic pluggable device plugin framework

2018-11-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689699#comment-16689699 ] Wangda Tan commented on YARN-8881: -- +1 to the latest patch, will commit later today if no objections.

[jira] [Updated] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-11-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8917: - Target Version/s: 3.2.1 > Absolute (maximum) capacity of level3+ queues is wrongly calculated for >

[jira] [Updated] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-11-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8917: - Priority: Critical (was: Major) > Absolute (maximum) capacity of level3+ queues is wrongly calculated

[jira] [Resolved] (YARN-9020) set a wrong AbsoluteCapacity when call ParentQueue#setAbsoluteCapacity

2018-11-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-9020. -- Resolution: Duplicate Thanks [~jutia] for reporting this. It is a valid issue. This is dup of

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-11-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686903#comment-16686903 ] Wangda Tan commented on YARN-8917: -- This JIRA somehow dropped from our radar, retriggering Jenkins job

[jira] [Assigned] (YARN-6223) [Umbrella] Natively support GPU configuration/discovery/scheduling/isolation on YARN

2018-11-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reassigned YARN-6223: Assignee: Wangda Tan (was: Antal Bálint Steinbach) > [Umbrella] Natively support GPU

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685750#comment-16685750 ] Wangda Tan commented on YARN-9001: -- Pushed to trunk, but backport to branch-3.2 failed, [~yuan_zac], if

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9001: - Fix Version/s: 3.3.0 > [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs >

[jira] [Comment Edited] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685618#comment-16685618 ] Wangda Tan edited comment on YARN-8960 at 11/13/18 7:50 PM: Thanks

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685635#comment-16685635 ] Wangda Tan commented on YARN-9001: -- Rebased to latest trunk to run Jenkins. > [Submarine] Use

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-9001: - Attachment: YARN-9001.005.patch > [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685631#comment-16685631 ] Wangda Tan commented on YARN-9001: -- Thanks [~yuan_zac], +1, committing the patch. > [Submarine] Use

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685618#comment-16685618 ] Wangda Tan commented on YARN-8960: -- Thanks [~yuan_zac], Some comments: 1) doLoginIfSecure, could u

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8960: - Description: After submitting a submarine job, we tried to get service status using the following

[jira] [Commented] (YARN-8881) Phase 1 - Add basic pluggable device plugin framework

2018-11-13 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685605#comment-16685605 ] Wangda Tan commented on YARN-8881: -- Thanks [~tangzhankun], Regarding Integer vs. int, I would suggest

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-12 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684468#comment-16684468 ] Wangda Tan commented on YARN-9001: -- [~yuan_zac], checked the patch, in general patch looks good, could u

[jira] [Created] (YARN-8993) [Submarine] Add support to run deep learning workload in non-Docker containers

2018-11-08 Thread Wangda Tan (JIRA)
Wangda Tan created YARN-8993: Summary: [Submarine] Add support to run deep learning workload in non-Docker containers Key: YARN-8993 URL: https://issues.apache.org/jira/browse/YARN-8993 Project: Hadoop

[jira] [Commented] (YARN-8877) Extend service spec to allow setting resource attributes

2018-11-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680617#comment-16680617 ] Wangda Tan commented on YARN-8877: -- [~cheersyang], make sense to me. > Extend service spec to allow

[jira] [Commented] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680615#comment-16680615 ] Wangda Tan commented on YARN-8714: -- [~tangzhankun] , I'm still not quite sure about: {code:java}

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680601#comment-16680601 ] Wangda Tan commented on YARN-8960: -- [~yuan_zac] , as we discussed offline, do we still need the service

[jira] [Updated] (YARN-8135) Hadoop {Submarine} Project: Simple and scalable deployment of deep learning training / serving jobs on Hadoop

2018-11-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8135: - Description: Description: *Goals:* - Allow infra engineer / data scientist to run *unmodified*

[jira] [Commented] (YARN-8763) Add WebSocket logic to the Node Manager web server to establish servlet

2018-11-08 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680570#comment-16680570 ] Wangda Tan commented on YARN-8763: -- [~sunilg] , I highly suggest reverting this from branch-3.2 if

[jira] [Updated] (YARN-8220) Running Tensorflow on YARN with GPU and Docker - Examples

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8220: - Description: -Tensorflow could be run on YARN and could leverage YARN's distributed features.- -This

[jira] [Updated] (YARN-8237) mxnet yarn spec file to add to native service examples

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8237: - Description: Mxnet -could be run on YARN. This- jira -will help to add examples,- yarnfile-, docker

[jira] [Updated] (YARN-8238) [Umbrella] YARN deep learning framework examples to run on native service

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8238: - Description: -Umbrella- jira -to track various deep learning frameworks which can run on yarn native

[jira] [Resolved] (YARN-8237) mxnet yarn spec file to add to native service examples

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-8237. -- Resolution: Duplicate > mxnet yarn spec file to add to native service examples >

[jira] [Resolved] (YARN-8238) [Umbrella] YARN deep learning framework examples to run on native service

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-8238. -- Resolution: Fixed Closing as dup of YARN-8135.  > [Umbrella] YARN deep learning framework examples to

[jira] [Commented] (YARN-8877) Extend service spec to allow setting resource attributes

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677039#comment-16677039 ] Wangda Tan commented on YARN-8877: -- [~cheersyang],  If YARN-8940 will satisfy all needs for volume,

[jira] [Commented] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677029#comment-16677029 ] Wangda Tan commented on YARN-8714: -- [~tangzhankun] , could u please explain a little bit about what does

[jira] [Commented] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677030#comment-16677030 ] Wangda Tan commented on YARN-8714: -- + [~liuxun323] / [~yuan_zac] to take a look at this as well. >

[jira] [Commented] (YARN-8902) Add volume manager that manages CSI volume lifecycle

2018-11-06 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677027#comment-16677027 ] Wangda Tan commented on YARN-8902: -- {quote}I prefer not to do this rename. As the package already has 

[jira] [Commented] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676232#comment-16676232 ] Wangda Tan commented on YARN-8858: -- Thanks [~cheersyang] / [~ajisakaa] for rebasing and committing the

[jira] [Commented] (YARN-8867) Retrieve the status of resource localization

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675664#comment-16675664 ] Wangda Tan commented on YARN-8867: -- Thanks [~csingh] for working on this, from high-level I think the

[jira] [Commented] (YARN-8851) [Umbrella] A new pluggable device plugin framework to ease vendor plugin development

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675606#comment-16675606 ] Wangda Tan commented on YARN-8851: -- Thanks [~tangzhankun] ,  1) Regarding to the

[jira] [Commented] (YARN-8858) CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used.

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675572#comment-16675572 ] Wangda Tan commented on YARN-8858: -- Triggered Jenkins build to find flaky tests.  > CapacityScheduler

[jira] [Commented] (YARN-8902) Add volume manager that manages CSI volume lifecycle

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675532#comment-16675532 ] Wangda Tan commented on YARN-8902: -- Thanks [~cheersyang] , a couple of high-level questions and miscs: 

[jira] [Commented] (YARN-8877) Extend service spec to allow setting resource attributes

2018-11-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675496#comment-16675496 ] Wangda Tan commented on YARN-8877: -- Thanks [~cheersyang],  In general patch looks good, could u update

[jira] [Commented] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-10-30 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669392#comment-16669392 ] Wangda Tan commented on YARN-8714: -- [~tangzhankun] , sure, please go ahead. > [Submarine] Support

[jira] [Commented] (YARN-8944) TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896

2018-10-26 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665445#comment-16665445 ] Wangda Tan commented on YARN-8944: -- Thanks [~wilfreds] , Patch LGTM, will commit today. >

[jira] [Commented] (YARN-8866) Fix a parsing error for crossdomain.xml

2018-10-26 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16665444#comment-16665444 ] Wangda Tan commented on YARN-8866: -- Thanks [~tasanuma0829],  Patch LGTM, will commit today. > Fix a

[jira] [Resolved] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

2018-10-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-8513. -- Resolution: Duplicate Fix Version/s: (was: 3.2.1) (was: 3.1.2) Reopen

[jira] [Reopened] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

2018-10-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan reopened YARN-8513: -- > CapacityScheduler infinite loop when queue is near fully utilized >

[jira] [Commented] (YARN-8895) Improve YARN Error diagnostics

2018-10-24 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662914#comment-16662914 ] Wangda Tan commented on YARN-8895: -- [~youchen] , Thanks, I believe this will be a very useful Jira. My

[jira] [Commented] (YARN-8927) Better handling of "docker.trusted.registries" in container-executor's "trusted_image_check" function

2018-10-23 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660904#comment-16660904 ] Wangda Tan commented on YARN-8927: -- [~tangzhankun], thanks for filing the Jira, I encountered the issue

[jira] [Commented] (YARN-8851) [Umbrella] A new pluggable device plugin framework to ease vendor plugin development

2018-10-22 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659747#comment-16659747 ] Wangda Tan commented on YARN-8851: -- [~tangzhankun], Thanks for updating the patch, the latest patch

[jira] [Commented] (YARN-8918) [Submarine] Correct method usage of str.subString in CliUtils

2018-10-22 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659725#comment-16659725 ] Wangda Tan commented on YARN-8918: -- +1, thanks [~tangzhankun]. > [Submarine] Correct method usage of

[jira] [Commented] (YARN-8924) Refine the document or code related to legacy CPU isolation/enforcement

2018-10-22 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659723#comment-16659723 ] Wangda Tan commented on YARN-8924: -- Thanks [~tangzhankun] for filing and Jira and put analysis.

[jira] [Commented] (YARN-6167) RM option to delegate NM loss container action to AM

2018-10-22 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16659719#comment-16659719 ] Wangda Tan commented on YARN-6167: -- Thanks [~billie.rinaldi], 1) Inside releaseContainers, why add

[jira] [Commented] (YARN-8920) LogAggregation should be configurable to allow writing to underlying storage as appOwner or yarn user

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657413#comment-16657413 ] Wangda Tan commented on YARN-8920: -- Thanks [~suma.shivaprasad], 1) Inside YarnConfiguration, We should

[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657113#comment-16657113 ] Wangda Tan commented on YARN-8917: -- Nice catch [~Tao Yang]! Fix makes sense to me. [~sunilg], this

[jira] [Updated] (YARN-8916) Define a constant "docker" string in "ContainerRuntimeConstants.java" for better maintainability

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8916: - Fix Version/s: 3.3.0 > Define a constant "docker" string in "ContainerRuntimeConstants.java" for >

[jira] [Updated] (YARN-8918) [Submarine] Remove redundant method of str.subString(0, str.length())

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8918: - Fix Version/s: (was: 3.2.1) (was: 3.3.0) (was: 3.1.2) >

[jira] [Updated] (YARN-8918) [Submarine] Remove redundant method of str.subString(0, str.length())

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8918: - Fix Version/s: 3.2.1 3.3.0 3.1.2 > [Submarine] Remove redundant

[jira] [Updated] (YARN-8908) Fix errors in yarn-default.xml related to GPU/FPGA

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8908: - Fix Version/s: 3.2.1 3.3.0 3.1.2 > Fix errors in yarn-default.xml

[jira] [Updated] (YARN-8916) Define a constant "docker" string in "ContainerRuntimeConstants.java" for better maintainability

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8916: - Fix Version/s: (was: 3.2.0) 3.2.1 > Define a constant "docker" string in

[jira] [Commented] (YARN-8916) Define a constant "docker" string in "ContainerRuntimeConstants.java" for better maintainability

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657057#comment-16657057 ] Wangda Tan commented on YARN-8916: -- +1, thanks [~tangzhankun]. > Define a constant "docker" string in

[jira] [Commented] (YARN-8908) Fix errors in yarn-default.xml related to GPU/FPGA

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657005#comment-16657005 ] Wangda Tan commented on YARN-8908: -- +1, thanks [~tangzhankun].  > Fix errors in yarn-default.xml related

[jira] [Commented] (YARN-8918) [Submarine] Remove redundant method of str.subString(0, str.length())

2018-10-19 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657004#comment-16657004 ] Wangda Tan commented on YARN-8918: -- [~tangzhankun],  I think the correct logic should be:  {code:java}

[jira] [Commented] (YARN-6098) Add documentation for Delete Queue

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655793#comment-16655793 ] Wangda Tan commented on YARN-6098: -- Backported to branch-3.1 as well. > Add documentation for Delete

[jira] [Updated] (YARN-6098) Add documentation for Delete Queue

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6098: - Fix Version/s: 3.1.2 > Add documentation for Delete Queue > -- > >

[jira] [Commented] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655790#comment-16655790 ] Wangda Tan commented on YARN-8896: -- Committed to trunk/branch-3.1/branch-3.2. > Limit the maximum number

[jira] [Updated] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8896: - Fix Version/s: 3.2.1 3.1.2 > Limit the maximum number of container assignments per

[jira] [Resolved] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan resolved YARN-8896. -- Resolution: Fixed > Limit the maximum number of container assignments per heartbeat >

[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655670#comment-16655670 ] Wangda Tan commented on YARN-8489: -- [~billie.rinaldi]/[~eyang],  Suggestions make sense to me. I will +1

[jira] [Commented] (YARN-8456) Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655664#comment-16655664 ] Wangda Tan commented on YARN-8456: -- +1, thanks [~tangzhankun].  > Fix a configuration handling bug when

[jira] [Updated] (YARN-8870) [Submarine] Add submarine installation scripts

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8870: - Fix Version/s: 3.2.0 > [Submarine] Add submarine installation scripts >

[jira] [Commented] (YARN-8896) Limit the maximum number of container assignments per heartbeat

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655577#comment-16655577 ] Wangda Tan commented on YARN-8896: -- +1, patch LGTM, thanks [~tangzhankun].  > Limit the maximum number

[jira] [Commented] (YARN-8908) Fix errors in yarn-default.xml related to GPU/FPGA

2018-10-18 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655575#comment-16655575 ] Wangda Tan commented on YARN-8908: -- +1, patch LGMT.  Thanks [~tangzhankun]. > Fix errors in

[jira] [Commented] (YARN-8851) [Umbrella] A new pluggable device plugin framework to ease vendor plugin development

2018-10-17 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654192#comment-16654192 ] Wangda Tan commented on YARN-8851: -- Thanks [~tangzhankun],  mostly high level comments.  item #6 will be

[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652717#comment-16652717 ] Wangda Tan commented on YARN-8489: -- [~eyang],  {quote} A safer approach to enable this logic is to have a

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652645#comment-16652645 ] Wangda Tan commented on YARN-8513: -- Sounds like a plan, default value set to 100 may make more sense.

[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652606#comment-16652606 ] Wangda Tan commented on YARN-8489: -- [~eyang], let me try to answer your questions:  {quote}Data

[jira] [Updated] (YARN-8892) YARN UI2 doc changes to update security status (verified under security environment)

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8892: - Target Version/s: 3.2.0, 3.1.2 (was: 3.2.0) > YARN UI2 doc changes to update security status (verified

[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652453#comment-16652453 ] Wangda Tan commented on YARN-8489: -- [~eyang],  This is bit different from Spark executors.  For Spark,

[jira] [Updated] (YARN-8892) YARN UI2 doc changes to update security status (verified under security environment)

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8892: - Summary: YARN UI2 doc changes to update security status (verified under security environment) (was: YARN

[jira] [Commented] (YARN-8892) YARN UI2 doc improvement to update security status

2018-10-16 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652405#comment-16652405 ] Wangda Tan commented on YARN-8892: -- +1, committing, thanks [~sunilg]. > YARN UI2 doc improvement to

<    1   2   3   4   5   6   7   8   9   10   >