[jira] [Comment Edited] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310909#comment-16310909 ] lujie edited comment on YARN-7663 at 1/4/18 7:39 AM: - After reading Jason Lowe useful

[jira] [Comment Edited] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310909#comment-16310909 ] lujie edited comment on YARN-7663 at 1/4/18 7:39 AM: - After reading Jason Lowe useful

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_3.patch After reading Jason Lowe useful suggestion. I change the unit test and attach the new

[jira] [Commented] (YARN-7346) Fix compilation errors against hbase2 alpha release

2018-01-03 Thread Vrushali C (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310819#comment-16310819 ] Vrushali C commented on YARN-7346: -- Looks like HBase beta1 RC is out. We should try this with 2.0 beta1

[jira] [Commented] (YARN-6894) RM Apps API returns only active apps when query parameter queue used

2018-01-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310763#comment-16310763 ] Miklos Szegedi commented on YARN-6894: -- Thank you, [~gsohn]! > RM Apps API returns only active apps

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310726#comment-16310726 ] Miklos Szegedi commented on YARN-7516: -- [~eyang], [~ebadger], [~vinodkv], thank you for the patch and

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-7697: -- Priority: Critical (was: Major) Target Version/s: 3.1.0 This will

[jira] [Commented] (YARN-7655) avoid AM preemption caused by RRs for specific nodes or racks

2018-01-03 Thread Steven Rand (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310553#comment-16310553 ] Steven Rand commented on YARN-7655: --- Tagging [~yufeigu] and [~templedf] for thoughts. I can work through

[jira] [Commented] (YARN-7590) Improve container-executor validation check

2018-01-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310534#comment-16310534 ] Miklos Szegedi commented on YARN-7590: -- Thank you for the patch [~eyang]. I have two more style

[jira] [Commented] (YARN-7605) Implement doAs for Api Service REST API

2018-01-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310526#comment-16310526 ] Jian He commented on YARN-7605: --- - the addInternalServlet has an unused parameter clazz > Implement doAs for

[jira] [Commented] (YARN-7605) Implement doAs for Api Service REST API

2018-01-03 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310519#comment-16310519 ] Jian He commented on YARN-7605: --- - we can't assume if result != 0, it is ApplicationNotFoundException ? and

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310492#comment-16310492 ] Hudson commented on YARN-7622: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13446 (See

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310482#comment-16310482 ] Robert Kanter commented on YARN-7622: - Thanks [~gphillips] and [~wilfreds] for review. Committed to

[jira] [Commented] (YARN-7678) Ability to enable logging of container memory stats

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310467#comment-16310467 ] genericqa commented on YARN-7678: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310459#comment-16310459 ] Robert Kanter commented on YARN-7622: - +1 LGTM > Allow fair-scheduler configuration on HDFS >

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310455#comment-16310455 ] Eric Yang commented on YARN-7516: - [~ebadger] The same question has been ask by Vinod on Dec 11. It might

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310440#comment-16310440 ] genericqa commented on YARN-7516: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310426#comment-16310426 ] genericqa commented on YARN-7102: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310417#comment-16310417 ] Jason Lowe commented on YARN-7663: -- Thanks for updating the patch! The unit test passes even without the

[jira] [Commented] (YARN-7693) ContainersMonitor support configurable

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310402#comment-16310402 ] Arun Suresh commented on YARN-7693: --- I agree with [~miklos.szeg...@cloudera.com], there are better ways,

[jira] [Updated] (YARN-7678) Ability to enable logging of container memory stats

2018-01-03 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jim Brennan updated YARN-7678: -- Attachment: YARN-7678-branch-2.001.patch Providing patch for branch-2. Repeated manual tests described

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310385#comment-16310385 ] Eric Yang commented on YARN-7677: - [~ebadger] I think I understand the goal now. Thank you. In current

[jira] [Commented] (YARN-5366) Improve handling of the Docker container life cycle

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310369#comment-16310369 ] Eric Badger commented on YARN-5366: --- Hey [~shaneku...@gmail.com], sorry for the delay. I plan to start

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310354#comment-16310354 ] genericqa commented on YARN-7622: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310344#comment-16310344 ] Eric Badger commented on YARN-7516: --- {quote} + public static final String TRUSTED_DOCKER_REGISTRY =

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310327#comment-16310327 ] Eric Badger commented on YARN-7677: --- bq. Some Hadoop features will not work, i.e. short circuits read, if

[jira] [Comment Edited] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310288#comment-16310288 ] Eric Yang edited comment on YARN-7677 at 1/3/18 9:33 PM: - I think I am stuck on

[jira] [Commented] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

2018-01-03 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310312#comment-16310312 ] Wangda Tan commented on YARN-7666: -- Failed tests look like related, beyond that, patch LGTM. Thanks

[jira] [Updated] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-7516: Attachment: YARN-7516.004.patch [~ebadger] Patch applies. The rebased patch is attached in 004. >

[jira] [Commented] (YARN-7678) Ability to enable logging of container memory stats

2018-01-03 Thread Jim Brennan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310302#comment-16310302 ] Jim Brennan commented on YARN-7678: --- Will do. Thanks! > Ability to enable logging of container memory

[jira] [Updated] (YARN-7678) Ability to enable logging of container memory stats

2018-01-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-7678: - Summary: Ability to enable logging of container memory stats (was: Logging of container memory stats is

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310288#comment-16310288 ] Eric Yang commented on YARN-7677: - I think I am stuck on understanding: {quote} It completely bypasses the

[jira] [Commented] (YARN-7693) ContainersMonitor support configurable

2018-01-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310276#comment-16310276 ] Miklos Szegedi commented on YARN-7693: -- Thank you for the reply [~yangjiandan]. +0 on the approach

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310273#comment-16310273 ] Eric Yang commented on YARN-7677: - [~ebadger] My initial reaction would be to make docker container to

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310268#comment-16310268 ] Eric Badger commented on YARN-7516: --- Hey [~eyang], I tried to apply the patch and it doesn't apply (at

[jira] [Commented] (YARN-6894) RM Apps API returns only active apps when query parameter queue used

2018-01-03 Thread Grant Sohn (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310250#comment-16310250 ] Grant Sohn commented on YARN-6894: -- Thanks [~miklos.szeg...@cloudera.com] for committing this on

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310247#comment-16310247 ] Eric Badger commented on YARN-7677: --- [~eyang], it doesn't necessarily need to be separate hadoop

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310246#comment-16310246 ] genericqa commented on YARN-7697: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Phillips updated YARN-7622: Attachment: (was: YARN-7622.006.patch) > Allow fair-scheduler configuration on HDFS >

[jira] [Updated] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Phillips updated YARN-7622: Attachment: YARN-7622.006.patch > Allow fair-scheduler configuration on HDFS >

[jira] [Updated] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT

2018-01-03 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7102: --- Attachment: YARN-7102.v16.patch > NM heartbeat stuck when responseId overflows MAX_INT >

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310224#comment-16310224 ] genericqa commented on YARN-7622: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310216#comment-16310216 ] Greg Phillips edited comment on YARN-7622 at 1/3/18 8:09 PM: - [~rkanter] - s3 &

[jira] [Commented] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310216#comment-16310216 ] Greg Phillips commented on YARN-7622: - [~rkanter] - s3 & viewfs have been added to allowed filesystems

[jira] [Updated] (YARN-7622) Allow fair-scheduler configuration on HDFS

2018-01-03 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Phillips updated YARN-7622: Attachment: YARN-7622.006.patch > Allow fair-scheduler configuration on HDFS >

[jira] [Commented] (YARN-6971) Clean up different ways to create resources

2018-01-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310210#comment-16310210 ] Yufei Gu commented on YARN-6971: [~lovekesh.bansal], I am not working on this. Feel free to take it. >

[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310176#comment-16310176 ] genericqa commented on YARN-7102: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Attachment: YARN-7697.1.patch > NM goes down with OOM due to leak in log-aggregation >

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310159#comment-16310159 ] Xuan Gong commented on YARN-7697: - Upload a fix for this issue which includes two parts: 1) not use

[jira] [Commented] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310156#comment-16310156 ] Xuan Gong commented on YARN-7697: - The issue happens after file truncate process. Looks like the truncate

[jira] [Updated] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuan Gong updated YARN-7697: Description: 2017-12-29 01:43:50,601 FATAL yarn.YarnUncaughtExceptionHandler

[jira] [Created] (YARN-7697) NM goes down with OOM due to leak in log-aggregation

2018-01-03 Thread Xuan Gong (JIRA)
Xuan Gong created YARN-7697: --- Summary: NM goes down with OOM due to leak in log-aggregation Key: YARN-7697 URL: https://issues.apache.org/jira/browse/YARN-7697 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310135#comment-16310135 ] genericqa commented on YARN-7663: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-7561) Why hasContainerForNode() return false directly when there is no request of ANY locality without considering NODE_LOCAL and RACK_LOCAL?

2018-01-03 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-7561: --- Affects Version/s: 3.1.0 > Why hasContainerForNode() return false directly when there is no request of > ANY

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310093#comment-16310093 ] genericqa commented on YARN-7663: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT

2018-01-03 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310045#comment-16310045 ] Botong Huang commented on YARN-7102: Hi [~jlowe], in v15 patch I moved part of the reconnect logic into

[jira] [Updated] (YARN-7102) NM heartbeat stuck when responseId overflows MAX_INT

2018-01-03 Thread Botong Huang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Botong Huang updated YARN-7102: --- Attachment: YARN-7102.v15.patch > NM heartbeat stuck when responseId overflows MAX_INT >

[jira] [Commented] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310026#comment-16310026 ] genericqa commented on YARN-7666: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310018#comment-16310018 ] lujie commented on YARN-7663: - YARN-7663_2.patch is the patch that contains unit test, i have deleted

[jira] [Updated] (YARN-7696) Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7696: -- Description: The NM needs to persist the Container tags so that on RM recovery, it is sent back to the

[jira] [Commented] (YARN-7516) Security check for untrusted docker image

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310005#comment-16310005 ] Eric Yang commented on YARN-7516: - Ping, can someone review 003 patch? Thanks > Security check for

[jira] [Created] (YARN-7696) Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases

2018-01-03 Thread Arun Suresh (JIRA)
Arun Suresh created YARN-7696: - Summary: Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases Key: YARN-7696 URL:

[jira] [Updated] (YARN-7696) Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7696: -- Environment: (was: The NM needs to persist the Container tags so that on RM recovery, it is sent

[jira] [Commented] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16310003#comment-16310003 ] Hudson commented on YARN-7602: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13443 (See

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309995#comment-16309995 ] Eric Yang commented on YARN-7677: - [~ebadger] I think I understand your scenario better now. When host and

[jira] [Commented] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309964#comment-16309964 ] Haibo Chen commented on YARN-7602: -- Checking this in. > NM should reference the singleton JvmMetrics

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309961#comment-16309961 ] genericqa commented on YARN-7663: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_1.patch > RMAppImpl:Invalid event: START at KILLED >

[jira] [Comment Edited] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309931#comment-16309931 ] lujie edited comment on YARN-7663 at 1/3/18 5:15 PM: - Sorry about delete old files, I

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309931#comment-16309931 ] lujie commented on YARN-7663: - Sorry about delete old files, thanks Jason Lowe's advise. I try add a unit test

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_2.patch > RMAppImpl:Invalid event: START at KILLED >

[jira] [Commented] (YARN-7677) HADOOP_CONF_DIR should not be automatically put in task environment

2018-01-03 Thread Eric Badger (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309906#comment-16309906 ] Eric Badger commented on YARN-7677: --- [~eyang], the docker container file system layout doesn't have to be

[jira] [Commented] (YARN-7590) Improve container-executor validation check

2018-01-03 Thread Eric Yang (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309902#comment-16309902 ] Eric Yang commented on YARN-7590: - Happy New Year [~miklos.szeg...@cloudera.com], Can you review the 006

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309884#comment-16309884 ] Jason Lowe commented on YARN-7663: -- Please do not remove old patches that were commented on. In addition

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: (was: YARN-7663_trunk.patch) > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: (was: YARN-7663_2.8.4.patch) > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

2018-01-03 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-7666: -- Attachment: YARN-7666.004.patch Fixing test case issue. > Introduce scheduler specific environment variable

[jira] [Commented] (YARN-7681) Scheduler should double-check placement constraint before actual allocation is made

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309837#comment-16309837 ] Arun Suresh commented on YARN-7681: --- [~cheersyang], now that YARN-7682 is done, maybe you want to give

[jira] [Updated] (YARN-7682) Expose canSatisfyConstraints utility function to validate a placement against a constraint

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7682: -- Summary: Expose canSatisfyConstraints utility function to validate a placement against a constraint

[jira] [Commented] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309827#comment-16309827 ] genericqa commented on YARN-7602: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7682) Expose canSatisfyConstraints utility function to validate a placement with a constraint

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309821#comment-16309821 ] Arun Suresh commented on YARN-7682: --- +1 for the latest patch. Committing shortly > Expose

[jira] [Updated] (YARN-7682) Expose canSatisfyConstraints utility function to validate a placement with a constraint

2018-01-03 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated YARN-7682: -- Summary: Expose canSatisfyConstraints utility function to validate a placement with a constraint (was:

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_trunk.patch > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: (was: YARN-7663_1.patch) > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_2.8.4.patch > RMAppImpl:Invalid event: START at KILLED >

[jira] [Commented] (YARN-7682) Expose canAssign method in the PlacementConstraintManager

2018-01-03 Thread Panagiotis Garefalakis (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309790#comment-16309790 ] Panagiotis Garefalakis commented on YARN-7682: -- A temporary maven dependency error made

[jira] [Commented] (YARN-7557) It should be possible to specify resource types in the fair scheduler increment value

2018-01-03 Thread Gergo Repas (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309779#comment-16309779 ] Gergo Repas commented on YARN-7557: --- Regarding the deprecation warnings: I do not want to annotate

[jira] [Commented] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309762#comment-16309762 ] genericqa commented on YARN-7663: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309735#comment-16309735 ] genericqa commented on YARN-7666: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7682) Expose canAssign method in the PlacementConstraintManager

2018-01-03 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309729#comment-16309729 ] genericqa commented on YARN-7682: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Assigned] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe reassigned YARN-7663: Assignee: lujie Thanks for the report and the patch! Ignoring the START event seems appropriate

[jira] [Commented] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309692#comment-16309692 ] Haibo Chen commented on YARN-7602: -- v4 of the patch reverts making NodeManagerMetrics final. > NM should

[jira] [Updated] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated YARN-7602: - Attachment: YARN-7602.04.patch > NM should reference the singleton JvmMetrics instance >

[jira] [Commented] (YARN-7602) NM should reference the singleton JvmMetrics instance

2018-01-03 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309688#comment-16309688 ] Haibo Chen commented on YARN-7602: -- Thanks [~mdrob] [~rkanter] and [~rohithsharma] for the reviews! Looks

[jira] [Commented] (YARN-7692) Skip validating priority acls while recovering applications

2018-01-03 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309627#comment-16309627 ] Hudson commented on YARN-7692: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13441 (See

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663_1.patch > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: (was: YARN-7663.patch) > RMAppImpl:Invalid event: START at KILLED >

[jira] [Updated] (YARN-7663) RMAppImpl:Invalid event: START at KILLED

2018-01-03 Thread lujie (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] lujie updated YARN-7663: Attachment: YARN-7663.patch > RMAppImpl:Invalid event: START at KILLED > >

[jira] [Commented] (YARN-7692) Skip validating priority acls while recovering applications

2018-01-03 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309613#comment-16309613 ] Rohith Sharma K S commented on YARN-7692: - committing shortly > Skip validating priority acls

[jira] [Updated] (YARN-7692) Skip validating priority acls while recovering applications

2018-01-03 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-7692: Summary: Skip validating priority acls while recovering applications (was: Resource Manager

  1   2   >