[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113592#comment-16113592 ] Miklos Szegedi commented on YARN-6033: -- I am not a committer, so you do not need my approval. I just

[jira] [Comment Edited] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113478#comment-16113478 ] Miklos Szegedi edited comment on YARN-6033 at 8/3/17 9:19 PM: -- Thank you,

[jira] [Created] (YARN-6943) Update Yarn to YARN in documentation

2017-08-03 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6943: Summary: Update Yarn to YARN in documentation Key: YARN-6943 URL: https://issues.apache.org/jira/browse/YARN-6943 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113506#comment-16113506 ] Miklos Szegedi commented on YARN-6757: -- I attached the patch. > Refactor the usage of >

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Attachment: YARN-6757.005.patch > Refactor the usage of >

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113478#comment-16113478 ] Miklos Szegedi commented on YARN-6033: -- Thank you, [~vvasudev] and [~wangda]. I looked through the

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113376#comment-16113376 ] Miklos Szegedi commented on YARN-6757: -- Hmm, I do not understand. If YARN is the official naming

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.002.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113338#comment-16113338 ] Miklos Szegedi commented on YARN-6668: -- bq. 4) Is the process jiff too large to store in a long

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113337#comment-16113337 ] Miklos Szegedi commented on YARN-6668: -- Thanks for the review [~haibo.chen]. 3)

[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113231#comment-16113231 ] Miklos Szegedi commented on YARN-5534: -- [~ebadger], only the ones that need root access. > Allow

[jira] [Comment Edited] (YARN-5534) Allow whitelisted volume mounts

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113114#comment-16113114 ] Miklos Szegedi edited comment on YARN-5534 at 8/3/17 5:18 PM: -- Thank you,

[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113114#comment-16113114 ] Miklos Szegedi commented on YARN-5534: -- Thank you, [~shaneku...@gmail.com] and [~vinodkv] for the

[jira] [Comment Edited] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113095#comment-16113095 ] Miklos Szegedi edited comment on YARN-6930 at 8/3/17 5:06 PM: -- The disable

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113095#comment-16113095 ] Miklos Szegedi commented on YARN-6930: -- The disable config is actually smart enough to set the mode in

[jira] [Comment Edited] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112193#comment-16112193 ] Miklos Szegedi edited comment on YARN-6852 at 8/3/17 5:05 AM: -- … {{Return a

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112193#comment-16112193 ] Miklos Szegedi commented on YARN-6852: -- … {{Return a parsed commandline options.}} There is a typo in

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112127#comment-16112127 ] Miklos Szegedi commented on YARN-6852: -- … Never mind about the function header comments above. I see

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111936#comment-16111936 ] Miklos Szegedi commented on YARN-6852: -- Thank you for the patch [~wangda] We have now get_executable.c

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111913#comment-16111913 ] Miklos Szegedi commented on YARN-6033: -- I found one more thing: {code} struct configuration CFG =

[jira] [Commented] (YARN-6674) Add memory cgroup settings for opportunistic containers

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111864#comment-16111864 ] Miklos Szegedi commented on YARN-6674: -- As I expressed there, this is intentional. Any unintentional

[jira] [Commented] (YARN-6673) Add cpu cgroup configurations for opportunistic containers

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111848#comment-16111848 ] Miklos Szegedi commented on YARN-6673: -- [~haibochen], I intentionally put the hardcoded value there.

[jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111776#comment-16111776 ] Miklos Szegedi commented on YARN-6623: -- I updated this to blocker, because I think 3.0 should not ship

[jira] [Updated] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6623: - Priority: Blocker (was: Major) > Add support to turn off launching privileged containers in the

[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111750#comment-16111750 ] Miklos Szegedi commented on YARN-5534: -- bq. The config should be there in both the places -

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111677#comment-16111677 ] Miklos Szegedi commented on YARN-6930: -- JavaSandboxLinuxContainerRuntime already works like this. See

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111531#comment-16111531 ] Miklos Szegedi commented on YARN-6033: -- Thanks, [~vvasudev]. bq. I would prefer not to do this, we

[jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111499#comment-16111499 ] Miklos Szegedi commented on YARN-6623: -- [~dan...@cloudera.com], this is needed I think for defense in

[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111465#comment-16111465 ] Miklos Szegedi commented on YARN-5534: -- [~shaneku...@gmail.com], container-executor.cfg is only

[jira] [Updated] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-02 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6895: - Attachment: YARN-6895.branch-2.000.patch Thank you. I attached the branch-2 patch >

[jira] [Commented] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109986#comment-16109986 ] Miklos Szegedi commented on YARN-6895: -- I opened YARN-6925 and YARN-6926. Thank you! >

[jira] [Created] (YARN-6926) FSSchedulerNode reservation conflict

2017-08-01 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6926: Summary: FSSchedulerNode reservation conflict Key: YARN-6926 URL: https://issues.apache.org/jira/browse/YARN-6926 Project: Hadoop YARN Issue Type:

[jira] [Created] (YARN-6925) FSSchedulerNode could be simplified extracting preemption fields into a class

2017-08-01 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6925: Summary: FSSchedulerNode could be simplified extracting preemption fields into a class Key: YARN-6925 URL: https://issues.apache.org/jira/browse/YARN-6925 Project:

[jira] [Commented] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109978#comment-16109978 ] Miklos Szegedi commented on YARN-6895: -- The failing unit tests are not related to the change. >

[jira] [Commented] (YARN-5534) Allow whitelisted volume mounts

2017-08-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109862#comment-16109862 ] Miklos Szegedi commented on YARN-5534: -- Thank you for the patch [~shaneku...@gmail.com]. Quick

[jira] [Updated] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6895: - Attachment: YARN-6895.001.patch > [FairScheduler] Preemption reservation may cause regular

[jira] [Commented] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109701#comment-16109701 ] Miklos Szegedi commented on YARN-6895: -- Thank you, [~yufeigu]. We will do normal reservation, if there

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-07-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108256#comment-16108256 ] Miklos Szegedi commented on YARN-6757: -- I changed only documentation in the recent patch, so the unit

[jira] [Created] (YARN-6913) Some cgroup settings are not documented in yarn-default.xml

2017-07-31 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6913: Summary: Some cgroup settings are not documented in yarn-default.xml Key: YARN-6913 URL: https://issues.apache.org/jira/browse/YARN-6913 Project: Hadoop YARN

[jira] [Updated] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-07-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6895: - Summary: [FairScheduler] Preemption reservation may cause regular reservation leaks (was:

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-07-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Attachment: YARN-6757.004.patch Thank you. I attached the update. > Refactor the usage of >

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-07-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107847#comment-16107847 ] Miklos Szegedi commented on YARN-6033: -- Thank you, [~vvasudev] for the patch. I have some

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-07-28 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Attachment: YARN-6757.003.patch > Refactor the usage of >

[jira] [Commented] (YARN-6895) Preemption reservation may cause regular reservation leaks

2017-07-28 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105632#comment-16105632 ] Miklos Szegedi commented on YARN-6895: -- I verified and TestSubmitApplicationWithRMHA fails without the

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-07-28 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105581#comment-16105581 ] Miklos Szegedi commented on YARN-6757: -- Thanks. I updated the documentation including the html. I

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-07-28 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Attachment: YARN-6757.002.patch > Refactor the usage of >

[jira] [Updated] (YARN-6895) Preemption reservation may cause regular reservation leaks

2017-07-28 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6895: - Attachment: YARN-6895.000.patch > Preemption reservation may cause regular reservation leaks >

[jira] [Created] (YARN-6895) Preemption reservation may cause regular reservation leaks

2017-07-27 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6895: Summary: Preemption reservation may cause regular reservation leaks Key: YARN-6895 URL: https://issues.apache.org/jira/browse/YARN-6895 Project: Hadoop YARN

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-06-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070635#comment-16070635 ] Miklos Szegedi commented on YARN-6757: -- See also the discussion in YARN-6515, why we need this. >

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-06-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Attachment: YARN-6757.000.patch > Refactor the usage of >

[jira] [Updated] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-06-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6757: - Summary: Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path (was:

[jira] [Created] (YARN-6757) Refactor the setting yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-06-30 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6757: Summary: Refactor the setting yarn.nodemanager.linux-container-executor.cgroups.mount-path Key: YARN-6757 URL: https://issues.apache.org/jira/browse/YARN-6757

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-06-23 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.001.patch > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-6674) Add memory cgroup settings for opportunistic containers

2017-06-23 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6674: - Attachment: YARN-6674.000.patch > Add memory cgroup settings for opportunistic containers >

[jira] [Updated] (YARN-6673) Add cpu cgroup configurations for opportunistic containers

2017-06-23 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6673: - Attachment: YARN-6673.000.patch > Add cpu cgroup configurations for opportunistic containers >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-06-23 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.000.patch > Use cgroup to get container resource utilization >

[jira] [Created] (YARN-6732) Could not find artifact org.apache.hadoop:hadoop-azure-datalake:jar

2017-06-22 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6732: Summary: Could not find artifact org.apache.hadoop:hadoop-azure-datalake:jar Key: YARN-6732 URL: https://issues.apache.org/jira/browse/YARN-6732 Project: Hadoop YARN

[jira] [Commented] (YARN-6723) NM overallocation based on over-time rather than snapshot utilization

2017-06-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16054885#comment-16054885 ] Miklos Szegedi commented on YARN-6723: -- The suggestion is to add a period counter for

[jira] [Updated] (YARN-6673) Add cpu cgroup configurations for opportunistic containers

2017-06-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6673: - Description: In addition to setting cpu.cfs_period_us on a per-container basis, we could also set

[jira] [Commented] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16046813#comment-16046813 ] Miklos Szegedi commented on YARN-6670: -- It applies to both CPU and memory. Let's say we have a

[jira] [Commented] (YARN-6670) Add separate NM overallocation thresholds for cpu and memory

2017-06-09 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045359#comment-16045359 ] Miklos Szegedi commented on YARN-6670: -- Thanks for the patch [~haibochen]. Quick question. Usually the

[jira] [Assigned] (YARN-5600) Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-application basis

2017-06-09 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi reassigned YARN-5600: Assignee: (was: Miklos Szegedi) > Allow setting yarn.nodemanager.delete.debug-delay-sec

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-06-06 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040014#comment-16040014 ] Miklos Szegedi commented on YARN-6668: -- I attached a prototype to start the discussion. We have an

[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-05-09 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003194#comment-16003194 ] Miklos Szegedi commented on YARN-5301: -- Thank you for the review [~templedf]. I think, there is no

[jira] [Commented] (YARN-6562) Unable to delete cgroup to /xxx, system cpu utility continuously rise

2017-05-05 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999035#comment-15999035 ] Miklos Szegedi commented on YARN-6562: -- Thank you, [~yangjiandan] for reporting this. Which process

[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager

2017-05-05 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999026#comment-15999026 ] Miklos Szegedi commented on YARN-6515: -- Thank you for the response [~shaneku...@gmail.com]. In this

[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues

2017-05-05 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15998949#comment-15998949 ] Miklos Szegedi commented on YARN-6475: -- +1 LGTM (non-binding) Thank you for the patch [~soumabrata]!

[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager

2017-05-04 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15997197#comment-15997197 ] Miklos Szegedi commented on YARN-6515: -- Thank you, [~shaneku...@gmail.com] for the explanation. bq.

[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager

2017-05-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15991161#comment-15991161 ] Miklos Szegedi commented on YARN-6515: -- Thank you, [~ajisakaa] for raising this issue.

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-27 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.010.patch > NM mount cpu cgroups failed on some systems >

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-26 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.009.patch Fixed checkstyle, changed all cgroup List references to Set > NM

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-26 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.008.patch > NM mount cpu cgroups failed on some systems >

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-26 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.007.patch Thank you for the review [~templedf]. I updated the patch. > NM

[jira] [Created] (YARN-6525) Linux container executor should not propagate application errors

2017-04-25 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6525: Summary: Linux container executor should not propagate application errors Key: YARN-6525 URL: https://issues.apache.org/jira/browse/YARN-6525 Project: Hadoop YARN

[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues

2017-04-24 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981547#comment-15981547 ] Miklos Szegedi commented on YARN-6475: -- Thank you, [~soumabrata]. I think you can address them as this

[jira] [Commented] (YARN-6417) TestContainerManager.testContainerLaunchAndStop disabled

2017-04-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979452#comment-15979452 ] Miklos Szegedi commented on YARN-6417: -- You can build with native profile and run the test specifying

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-04-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979067#comment-15979067 ] Miklos Szegedi commented on YARN-6384: -- Sure. > Add configuratin to set max cpu usage when

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-20 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6500: - Attachment: YARN-6500.003.patch > Do not mount inaccessible cgroups directories in

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-20 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6500: - Attachment: YARN-6500.002.patch Thank you, [~templedf], [~haibochen], [~rkanter] for the review. I

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-20 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6500: - Attachment: YARN-6500.001.patch Fixing checkstyle > Do not mount inaccessible cgroups directories

[jira] [Commented] (YARN-6432) FairScheduler: Reserve preempted resources for corresponding applications

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976044#comment-15976044 ] Miklos Szegedi commented on YARN-6432: -- The original patch does not map null, however this might be

[jira] [Updated] (YARN-6432) FairScheduler: Reserve preempted resources for corresponding applications

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6432: - Attachment: YARN-6432.branch-2.addendum.patch Thank you, [~jzhuge] for the report. I attached an

[jira] [Updated] (YARN-6302) Fail the node if Linux Container Executor is not configured properly

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6302: - Attachment: YARN-6302.005.branch-2.patch Thank you, [~templedf]. I attached the branch-2 patch. >

[jira] [Updated] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6500: - Attachment: YARN-6500.000.patch > Do not mount inaccessible cgroups directories in

[jira] [Created] (YARN-6500) Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler

2017-04-19 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6500: Summary: Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler Key: YARN-6500 URL: https://issues.apache.org/jira/browse/YARN-6500 Project:

[jira] [Commented] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15975742#comment-15975742 ] Miklos Szegedi commented on YARN-5301: -- Yes, it is safe in this code. cgroups is available only on

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.006.patch Attaching a patch to fix unit test regression on Mac > NM mount

[jira] [Updated] (YARN-5301) NM mount cpu cgroups failed on some systems

2017-04-19 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5301: - Attachment: YARN-5301.005.patch I rebased the patch. > NM mount cpu cgroups failed on some

[jira] [Commented] (YARN-5894) license warning in de.ruedigermoeller:fst:jar:2.24

2017-04-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971617#comment-15971617 ] Miklos Szegedi commented on YARN-5894: -- I checked and 2.41 and above versions of the artifact

[jira] [Updated] (YARN-5894) license warning in de.ruedigermoeller:fst:jar:2.24

2017-04-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-5894: - Priority: Blocker (was: Major) > license warning in de.ruedigermoeller:fst:jar:2.24 >

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-04-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971378#comment-15971378 ] Miklos Szegedi commented on YARN-6384: -- Thank you, [~lolee_k]. How about others? If there are more

[jira] [Commented] (YARN-6472) Possible Java sandbox improvements

2017-04-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15971362#comment-15971362 ] Miklos Szegedi commented on YARN-6472: -- Thank you, [~gphillips]. +1 (non-binding) for the patch >

[jira] [Updated] (YARN-6438) Code can be improved in ContainersMonitorImpl.java

2017-04-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6438: - Attachment: YARN-6438.003.patch > Code can be improved in ContainersMonitorImpl.java >

[jira] [Commented] (YARN-6438) Code can be improved in ContainersMonitorImpl.java

2017-04-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969569#comment-15969569 ] Miklos Szegedi commented on YARN-6438: -- Good point [~templedf]. I meant to keep both, because the

[jira] [Updated] (YARN-6432) FairScheduler: Reserve preempted resources for corresponding applications

2017-04-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6432: - Attachment: YARN-6432.branch-2.patch Thank you for checking this in. Attached the branch-2 patch.

[jira] [Commented] (YARN-6472) Possible Java sandbox improvements

2017-04-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15969372#comment-15969372 ] Miklos Szegedi commented on YARN-6472: -- Thank you, [~gphillips]. Is there any reason why

[jira] [Updated] (YARN-6433) Only accessible cgroup mount directories should be selected for a controller

2017-04-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6433: - Attachment: YARN-6433.002.patch > Only accessible cgroup mount directories should be selected for

[jira] [Commented] (YARN-6423) Queue metrics doesn't work for Fair Scheduler in SLS

2017-04-13 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967843#comment-15967843 ] Miklos Szegedi commented on YARN-6423: -- +1 (non-binding) Thank you, [~yufeigu]. > Queue metrics

[jira] [Comment Edited] (YARN-6472) Possible Java sandbox improvements

2017-04-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966538#comment-15966538 ] Miklos Szegedi edited comment on YARN-6472 at 4/12/17 8:20 PM: --- Thank you,

[jira] [Commented] (YARN-6472) Possible Java sandbox improvements

2017-04-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966538#comment-15966538 ] Miklos Szegedi commented on YARN-6472: -- Thank you, [~gphillips] for the patch. I think we need to

<    2   3   4   5   6   7   8   9   10   11   >