[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167172#comment-16167172 ] Miklos Szegedi commented on YARN-7192: -- Thank you, [~asuresh] for the patch. I have a couple of

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167130#comment-16167130 ] Miklos Szegedi commented on YARN-7034: -- I could not reproduce the unit test error locally. >

[jira] [Commented] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167128#comment-16167128 ] Miklos Szegedi commented on YARN-7009: -- Thank you, [~asuresh] for the feedback. Yes indeed there is no

[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167124#comment-16167124 ] Miklos Szegedi commented on YARN-7192: -- Thank you, [~asuresh] and [~roniburd] for the replies. I

[jira] [Commented] (YARN-7192) Add a pluggable StateMachine Listener that is notified of NM Container State changes

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166969#comment-16166969 ] Miklos Szegedi commented on YARN-7192: -- [~asuresh], thank you, I think this is very useful. I worked

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.005.patch YARN-7034.branch-2.005.patch

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.004.patch YARN-7034.branch-2.004.patch

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166791#comment-16166791 ] Miklos Szegedi commented on YARN-7034: -- Thank you, [~shaneku...@gmail.com] for the feedback. I will

[jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-09-13 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165527#comment-16165527 ] Miklos Szegedi commented on YARN-6623: -- Thank you for the patch [~vvasudev] and for the reviews

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-09-13 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165235#comment-16165235 ] Miklos Szegedi commented on YARN-6384: -- [~lolee_k], CGROUP_CPU_PERIOD_US should never be set in

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163934#comment-16163934 ] Miklos Szegedi commented on YARN-7034: -- The branch-2 patches are failing due to YARN-7187. >

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163932#comment-16163932 ] Miklos Szegedi commented on YARN-7034: -- The unit tests should be unrelated. The previous test without

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Priority: Blocker (was: Critical) > DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Priority: Critical (was: Blocker) > DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.003.patch Fixing checkstyle. > DefaultLinuxContainerRuntime and

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.002.patch Rebasing the trunk patch. > DefaultLinuxContainerRuntime and

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163766#comment-16163766 ] Miklos Szegedi commented on YARN-7034: -- [~vinodkv], I think we should apply this jira without any

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-12 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.branch-2.000.patch YARN-7034.branch-2.8.000.patch Adding

[jira] [Updated] (YARN-7181) CPUTimeTracker.updateElapsedJiffies can report negative usage

2017-09-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7181: - Attachment: YARN-7181.000.patch > CPUTimeTracker.updateElapsedJiffies can report negative usage >

[jira] [Created] (YARN-7181) CPUTimeTracker.updateElapsedJiffies can report negative usage

2017-09-08 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7181: Summary: CPUTimeTracker.updateElapsedJiffies can report negative usage Key: YARN-7181 URL: https://issues.apache.org/jira/browse/YARN-7181 Project: Hadoop YARN

[jira] [Updated] (YARN-7064) Use cgroup to get container resource utilization

2017-09-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7064: - Attachment: YARN-7064.004.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues

2017-09-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157267#comment-16157267 ] Miklos Szegedi commented on YARN-6475: -- Patches for the next release of Hadoop 2.x. > Fix some long

[jira] [Updated] (YARN-7064) Use cgroup to get container resource utilization

2017-09-06 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7064: - Attachment: YARN-7064.003.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-7064) Use cgroup to get container resource utilization

2017-09-06 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156154#comment-16156154 ] Miklos Szegedi commented on YARN-7064: -- Fixed the flakiness and the constructor now does not throw an

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-09-05 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154648#comment-16154648 ] Miklos Szegedi commented on YARN-7034: -- Thank you, [~rkanter]. My concern is that I see little

[jira] [Commented] (YARN-7145) Identify potential flaky unit tests

2017-09-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151263#comment-16151263 ] Miklos Szegedi commented on YARN-7145: -- These are the tests that are likely to be flaky in case of a

[jira] [Updated] (YARN-7145) Identify potential flaky unit tests

2017-09-01 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7145: - Attachment: YARN-7145.001.patch [~bibinchundatt], good point, thank you. I attached a new patch.

[jira] [Updated] (YARN-7145) Identify potential flaky unit tests

2017-08-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7145: - Attachment: YARN-7145.000.patch > Identify potential flaky unit tests >

[jira] [Created] (YARN-7145) Identify potential flaky unit tests

2017-08-31 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7145: Summary: Identify potential flaky unit tests Key: YARN-7145 URL: https://issues.apache.org/jira/browse/YARN-7145 Project: Hadoop YARN Issue Type: Test

[jira] [Updated] (YARN-7064) Use cgroup to get container resource utilization

2017-08-31 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7064: - Attachment: YARN-7064.002.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-6475) Fix some long function checkstyle issues

2017-08-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148060#comment-16148060 ] Miklos Szegedi commented on YARN-6475: -- [~Soumabrata Chakraborty], could you provide the patch since

[jira] [Comment Edited] (YARN-6475) Fix some long function checkstyle issues

2017-08-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148060#comment-16148060 ] Miklos Szegedi edited comment on YARN-6475 at 8/30/17 9:04 PM: ---

[jira] [Assigned] (YARN-6997) org.apache.hadoop.yarn.client.SCMAdmin wrong package name

2017-08-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi reassigned YARN-6997: Assignee: HondaWei > org.apache.hadoop.yarn.client.SCMAdmin wrong package name >

[jira] [Commented] (YARN-6997) org.apache.hadoop.yarn.client.SCMAdmin wrong package name

2017-08-30 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148016#comment-16148016 ] Miklos Szegedi commented on YARN-6997: -- Yes, please take it. > org.apache.hadoop.yarn.client.SCMAdmin

[jira] [Commented] (YARN-7099) ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users.

2017-08-25 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16142600#comment-16142600 ] Miklos Szegedi commented on YARN-7099: -- I removed half a line, so it would be complicated to add a

[jira] [Updated] (YARN-7099) ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users.

2017-08-24 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7099: - Attachment: YARN-7099.000.patch > ResourceHandlerModule.parseConfiguredCGroupPath only works for

[jira] [Updated] (YARN-7099) ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users.

2017-08-24 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7099: - Description: canWrite is failing below: {code} if (candidate.isDirectory() &&

[jira] [Created] (YARN-7099) ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users.

2017-08-24 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7099: Summary: ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. Key: YARN-7099 URL: https://issues.apache.org/jira/browse/YARN-7099

[jira] [Commented] (YARN-7066) Add ability to specify volumes to mount for DockerContainerRuntime

2017-08-23 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139138#comment-16139138 ] Miklos Szegedi commented on YARN-7066: -- [~eyang], is not this a security issue? What protects against

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-08-22 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137069#comment-16137069 ] Miklos Szegedi commented on YARN-6384: -- Thank you, [~lolee_k], {code} 212 if

[jira] [Updated] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7009: - Attachment: YARN-7009.002.patch > TestNMClient.testNMClientNoCleanupOnStop is flaky by design >

[jira] [Commented] (YARN-7066) Add ability to specify volumes to mount for DockerContainerExecutor

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136018#comment-16136018 ] Miklos Szegedi commented on YARN-7066: -- [~eyang], do you mean DockerContainerRuntime?

[jira] [Updated] (YARN-7064) Use cgroup to get container resource utilization

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7064: - Attachment: YARN-7064.001.patch > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7009: - Attachment: YARN-7009.001.patch Fixing checkstyle and findbugs. >

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135842#comment-16135842 ] Miklos Szegedi commented on YARN-6668: -- Opened YARN-7064 to continue with this patch. > Use cgroup to

[jira] [Updated] (YARN-7064) Use cgroup to get container resource utilization

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7064: - Attachment: YARN-7064.000.patch [~haibochen], please review this patch. > Use cgroup to get

[jira] [Created] (YARN-7064) Use cgroup to get container resource utilization

2017-08-21 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7064: Summary: Use cgroup to get container resource utilization Key: YARN-7064 URL: https://issues.apache.org/jira/browse/YARN-7064 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135811#comment-16135811 ] Miklos Szegedi commented on YARN-6384: -- Please rebase your patch against the latest trunk. > Add

[jira] [Commented] (YARN-6384) Add configuratin to set max cpu usage when strict-resource-usage is false with cgroups

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135808#comment-16135808 ] Miklos Szegedi commented on YARN-6384: -- Thank you for the patch [~lolee_k], I have a few comments.

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-21 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.009.patch Thank you for the reviews [~haibochen]. I attached a test that is

[jira] [Comment Edited] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131651#comment-16131651 ] Miklos Szegedi edited comment on YARN-6623 at 8/18/17 2:53 AM: --- Thank you for

[jira] [Commented] (YARN-6623) Add support to turn off launching privileged containers in the container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131651#comment-16131651 ] Miklos Szegedi commented on YARN-6623: -- Thank you for the patch, [~vvasudev]. I had time to check only

[jira] [Commented] (YARN-7025) Make CGROUPS_ROOT_DIRECTORY configurable in DockerLinuxContainerRuntime

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130936#comment-16130936 ] Miklos Szegedi commented on YARN-7025: -- [~ebadger], Would you like to work on YARN-6968? Feel free to

[jira] [Commented] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130925#comment-16130925 ] Miklos Szegedi commented on YARN-7034: -- [~vinodkv], I have a bit of a concern, while writing the patch

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.001.patch Fixing automation bugs. > DefaultLinuxContainerRuntime and

[jira] [Assigned] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi reassigned YARN-7034: Assignee: Miklos Szegedi > DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime

[jira] [Updated] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7034: - Attachment: YARN-7034.000.patch I created a minimalist patch to start the discussion. Are there

[jira] [Created] (YARN-7034) DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor

2017-08-17 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7034: Summary: DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor Key: YARN-7034 URL:

[jira] [Commented] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-08-16 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129937#comment-16129937 ] Miklos Szegedi commented on YARN-7009: -- The exception above was caused by the fact that we are always

[jira] [Updated] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-08-16 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-7009: - Attachment: YARN-7009.000.patch > TestNMClient.testNMClientNoCleanupOnStop is flaky by design >

[jira] [Commented] (YARN-7025) Make CGROUPS_ROOT_DIRECTORY configurable in DockerLinuxContainerRuntime

2017-08-16 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129828#comment-16129828 ] Miklos Szegedi commented on YARN-7025: -- [~ebadger], YARN-6757 updated the use of

[jira] [Created] (YARN-7009) TestNMClient.testNMClientNoCleanupOnStop is flaky by design

2017-08-14 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7009: Summary: TestNMClient.testNMClientNoCleanupOnStop is flaky by design Key: YARN-7009 URL: https://issues.apache.org/jira/browse/YARN-7009 Project: Hadoop YARN

[jira] [Commented] (YARN-5764) NUMA awareness support for launching containers

2017-08-14 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125987#comment-16125987 ] Miklos Szegedi commented on YARN-5764: -- Thank you, [~devaraj.k] for the patch. I have a few quick

[jira] [Created] (YARN-7001) If shared cache upload is terminated in the middle, the temp file will never be deleted

2017-08-11 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-7001: Summary: If shared cache upload is terminated in the middle, the temp file will never be deleted Key: YARN-7001 URL: https://issues.apache.org/jira/browse/YARN-7001

[jira] [Created] (YARN-6998) Dead code in SCMAdmin

2017-08-11 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6998: Summary: Dead code in SCMAdmin Key: YARN-6998 URL: https://issues.apache.org/jira/browse/YARN-6998 Project: Hadoop YARN Issue Type: Bug

[jira] [Created] (YARN-6997) org.apache.hadoop.yarn.client.SCMAdmin wrong package name

2017-08-11 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6997: Summary: org.apache.hadoop.yarn.client.SCMAdmin wrong package name Key: YARN-6997 URL: https://issues.apache.org/jira/browse/YARN-6997 Project: Hadoop YARN

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.008.patch > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: (was: YARN-6668.008.patch) > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.008.patch > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: (was: YARN-6668.008.patch) > Use cgroup to get container resource utilization >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.008.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-5736) YARN container executor config does not handle white space

2017-08-10 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122091#comment-16122091 ] Miklos Szegedi commented on YARN-5736: -- Sure, please include the addendum as well. > YARN container

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119277#comment-16119277 ] Miklos Szegedi commented on YARN-6852: -- +1 (non-binding) Thanks for the latest patch [~wangda]. >

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.007.patch I rebased the patch on top of trunk instead of YARN-1011 > Use

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.006.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118834#comment-16118834 ] Miklos Szegedi commented on YARN-6033: -- +1 (non-binding). Thank you for the fixes, [~wangda]. Note:

[jira] [Commented] (YARN-6757) Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118767#comment-16118767 ] Miklos Szegedi commented on YARN-6757: -- Yes please. > Refactor the usage of >

[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118688#comment-16118688 ] Miklos Szegedi commented on YARN-6515: -- Thanks [~Naganarasimha], I created YARN-6968 for the Docker

[jira] [Created] (YARN-6968) Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(Contai

2017-08-08 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6968: Summary: Hard coded reference to an absolute pathname in org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.005.patch Attaching the same patch again with my previous checkstyle fixes.

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118660#comment-16118660 ] Miklos Szegedi commented on YARN-6668: -- They are dead code but generate compiler warnings, that is why

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118650#comment-16118650 ] Miklos Szegedi commented on YARN-6930: -- Thank you [~shaneku...@gmail.com]. Yes, let's address the user

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117702#comment-16117702 ] Miklos Szegedi commented on YARN-6852: -- No comments on the design, thanks for the explanation. We can

[jira] [Commented] (YARN-6964) Fair scheduler misuses Resources operations

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117694#comment-16117694 ] Miklos Szegedi commented on YARN-6964: -- Thanks for the patch [~templedf]. {code} 566 Resource

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117675#comment-16117675 ] Miklos Szegedi commented on YARN-6033: -- Thank you for the patch [~wangda]. The last comments are style

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.004.patch > Use cgroup to get container resource utilization >

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117604#comment-16117604 ] Miklos Szegedi commented on YARN-6033: -- {code} 485 cfg->size = 0; 486 conf_file =

[jira] [Updated] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6668: - Attachment: YARN-6668.003.patch Thank you for the comments [~haibochen], I updated the patch. >

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117467#comment-16117467 ] Miklos Szegedi commented on YARN-6668: -- bq. 3) The CGroupsResoureCalculator now takes precedence over

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117464#comment-16117464 ] Miklos Szegedi commented on YARN-6668: -- bq. 5) ContainersMonitorImpl is missing an import of

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117454#comment-16117454 ] Miklos Szegedi commented on YARN-6668: -- bq. 9) We call setCGroupFilePaths() in constructor which

[jira] [Commented] (YARN-6668) Use cgroup to get container resource utilization

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117445#comment-16117445 ] Miklos Szegedi commented on YARN-6668: -- bq. the lock around firstError is unnecessary if we make

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117380#comment-16117380 ] Miklos Szegedi commented on YARN-6930: -- Thank you for the patch, [~shaneku...@gmail.com]. Would you

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117247#comment-16117247 ] Miklos Szegedi commented on YARN-6033: -- [~wangda], sorry there is an issue in the latest change:

[jira] [Commented] (YARN-6852) [YARN-6223] Native code changes to support isolate GPU devices by using CGroups

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117175#comment-16117175 ] Miklos Szegedi commented on YARN-6852: -- Thank you [~wangda], I have a second batch. I could not find a

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-07 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16116907#comment-16116907 ] Miklos Szegedi commented on YARN-6033: -- [~wangda], I agree that is a much cleaner and standard

[jira] [Commented] (YARN-6033) Add support for sections in container-executor configuration file

2017-08-04 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115181#comment-16115181 ] Miklos Szegedi commented on YARN-6033: -- Sorry, I think I found one more in the latest patch. {code} 58

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113713#comment-16113713 ] Miklos Szegedi commented on YARN-6930: -- + [~gphillips], in case he would like to comment. > Admins

[jira] [Updated] (YARN-6895) [FairScheduler] Preemption reservation may cause regular reservation leaks

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szegedi updated YARN-6895: - Attachment: YARN-6895.branch-2.001.patch Attached a new patch. > [FairScheduler] Preemption

[jira] [Commented] (YARN-6930) Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager

2017-08-03 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113698#comment-16113698 ] Miklos Szegedi commented on YARN-6930: -- Can you use yarn.nodemanager.runtime.linux.sandbox-mode and

<    1   2   3   4   5   6   7   8   9   10   >