[jira] [Commented] (YARN-9760) Support configuring application priorities on a workflow level

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939818#comment-16939818 ] Jonathan Hung commented on YARN-9760: - Thanks [~varun_saxena]. Seems fine. A couple nits: * Do we

[jira] [Commented] (YARN-9859) Refactor OpportunisticContainerAllocator

2019-09-27 Thread Jira
[ https://issues.apache.org/jira/browse/YARN-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939815#comment-16939815 ] Íñigo Goiri commented on YARN-9859: --- I think the correct indentation would be: {code}

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939786#comment-16939786 ] Jonathan Hung commented on YARN-9858: - trunk/branch-2 runs pass. The branch-3.2 failures don't seem

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939761#comment-16939761 ] Hadoop QA commented on YARN-9858: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939759#comment-16939759 ] Hadoop QA commented on YARN-9858: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939752#comment-16939752 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939751#comment-16939751 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939748#comment-16939748 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-9864) Format CS Configuration present in Configuration Store

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9864: Attachment: YARN-9864-001.patch > Format CS Configuration present in Configuration Store >

[jira] [Updated] (YARN-9864) Format CS Configuration present in Configuration Store

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9864: Component/s: capacity scheduler > Format CS Configuration present in Configuration Store >

[jira] [Created] (YARN-9864) Format CS Configuration present in Configuration Store

2019-09-27 Thread Prabhu Joseph (Jira)
Prabhu Joseph created YARN-9864: --- Summary: Format CS Configuration present in Configuration Store Key: YARN-9864 URL: https://issues.apache.org/jira/browse/YARN-9864 Project: Hadoop YARN Issue

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939688#comment-16939688 ] Jonathan Hung commented on YARN-9858: - * Attached 003 patch which removes the null check as suggested

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-2.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-3.1.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: (was: YARN-9858-branch-3.1.001.patch) > Optimize RMContext

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: (was: YARN-9858-branch-2.001.patch) > Optimize RMContext

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939680#comment-16939680 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-2.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Commented] (YARN-9860) Enable service mode for Docker containers on YARN

2019-09-27 Thread Eric Badger (Jira)
[ https://issues.apache.org/jira/browse/YARN-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939677#comment-16939677 ] Eric Badger commented on YARN-9860: --- If there are no log directories, how would you attack debugging

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-3.1.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: (was: YARN-9858-branch-3.1.001.patch) > Optimize RMContext

[jira] [Commented] (YARN-8403) Nodemanager logs failed to download file with INFO level

2019-09-27 Thread Brahma Reddy Battula (Jira)
[ https://issues.apache.org/jira/browse/YARN-8403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939660#comment-16939660 ] Brahma Reddy Battula commented on YARN-8403: IMO,this can be backported to 2.X also can be

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-3.1.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858-branch-3.2.001.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9737) Performance degradation, Distributed Opportunistic Scheduling

2019-09-27 Thread Babble Shack (Jira)
[ https://issues.apache.org/jira/browse/YARN-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated YARN-9737: --- Attachment: task_throughput_bp_100j_30t.svg > Performance degradation, Distributed Opportunistic

[jira] [Updated] (YARN-9737) Performance degradation, Distributed Opportunistic Scheduling

2019-09-27 Thread Babble Shack (Jira)
[ https://issues.apache.org/jira/browse/YARN-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated YARN-9737: --- Attachment: jct_100j_30t.svg > Performance degradation, Distributed Opportunistic Scheduling >

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858.003.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Updated] (YARN-9737) Performance degradation, Distributed Opportunistic Scheduling

2019-09-27 Thread Babble Shack (Jira)
[ https://issues.apache.org/jira/browse/YARN-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Babble Shack updated YARN-9737: --- Attachment: jct_100j_30t.png task_throughput_bp_100j_30t.png > Performance

[jira] [Moved] (YARN-9863) Randomize List of Resources to Localize

2019-09-27 Thread David Mollitor (Jira)
[ https://issues.apache.org/jira/browse/YARN-9863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor moved MAPREDUCE-7243 to YARN-9863: - Component/s: (was: performance) (was:

[jira] [Updated] (YARN-9862) yarn-services-core test timeout

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9862: Attachment: YARN-9862-001.patch > yarn-services-core test timeout > ---

[jira] [Commented] (YARN-9860) Enable service mode for Docker containers on YARN

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939534#comment-16939534 ] Prabhu Joseph commented on YARN-9860: - [~skumpf] [~sunilg] Can you review this Jira when you get time.

[jira] [Updated] (YARN-9860) Enable service mode for Docker containers on YARN

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9860: Attachment: YARN-9860-002.patch > Enable service mode for Docker containers on YARN >

[jira] [Created] (YARN-9862) yarn-services-core test timeout

2019-09-27 Thread Prabhu Joseph (Jira)
Prabhu Joseph created YARN-9862: --- Summary: yarn-services-core test timeout Key: YARN-9862 URL: https://issues.apache.org/jira/browse/YARN-9862 Project: Hadoop YARN Issue Type: Bug

[jira] [Commented] (YARN-9859) Refactor OpportunisticContainerAllocator

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939418#comment-16939418 ] Hadoop QA commented on YARN-9859: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939409#comment-16939409 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-9860) Enable service mode for Docker containers on YARN

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939354#comment-16939354 ] Hadoop QA commented on YARN-9860: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4946) RM should not consider an application as COMPLETED when log aggregation is not in a terminal state

2019-09-27 Thread Adam Antal (Jira)
[ https://issues.apache.org/jira/browse/YARN-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939352#comment-16939352 ] Adam Antal commented on YARN-4946: -- I'm +1 (non-binding) on reverting this patch, and check it in detail.

[jira] [Commented] (YARN-9699) Migration tool that help to generate CS config based on FS config

2019-09-27 Thread Peter Bacsko (Jira)
[ https://issues.apache.org/jira/browse/YARN-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939325#comment-16939325 ] Peter Bacsko commented on YARN-9699: +1 for standalone tool. Having to have a running RM instance is a

[jira] [Commented] (YARN-9861) The ResourceManager log reports an error "Too many open files", the analysis is related to the service

2019-09-27 Thread Zhankun Tang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939320#comment-16939320 ] Zhankun Tang commented on YARN-9861: [~billie.rinaldi], if any chance, could you please take a look at

[jira] [Updated] (YARN-9861) The ResourceManager log reports an error "Too many open files", the analysis is related to the service

2019-09-27 Thread Zhankun Tang (Jira)
[ https://issues.apache.org/jira/browse/YARN-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhankun Tang updated YARN-9861: --- Attachment: submarine_kerasgesv2date20190807.json > The ResourceManager log reports an error "Too

[jira] [Commented] (YARN-9699) Migration tool that help to generate CS config based on FS config

2019-09-27 Thread Sunil G (Jira)
[ https://issues.apache.org/jira/browse/YARN-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939317#comment-16939317 ] Sunil G commented on YARN-9699: --- I would vote for standalone like below {code:java} Usage: yarn

[jira] [Created] (YARN-9861) The ResourceManager log reports an error "Too many open files", the analysis is related to the service

2019-09-27 Thread jason (Jira)
jason created YARN-9861: --- Summary: The ResourceManager log reports an error "Too many open files", the analysis is related to the service Key: YARN-9861 URL: https://issues.apache.org/jira/browse/YARN-9861

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Hadoop QA (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939291#comment-16939291 ] Hadoop QA commented on YARN-9858: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (YARN-9860) Enable service mode for Docker containers on YARN

2019-09-27 Thread Prabhu Joseph (Jira)
[ https://issues.apache.org/jira/browse/YARN-9860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9860: Attachment: YARN-9860-001.patch > Enable service mode for Docker containers on YARN >

[jira] [Commented] (YARN-9699) Migration tool that help to generate CS config based on FS config

2019-09-27 Thread Szilard Nemeth (Jira)
[ https://issues.apache.org/jira/browse/YARN-9699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939248#comment-16939248 ] Szilard Nemeth commented on YARN-9699: -- Hi [~sunilg] / [~Prabhu Joseph]! / [~pbacsko]! We have been

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Bibin Chundatt (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939245#comment-16939245 ] Bibin Chundatt commented on YARN-9858: -- I think we should fix the testcase. Setting conf to rmcontext

[jira] [Comment Edited] (YARN-5277) when localizers fail due to resource timestamps being out, provide more diagnostics

2019-09-27 Thread Adam Antal (Jira)
[ https://issues.apache.org/jira/browse/YARN-5277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939189#comment-16939189 ] Adam Antal edited comment on YARN-5277 at 9/27/19 7:39 AM: --- Customer of ours is

[jira] [Commented] (YARN-5277) when localizers fail due to resource timestamps being out, provide more diagnostics

2019-09-27 Thread Adam Antal (Jira)
[ https://issues.apache.org/jira/browse/YARN-5277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939189#comment-16939189 ] Adam Antal commented on YARN-5277: -- Customer of ours is hitting this. If [~ste...@apache.org] is not

[jira] [Comment Edited] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939181#comment-16939181 ] Jonathan Hung edited comment on YARN-9858 at 9/27/19 7:29 AM: -- Thanks

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939181#comment-16939181 ] Jonathan Hung commented on YARN-9858: - Thanks [~bibinchundatt]. This won't trigger during normal

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Bibin Chundatt (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939177#comment-16939177 ] Bibin Chundatt commented on YARN-9858: -- Over all patch looks good to me. Minor query . {code} 3803

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939163#comment-16939163 ] Jonathan Hung commented on YARN-9858: - Uploaded 002 patch which implements the above approach.

[jira] [Updated] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9858: Attachment: YARN-9858.002.patch > Optimize RMContext getExclusiveEnforcedPartitions >

[jira] [Commented] (YARN-9858) Optimize RMContext getExclusiveEnforcedPartitions

2019-09-27 Thread Bibin Chundatt (Jira)
[ https://issues.apache.org/jira/browse/YARN-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939161#comment-16939161 ] Bibin Chundatt commented on YARN-9858: -- +1 for approach. > Optimize RMContext