[jira] [Commented] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.
[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284097#comment-17284097 ] Hadoop QA commented on YARN-10624: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Logfile || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 12s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color} | {color:green}test4tests{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 41s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green}{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 16m 39s{color} | {color:green}{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green}{color} | {color:green} trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s{color} | {color:green}{color} | {color:green} trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m 50s{color} | {color:blue}{color} | {color:blue} Used deprecated FindBugs config; considering switching to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green}{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 48s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 54s{color} | {color:green}{color} | {color:green} the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 54s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 44s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 40s{color} | {color:orange}https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/613/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 94 unchanged - 0 fixed = 95 total (was 94) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 15m 5s{color} | {color:green}{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | |
[jira] [Updated] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.
[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10624: -- Summary: Support max queues limit configuration in new auto created queue, consistent with old auto created. (was: Support max leaf queues limit configuration, consistent with old auto created.) > Support max queues limit configuration in new auto created queue, consistent > with old auto created. > --- > > Key: YARN-10624 > URL: https://issues.apache.org/jira/browse/YARN-10624 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > > Since old created leaf queue has the max leaf queues limit, i think we also > should support this in new auto created queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10624) Support max queues limit configuration in new auto created queue, consistent with old auto created.
[ https://issues.apache.org/jira/browse/YARN-10624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10624: -- Description: Since old created leaf queue has the max leaf queues limit, i think we also should support this in new auto created queue, both the auto created leaf and the auto created parent need limits. (was: Since old created leaf queue has the max leaf queues limit, i think we also should support this in new auto created queue.) > Support max queues limit configuration in new auto created queue, consistent > with old auto created. > --- > > Key: YARN-10624 > URL: https://issues.apache.org/jira/browse/YARN-10624 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Qi Zhu >Assignee: Qi Zhu >Priority: Major > > Since old created leaf queue has the max leaf queues limit, i think we also > should support this in new auto created queue, both the auto created leaf and > the auto created parent need limits. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/13/21, 2:11 AM: - !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~bteke] [~shuzirra] [~pbacsko] [~epayne] I have also confirmed it in my test cluster, i think this is very import for Auto created queue, if you could help review the latest patch? Thanks. was (Author: zhuqi): !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, i think this is very import for Auto created queue, if you could help review the latest patch? Thanks. > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 2:15 PM: - !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, i think this is very import for Auto created queue, if you could help review the latest patch? Thanks. was (Author: zhuqi): !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, i think this is very import for Auto created leaf queue, if you could help review the latest patch? Thanks. > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 2:15 PM: - !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, i think this is very import for Auto created leaf queue, if you could help review the latest patch? Thanks. was (Author: zhuqi): !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, if you could help review the latest patch? Thanks. > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10621) GPU management using OpenCL instead of vendor-specific solutions
[ https://issues.apache.org/jira/browse/YARN-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283717#comment-17283717 ] Adam Antal commented on YARN-10621: --- Thanks for bringing this issue to the community! >From the JIRA details I assume that you're using version 3.1.1, which has not >yet included the {{{DevicePlugin}} interface. This interface has been added to >the code for the very same purpose: discovering custom resources provided by >these plugin - just like the Nvidia GPUs. For more information look at the >umbrella jira: YARN-8851. I don't know if the changes you've been working on are based on this work, but it's the recommended way from 3.3.0 on. How is that aligns with you effort? > GPU management using OpenCL instead of vendor-specific solutions > > > Key: YARN-10621 > URL: https://issues.apache.org/jira/browse/YARN-10621 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, yarn >Reporter: Sotiris Niarchos >Priority: Minor > > As part of the [E2Data research project|https://e2data.eu/], we at the > [Institute of Communication and Computer Systems > (ICCS)|https://www.iccs.gr/en/?noredirect=en_US] of the National Technical > University of Athens, Greece, have been working on a modified version of > Hadoop Yarn where the GPU devices that are available in the underlying > cluster are discovered via a Java wrapper of the OpenCL framework API (namely > [JOCL|https://github.com/gpu/JOCL]), instead of vendor-specific binaries. > In other words, we have shifted towards *a more uniform and high-level > handling of GPUs as "OpenCL-enabled" devices*. This way, we manage to > *decouple GPU discovery/management from vendor-specific technicalities*; > every GPU, no matter the vendor, is the same for E2Data YARN (more > specifically, for the {{NodeManager}} component), provided that the OpenCL > runtime and drivers for the GPU(s) of interest are installed on the > respective node(s) of the cluster. > This way, we *managed to use GPUs other than NVIDIA* (which are the only ones > officially supported via the {{nvidia-smi}} binary) with minimal additional > effort, after our initial changes. > Ultimately, our goal is to *unify every processing unit* that YARN can > possible utilize (CPU cores, GPUs, FPGAs) *behind a common, simple, > high-level interface; that of the OpenCL-enabled device*. > The only drawback of our approach is that vendor-specific info regarding the > GPUs is lost (e.g. temperature). We believe, however, that the lost > information is not necessary for YARN; everything that Hadoop needs in order > to discover and handle GPU devices is provided by OpenCL. > This is just a proposition/a prompt for discussion for the time being. This > modified version is a work in progress. We consider community feedback > regarding the core concept (and the fact that it may constitute a paradigm > shift for YARN) crucial before attaching any patch file and diving into more > (technical) details. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu edited comment on YARN-10532 at 2/12/21, 1:34 PM: - !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, if you could help review the latest patch? Thanks. was (Author: zhuqi): !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, if you could review it ? Thanks. > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17283691#comment-17283691 ] Qi Zhu commented on YARN-10532: --- !image-2021-02-12-21-32-02-267.png|width=1085,height=764! cc [~gandras] [~snemeth] [~ztang] [~epayne] I have also confirmed it in my test cluster, if you could review it ? Thanks. > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10532) Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is not being used
[ https://issues.apache.org/jira/browse/YARN-10532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qi Zhu updated YARN-10532: -- Attachment: image-2021-02-12-21-32-02-267.png > Capacity Scheduler Auto Queue Creation: Allow auto delete queue when queue is > not being used > > > Key: YARN-10532 > URL: https://issues.apache.org/jira/browse/YARN-10532 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan >Assignee: Qi Zhu >Priority: Major > Attachments: YARN-10532.001.patch, YARN-10532.002.patch, > YARN-10532.003.patch, YARN-10532.004.patch, YARN-10532.005.patch, > YARN-10532.006.patch, YARN-10532.007.patch, YARN-10532.008.patch, > YARN-10532.009.patch, YARN-10532.010.patch, YARN-10532.011.patch, > YARN-10532.012.patch, YARN-10532.013.patch, YARN-10532.014.patch, > YARN-10532.015.patch, YARN-10532.016.patch, YARN-10532.017.patch, > YARN-10532.018.patch, YARN-10532.019.patch, YARN-10532.020.patch, > image-2021-02-12-21-32-02-267.png > > > It's better if we can delete auto-created queues when they are not in use for > a period of time (like 5 mins). It will be helpful when we have a large > number of auto-created queues (e.g. from 500 users), but only a small subset > of queues are actively used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10625) FairScheduler: add global flag to disable AM-preemption
Peter Bacsko created YARN-10625: --- Summary: FairScheduler: add global flag to disable AM-preemption Key: YARN-10625 URL: https://issues.apache.org/jira/browse/YARN-10625 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Affects Versions: 3.3.0 Reporter: Peter Bacsko Assignee: Peter Bacsko YARN-9537 added a feature to disable AM preemption on a per queue basis. This is a nice enhancement, but it's very inconvenient if the cluster has a lot of queues or queues dynamically created/deleted regularly (static queue configuration changes). It's a legitimate use-case to have AM preemption turned off completely. To make it easier, add property which acts as a global flag for this feature. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10588) Percentage of queue and cluster is zero in WebUI
[ https://issues.apache.org/jira/browse/YARN-10588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17282988#comment-17282988 ] Bilwa S T edited comment on YARN-10588 at 2/12/21, 8:23 AM: [~epayne] Modifying *DominantResourceCalculator#isInvalidDivisor* to match logic of *DominantResourceCalculator#divide* is nothing but returning true only if all resource value is *0*. We already have a method called *DominantResourceCalculator#isAllInvalidDivisor* which will return true only if all resources are *zero*. I think we can just change isInvalidDivisor to isAllInvalidDivisor. Correct me if i am wrong was (Author: bilwast): [~epayne] Modifying *DominantResourceCalculator#isInvalidDivisor* to match logic of *DominantResourceCalculator#divide* is nothing but returning true only if all resource value is *0*. We already have a method called *DominantResourceCalculator#isAllInvalidDivisor* which will return true only if all resources are *zero*. I think we can just change isInvalidDivisor to isAllInvalidDivisor. > Percentage of queue and cluster is zero in WebUI > - > > Key: YARN-10588 > URL: https://issues.apache.org/jira/browse/YARN-10588 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bilwa S T >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10588.001.patch, YARN-10588.002.patch, > YARN-10588.003.patch > > > Steps to reproduce: > Configure below property in resource-types.xml > {code:java} > > yarn.resource-types > yarn.io/gpu > {code} > Submit a job > In UI you can see % Of Queue and % Of Cluster is zero for the submitted > application > > This is because in SchedulerApplicationAttempt has below check for > calculating queueUsagePerc and clusterUsagePerc > {code:java} > if (!calc.isInvalidDivisor(cluster)) { > float queueCapacityPerc = queue.getQueueInfo(false, false) > .getCapacity(); > queueUsagePerc = calc.divide(cluster, usedResourceClone, > Resources.multiply(cluster, queueCapacityPerc)) * 100; > if (Float.isNaN(queueUsagePerc) || Float.isInfinite(queueUsagePerc)) { > queueUsagePerc = 0.0f; > } > clusterUsagePerc = > calc.divide(cluster, usedResourceClone, cluster) * 100; > } > {code} > calc.isInvalidDivisor(cluster) always returns true as gpu resource is 0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org