[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105156#comment-17105156 ] Kinga Marton commented on YUNIKORN-93: -- Thank you [~wilfreds] for the review! > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > Attachments: queuesResponse.json > > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[
https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100883#comment-17100883
]
Kinga Marton commented on YUNIKORN-93:
--
Thank you [~wilfreds] for the review! Before updating the PR I would like to
discuss/share my opinion on a few possible cases:
* *if {{capacity}} or {{used}} is nil*: instead of playing with +INF, I would
return nil, because we don't have enough data to calculate the percentage, .
Since this is a percentage value, what would be the meaning of +INF here? 100%?
I think the clear solution is to just not calculate it and in the web ui will
be shown as {{'n/a'}}
* *all zeros for {{used}}*: this will result in all zeros for the calculated
capacity as well
* *all zeros for {{capacity}}*: However, this is not a valid case, because the
config validator will [return an error if the total of max capacities =
0|https://github.com/apache/incubator-yunikorn-core/blob/master/pkg/common/configs/configvalidator.go#L99].
we should handle this case as well, so in this case my opinion is that we
should return zero resource.
* *{{capacity}} set to 0, usage is larger than 0*: I this this should not be a
valid case, but I would return zero resource because of the 0 capacity setting
* *no {{capacity}} set, but {{usage}} is set*: if no capacity is set but we
have resource usage I would return an empty resource, because we cannot
calculate the absolute resource usage, since we don’t know what would mean
100%, so the web ui will show n/a (I would not make here an exception for the
case when the usage = 0, however we know that this would mean 0%)
* *overflow case*: {{used}} = maxInt64, {{available}} is 10. Is this a valid
scenario? Do we allow ho have higher usage than the configured max capacity?
Note: above I am using capacity for simplicity, but that is max capacity, since
the abs value calculation is maxValue/usage*100
[~wilfreds], [~wwei] what do you think?
> Queue maximum capacity cleanup
> --
>
> Key: YUNIKORN-93
> URL: https://issues.apache.org/jira/browse/YUNIKORN-93
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
> Labels: pull-request-available
> Attachments: queuesResponse.json
>
>
> Right now the absolute used capacity is hardcoded to 20%.
> The usage bar is rendered by this value, but currently, it is hardcoded.
> Note, both capacity/max capacity could be 0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17100527#comment-17100527 ] Wilfred Spiegelenburg commented on YUNIKORN-93: --- I have reviewed the core PR and have some comments. I have merged the web PR, that one is done. > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Attachments: queuesResponse.json > > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17099988#comment-17099988 ] Kinga Marton commented on YUNIKORN-93: -- [~wilfreds], [~wwei] can you please help me with the review of the core side as well? ([https://github.com/apache/incubator-yunikorn-core/pull/134]) > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Attachments: queuesResponse.json > > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096379#comment-17096379 ] Kinga Marton commented on YUNIKORN-93: -- Sure, [~akhilpb]. I have attached a sample response to this Jira. > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Attachments: queuesResponse.json > > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096177#comment-17096177 ] Akhil PB commented on YUNIKORN-93: -- [~kmarton] Could you pls share latest /queues response? > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17095435#comment-17095435 ] Kinga Marton commented on YUNIKORN-93: -- I have created two PRs for this issue (one for the core side and one for the webapp side). [~akhilpb], [~wwei]/[~wilfreds] can you please help me with the review? I have addressed the following issues in the PR's: [core side] * corrected typos from [queue_config.md|https://github.com/apache/incubator-yunikorn-core/blob/master/docs/queue_config.md] * added a check for negative resources. There is already a check for total resources to be greater than 0 * removed the hardcoded 20% for absolute used resource and added a calculated value for each resource type what has a maximum resources defined * added some test cases for this calculation * refactored duplicated getQueueInfos method and made one recursive call. Also I am returning only one QueueDAOInfo object, because there was no sense to keep an array for it, because it contains only the root queue [webapp side] * set the queue label colour and the progress bar based on the max absolute used resource value * some smaller fixes as a consequence of the core side fixes * showing 'N/A' instead of 0 if there is no value defined for the resources is handled in YUNIKORN-113 > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093481#comment-17093481 ] Kinga Marton commented on YUNIKORN-93: -- Right now there is some queue label colouring performed based on the actual absolute used resource: [https://github.com/apache/incubator-yunikorn-web/blob/master/src/app/components/queue-rack/queue-rack.component.ts#L63-L73]. In my opinion we should calculate absolute usage for each resource type, and not one global value. And colour the queue labels according the max value from the absolute usage. For example if we will have the following usage: [memory:75%, vcores: 20%], decide on the colour based on the max value (what in our case is 75%). cc [~akhilpb], since there are some UI related questions as well. > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[
https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17093190#comment-17093190
]
Kinga Marton commented on YUNIKORN-93:
--
[~wwei], below you can find my answers to your observations:
{quote}But what I meant to say was the capacity is 0, which means all values
for different resource types are all 0. is that also a valid case? should we
allow that?
{quote}
I think doesn't make sense to have a queue having 0 for all capacities, that
queue would be unusable. However I can think about one scenario: what about is
a user wants to make a queue temporally unavailable, would that make sense to
set all the capacities to 0 instead of deleting and then recreating it?
{quote}Cache should be our source of truth, and UI should get info from the
cache. The scheduler package are the internal states for each scheduling thread
(in case we have more than 1 in feature). that is not supposed to be exposed
outside. Scheduler has more information, but I don't think we need to expose
all of that info via REST. A more appropriate place for them is the event
system we are going to build.
{quote}
I am not very familiar with the internals yet. Missing the reservations seemed
a valid thing for me. [~wilfreds] what do you think about this?
bq. apart from these, how we handle the unlimited case?
The unlimited case should not be equal to 0. We should set "N/A" or something
with the meaning of not set or not available. I created a subtask for this
because it seems that the webapp is setting it to 0 even if in the core I set
N/A or empty String. See YUNIKORN-113
> Queue maximum capacity cleanup
> --
>
> Key: YUNIKORN-93
> URL: https://issues.apache.org/jira/browse/YUNIKORN-93
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Minor
>
> Right now the absolute used capacity is hardcoded to 20%.
> The usage bar is rendered by this value, but currently, it is hardcoded.
> Note, both capacity/max capacity could be 0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
