[jira] [Created] (YUNIKORN-113) [webapp] Change 0 as default value for maximum queue capacity
Kinga Marton created YUNIKORN-113: - Summary: [webapp] Change 0 as default value for maximum queue capacity Key: YUNIKORN-113 URL: https://issues.apache.org/jira/browse/YUNIKORN-113 Project: Apache YuniKorn Issue Type: Sub-task Components: webapp Reporter: Kinga Marton Assignee: Kinga Marton If there is no maximum capacity set for a queue, in the response from the backend there will be an empty string. However in the webapp 0 value is displayed, what is wrong, because 0 means that the resource is not available. We should change its default value to "N\A" or something with the meaning of not available/not set instead of setting it to 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Updated] (YUNIKORN-113) [webapp] Change default value for maximum queue capacity
[ https://issues.apache.org/jira/browse/YUNIKORN-113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YUNIKORN-113: -- Summary: [webapp] Change default value for maximum queue capacity (was: [webapp] Change 0 as default value for maximum queue capacity) > [webapp] Change default value for maximum queue capacity > > > Key: YUNIKORN-113 > URL: https://issues.apache.org/jira/browse/YUNIKORN-113 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: webapp >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > > If there is no maximum capacity set for a queue, in the response from the > backend there will be an empty string. However in the webapp 0 value is > displayed, what is wrong, because 0 means that the resource is not available. > We should change its default value to "N\A" or something with the meaning of > not available/not set instead of setting it to 0. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[
https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093190#comment-17093190
]
Kinga Marton commented on YUNIKORN-93:
--
[~wwei], below you can find my answers to your observations:
{quote}But what I meant to say was the capacity is 0, which means all values
for different resource types are all 0. is that also a valid case? should we
allow that?
{quote}
I think doesn't make sense to have a queue having 0 for all capacities, that
queue would be unusable. However I can think about one scenario: what about is
a user wants to make a queue temporally unavailable, would that make sense to
set all the capacities to 0 instead of deleting and then recreating it?
{quote}Cache should be our source of truth, and UI should get info from the
cache. The scheduler package are the internal states for each scheduling thread
(in case we have more than 1 in feature). that is not supposed to be exposed
outside. Scheduler has more information, but I don't think we need to expose
all of that info via REST. A more appropriate place for them is the event
system we are going to build.
{quote}
I am not very familiar with the internals yet. Missing the reservations seemed
a valid thing for me. [~wilfreds] what do you think about this?
bq. apart from these, how we handle the unlimited case?
The unlimited case should not be equal to 0. We should set "N/A" or something
with the meaning of not set or not available. I created a subtask for this
because it seems that the webapp is setting it to 0 even if in the core I set
N/A or empty String. See YUNIKORN-113
> Queue maximum capacity cleanup
> --
>
> Key: YUNIKORN-93
> URL: https://issues.apache.org/jira/browse/YUNIKORN-93
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Minor
>
> Right now the absolute used capacity is hardcoded to 20%.
> The usage bar is rendered by this value, but currently, it is hardcoded.
> Note, both capacity/max capacity could be 0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Updated] (YUNIKORN-113) [webapp] Change default value for maximum queue capacity
[ https://issues.apache.org/jira/browse/YUNIKORN-113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YUNIKORN-113: -- Attachment: Screenshot 2020-04-27 at 10.10.48.png > [webapp] Change default value for maximum queue capacity > > > Key: YUNIKORN-113 > URL: https://issues.apache.org/jira/browse/YUNIKORN-113 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: webapp >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Attachments: Screenshot 2020-04-27 at 10.10.48.png > > > If there is no maximum capacity set for a queue, in the response from the > backend there will be an empty string. However in the webapp 0 value is > displayed, what is wrong, because 0 means that the resource is not available. > We should change its default value to "N\A" or something with the meaning of > not available/not set instead of setting it to 0. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Updated] (YUNIKORN-92) YuniKorn web site should have incubator logo
[ https://issues.apache.org/jira/browse/YUNIKORN-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YUNIKORN-92: --- Labels: pull-request-available (was: ) > YuniKorn web site should have incubator logo > > > Key: YUNIKORN-92 > URL: https://issues.apache.org/jira/browse/YUNIKORN-92 > Project: Apache YuniKorn > Issue Type: Bug > Components: website >Reporter: Weiwei Yang >Assignee: Wilfred Spiegelenburg >Priority: Major > Labels: pull-request-available > > Fix the logo in yunikorn web-site -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093481#comment-17093481 ] Kinga Marton commented on YUNIKORN-93: -- Right now there is some queue label colouring performed based on the actual absolute used resource: [https://github.com/apache/incubator-yunikorn-web/blob/master/src/app/components/queue-rack/queue-rack.component.ts#L63-L73]. In my opinion we should calculate absolute usage for each resource type, and not one global value. And colour the queue labels according the max value from the absolute usage. For example if we will have the following usage: [memory:75%, vcores: 20%], decide on the colour based on the max value (what in our case is 75%). cc [~akhilpb], since there are some UI related questions as well. > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Updated] (YUNIKORN-93) Queue maximum capacity cleanup
[ https://issues.apache.org/jira/browse/YUNIKORN-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kinga Marton updated YUNIKORN-93: - Priority: Major (was: Minor) > Queue maximum capacity cleanup > -- > > Key: YUNIKORN-93 > URL: https://issues.apache.org/jira/browse/YUNIKORN-93 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > > Right now the absolute used capacity is hardcoded to 20%. > The usage bar is rendered by this value, but currently, it is hardcoded. > Note, both capacity/max capacity could be 0. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Assigned] (YUNIKORN-55) Add pod labels and annotations to allocation ask attributes
[ https://issues.apache.org/jira/browse/YUNIKORN-55?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Antal reassigned YUNIKORN-55: -- Assignee: Adam Antal > Add pod labels and annotations to allocation ask attributes > --- > > Key: YUNIKORN-55 > URL: https://issues.apache.org/jira/browse/YUNIKORN-55 > Project: Apache YuniKorn > Issue Type: Bug > Components: core - scheduler, shim - kubernetes, webapp >Reporter: Weiwei Yang >Assignee: Adam Antal >Priority: Major > > In YUNIKORN-54, we simplify the way to generate application IDs. The side > effect is when we look at info from web UI, we lose some info about the pod > info such as pod name, namespace, etc. > A proper way to handle this is to get this info from pod and add labels, > annotations info to allocation ask as attributes, and then send to > scheduler-core. Also on web rest API, we need to display these attributes too. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-55) Add pod labels and annotations to allocation ask attributes
[
https://issues.apache.org/jira/browse/YUNIKORN-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093546#comment-17093546
]
Adam Antal commented on YUNIKORN-55:
Hi [~wwei],
Just assigned to me this and thought about it a bit. Currently I only see the
{{AllocationAsk}} object to be sent by the RM where we share any allocation
(pod) specific information, but nowhere else. There we have {{string
partitionName = 3}} which lets know the scheduler about the name of the
partition but at the point when we ask the scheduler for allocation we don't
have an existing pod, thus pod name - I don't know how could we simply inform
the scheduler about this. I could work on exposing only the partition name, but
only that. Any thoughts?
> Add pod labels and annotations to allocation ask attributes
> ---
>
> Key: YUNIKORN-55
> URL: https://issues.apache.org/jira/browse/YUNIKORN-55
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler, shim - kubernetes, webapp
>Reporter: Weiwei Yang
>Assignee: Adam Antal
>Priority: Major
> Time Spent: 0.1h
> Remaining Estimate: 4h
>
> In YUNIKORN-54, we simplify the way to generate application IDs. The side
> effect is when we look at info from web UI, we lose some info about the pod
> info such as pod name, namespace, etc.
> A proper way to handle this is to get this info from pod and add labels,
> annotations info to allocation ask as attributes, and then send to
> scheduler-core. Also on web rest API, we need to display these attributes too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Assigned] (YUNIKORN-53) Use asserts for nil error checks in test code
[
https://issues.apache.org/jira/browse/YUNIKORN-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Antal reassigned YUNIKORN-53:
--
Assignee: Adam Antal
> Use asserts for nil error checks in test code
> -
>
> Key: YUNIKORN-53
> URL: https://issues.apache.org/jira/browse/YUNIKORN-53
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: test - smoke, test - unit
>Reporter: Wilfred Spiegelenburg
>Assignee: Adam Antal
>Priority: Minor
> Labels: newbie
>
> In a lot of tests we use the construct:
> {code}
> if err != nil {
> t.Fatal("some text"
> }{code}
> We should replace that with the simple:
> {code}
> assert.NilError(t, err, "some text"){code}
> that is part of the standard assert package we already use in the test code
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Updated] (YUNIKORN-113) [webapp] Change default value for maximum queue capacity
[ https://issues.apache.org/jira/browse/YUNIKORN-113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated YUNIKORN-113: Labels: pull-request-available (was: ) > [webapp] Change default value for maximum queue capacity > > > Key: YUNIKORN-113 > URL: https://issues.apache.org/jira/browse/YUNIKORN-113 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: webapp >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2020-04-27 at 10.10.48.png > > > If there is no maximum capacity set for a queue, in the response from the > backend there will be an empty string. However in the webapp 0 value is > displayed, what is wrong, because 0 means that the resource is not available. > We should change its default value to "N\A" or something with the meaning of > not available/not set instead of setting it to 0. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-55) Add pod labels and annotations to allocation ask attributes
[
https://issues.apache.org/jira/browse/YUNIKORN-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093731#comment-17093731
]
Weiwei Yang commented on YUNIKORN-55:
-
Hi [~adam.antal]
{quote}
but at the point when we ask the scheduler for allocation we don't have an
existing pod, thus pod name - I don't know how could we simply inform the
scheduler about this.
{quote}
When we ask for the allocation, the pod is already created (this is triggered
when a pod is added to k8s). So we should be able to get pod spec info, such as
pod name, namespace, labels, and annotations. See more in
[https://github.com/apache/incubator-yunikorn-k8shim/blob/23e3f084d708096a90f54bfc969c3a9f08975fac/pkg/cache/context.go#L495.]
Can we get this info and put them into tags of AllocationAsk so they can be
carried over to the scheduler-core?
> Add pod labels and annotations to allocation ask attributes
> ---
>
> Key: YUNIKORN-55
> URL: https://issues.apache.org/jira/browse/YUNIKORN-55
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler, shim - kubernetes, webapp
>Reporter: Weiwei Yang
>Assignee: Adam Antal
>Priority: Major
> Time Spent: 0.6h
> Remaining Estimate: 3.5h
>
> In YUNIKORN-54, we simplify the way to generate application IDs. The side
> effect is when we look at info from web UI, we lose some info about the pod
> info such as pod name, namespace, etc.
> A proper way to handle this is to get this info from pod and add labels,
> annotations info to allocation ask as attributes, and then send to
> scheduler-core. Also on web rest API, we need to display these attributes too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-112) Fix ASF release issues
[ https://issues.apache.org/jira/browse/YUNIKORN-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093756#comment-17093756 ] Weiwei Yang commented on YUNIKORN-112: -- Another thing to fix is: we can probably add a symlink to the examples in the top-level dir. So it will be easier for users to run examples. > Fix ASF release issues > -- > > Key: YUNIKORN-112 > URL: https://issues.apache.org/jira/browse/YUNIKORN-112 > Project: Apache YuniKorn > Issue Type: Bug > Components: release >Reporter: Weiwei Yang >Priority: Blocker > > Felix gives us some feedback on our first release, we should make sure they > are fixed in the next release, items include: > - incubating in name > - signature and hash fine > - DISCLAIMER is ok - as above > - LICENSE and NOTICE are fine > - Ok with binary files - see note below > - See note below on ASF headers > - didn't compile from source > 1. SHOULD: instead of a personal directory, release SHOULD be staged on > [https://dist.apache.org/repos/dist/dev/incubator/yunikorn]. > [https://incubator.apache.org/guides/releasemanagement.html#podling_constraints] > this include the signing KEYS file that should be kept as a project KEYS file > instead of a personal one/location > 2. it is a bit strange that you have 5 LICENSE files, you can exclude the one > in subdirectories when you build the source tarball? > 3. might want to check the license for the image files: png, jpg - where are > they from? > 4. SHOULD: a large number of files should also have ASF headers, eg. .mod, > .yaml, .md, .go > 5. nice to have: more details on go setup for dev environment (maybe just a > quick link), go setup info without GoLand IDE. It could be helpful to know > more on how to build (is this the build step? > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/developer-guide.md#core-component-build]) > 6. SHOULD: put Apache Incubator logo on the > websitehttps://[incubator.apache.org/guides/press-kit.html|http://incubator.apache.org/guides/press-kit.html] > [http://yunikorn.apache.org/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
[jira] [Commented] (YUNIKORN-42) Better to support POD events for YuniKorn to troubleshoot allocation failures
[ https://issues.apache.org/jira/browse/YUNIKORN-42?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094145#comment-17094145 ] Weiwei Yang commented on YUNIKORN-42: - Hi [~adam.antal] Thanks for working on this!! I went through previous discussions and the design doc, I think there are some key areas we need to sort out. 1) expose via the rest server or flush to k8s event system The design addresses both, that makes sense. But which one is more important, and which one will come first? My opinion is the k8s event system. Because this is how users consume it. The key purpose of this Jira is not making our life easier, we need to make users' life easier. That said, by describing pods/nodes, they can understand e.g 90% of the reasons why a pod is not allocated. 2) the cache The key problem is the cache, how we build an efficient cache. The scheduler can push events (or records) to this cache, and this cache can be queried (via rest) or periodically flushed (to k8s event system). 3) aggregate records When the scheduler pushes events/records to the cache, dup records should be aggregated. Therefore it is important to design the schema of each record, so we can properly aggregate them. An example is, when we try to assign a pod, it may fail again and again in the scheduler loop, in such case, we would say "pod is unable to be allocated due to xxx reason, N times in past X seconds". > Better to support POD events for YuniKorn to troubleshoot allocation failures > - > > Key: YUNIKORN-42 > URL: https://issues.apache.org/jira/browse/YUNIKORN-42 > Project: Apache YuniKorn > Issue Type: Task >Reporter: Wangda Tan >Assignee: Adam Antal >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Now it is tricky to do troubleshoot for pod allocation, we need better expose > this information to POD description. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@yunikorn.apache.org For additional commands, e-mail: issues-h...@yunikorn.apache.org
