[jira] [Resolved] (YUNIKORN-140) Create helm chart repository and publish it to helmhub
[ https://issues.apache.org/jira/browse/YUNIKORN-140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-140. -- Fix Version/s: 0.9 Resolution: Fixed > Create helm chart repository and publish it to helmhub > -- > > Key: YUNIKORN-140 > URL: https://issues.apache.org/jira/browse/YUNIKORN-140 > Project: Apache YuniKorn > Issue Type: Improvement > Components: build >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > > Create a helm chart repository and publish the charts to > [https://hub.helm.sh/.|https://hub.helm.sh/] This will further help user to > try yunikorn out easily. > How to create the helm chart repository is described here: > [https://helm.sh/docs/topics/chart_repository/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-126) Add missing execute bit when generating the release package
[ https://issues.apache.org/jira/browse/YUNIKORN-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-126. -- Fix Version/s: 0.9 Resolution: Fixed > Add missing execute bit when generating the release package > --- > > Key: YUNIKORN-126 > URL: https://issues.apache.org/jira/browse/YUNIKORN-126 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: release >Reporter: Wilfred Spiegelenburg >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > > the release package does not set the execute bit on the build script. > The script must add have that added to allow unpack and run without changes > File: > [https://github.com/apache/incubator-yunikorn-release/blob/master/release-top-level-artifacts/build-docker-images.sh] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-164) Add initial testing bucket
Weiwei Yang created YUNIKORN-164: Summary: Add initial testing bucket Key: YUNIKORN-164 URL: https://issues.apache.org/jira/browse/YUNIKORN-164 Project: Apache YuniKorn Issue Type: Sub-task Components: test - e2e Reporter: Weiwei Yang For initial testing bucket, I propose to consider: * basic scheduling ** submit a job that simply runs several sleep pods ** validate the job can be scheduled correctly by YK ** validate the pods can be allocated correctly ** validate the job info from YK's rest API ** validate the queue info from YK's rest API ** validate the node info from YK's rest API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-163) Add initial testing framework code
Weiwei Yang created YUNIKORN-163: Summary: Add initial testing framework code Key: YUNIKORN-163 URL: https://issues.apache.org/jira/browse/YUNIKORN-163 Project: Apache YuniKorn Issue Type: Sub-task Components: test - e2e Reporter: Weiwei Yang The initial testing framework code. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-161) End to end testing framework for YuniKorn on K8s
Weiwei Yang created YUNIKORN-161: Summary: End to end testing framework for YuniKorn on K8s Key: YUNIKORN-161 URL: https://issues.apache.org/jira/browse/YUNIKORN-161 Project: Apache YuniKorn Issue Type: New Feature Components: test - e2e Reporter: Weiwei Yang Assignee: Ayub Pathan This effort is to design and build the e2e testing framework for YuniKorn. The tests should be able to run in any K8s deployment, even minikube, single node K8s, EKS, etc. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-158) Admission controller deployment file should use the same version as the scheduler
Weiwei Yang created YUNIKORN-158: Summary: Admission controller deployment file should use the same version as the scheduler Key: YUNIKORN-158 URL: https://issues.apache.org/jira/browse/YUNIKORN-158 Project: Apache YuniKorn Issue Type: Sub-task Components: shim - kubernetes Reporter: Weiwei Yang Today, the admission controller is deployed as a post-start hook in the scheduler pod, and the template file has hard coded docker image name, [https://github.com/apache/incubator-yunikorn-k8shim/blob/b8a4a01fa1f6149c8617c914a721296a71037736/deployments/admission-controllers/scheduler/templates/server.yaml.template#L33.] We should get this fixed and make sure the admission controller version is always aligned with the scheduler. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-157) Enable github badges for yunikorn-core
Weiwei Yang created YUNIKORN-157: Summary: Enable github badges for yunikorn-core Key: YUNIKORN-157 URL: https://issues.apache.org/jira/browse/YUNIKORN-157 Project: Apache YuniKorn Issue Type: Improvement Reporter: Weiwei Yang Assignee: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Reminder: YuniKorn 中文 community sync up today
Just a reminder, we have a community sync-up meeting for mandarin speakers today on *Wed 9:30pm PST/Thursday 12:30pm CST* Join by: https://github.com/apache/incubator-yunikorn-core/blob/master/docs/community-sync-up.md Hope to meet you on the call!
[jira] [Created] (YUNIKORN-150) Add a link on queue’s detail info page that links to the apps page to show running in this queue
Weiwei Yang created YUNIKORN-150: Summary: Add a link on queue’s detail info page that links to the apps page to show running in this queue Key: YUNIKORN-150 URL: https://issues.apache.org/jira/browse/YUNIKORN-150 Project: Apache YuniKorn Issue Type: Bug Components: webapp Reporter: Weiwei Yang Assignee: Akhil PB A problem we usually have is. When we look at queues, we don't know what are the apps using the queue resources. We could go back to the apps page and find out the apps by going over all apps, but pretty time-consuming. It would be good if we can add a quick link for this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-149) Watch K8s namespace and create unmanaged queues accordingly
Weiwei Yang created YUNIKORN-149: Summary: Watch K8s namespace and create unmanaged queues accordingly Key: YUNIKORN-149 URL: https://issues.apache.org/jira/browse/YUNIKORN-149 Project: Apache YuniKorn Issue Type: Sub-task Components: shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang Watch K8s namespace object and create queues accordingly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-148) Define API in scheduler interface to queue administration
Weiwei Yang created YUNIKORN-148: Summary: Define API in scheduler interface to queue administration Key: YUNIKORN-148 URL: https://issues.apache.org/jira/browse/YUNIKORN-148 Project: Apache YuniKorn Issue Type: Sub-task Components: scheduler-interface Reporter: Weiwei Yang Assignee: Weiwei Yang Expose queue mgmt API from scheduler interface. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-147) Extensive resource quota management
Weiwei Yang created YUNIKORN-147: Summary: Extensive resource quota management Key: YUNIKORN-147 URL: https://issues.apache.org/jira/browse/YUNIKORN-147 Project: Apache YuniKorn Issue Type: New Feature Components: core - cache, core - common, core - scheduler, scheduler-interface, shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang Today, in K8s. The resource quota is enforced by the quota admission controller. Resource quota is charged once a pod is *created* in a namespace, no matter it is running, pending (a pod only consumes resources when it is running), failed, or completed. The admission controller will reject further pods if all quota is exceeded. When we run batch workloads with such quota, this can lead to some issues. Let's use Spark as an example: Spark job pods could be pending for many reasons, volume not ready, picky on a host, etc. Such pending pods will consume the resource quota. And subsequentially causing the resources can not be efficiently used. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Reminder: YuniKorn 中文 community sync up this week
Hi all Just a reminder, we have a community sync-up meeting for mandarin speakers this week on *Wed 9:30pm PST/Thursday 12:30pm CST* Join by: https://github.com/apache/incubator-yunikorn-core/blob/master/docs/community-sync-up.md Hope to meet you on the call! Thanks
[jira] [Created] (YUNIKORN-145) Add travis integration for yunikorn-core repo
Weiwei Yang created YUNIKORN-145: Summary: Add travis integration for yunikorn-core repo Key: YUNIKORN-145 URL: https://issues.apache.org/jira/browse/YUNIKORN-145 Project: Apache YuniKorn Issue Type: Sub-task Components: build Reporter: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-146) Add travis integration for yunikorn-web repo
Weiwei Yang created YUNIKORN-146: Summary: Add travis integration for yunikorn-web repo Key: YUNIKORN-146 URL: https://issues.apache.org/jira/browse/YUNIKORN-146 Project: Apache YuniKorn Issue Type: Sub-task Components: buid Reporter: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-144) Add daily integration tests
Weiwei Yang created YUNIKORN-144: Summary: Add daily integration tests Key: YUNIKORN-144 URL: https://issues.apache.org/jira/browse/YUNIKORN-144 Project: Apache YuniKorn Issue Type: Sub-task Components: shim - kubernetes, test - smoke Reporter: Weiwei Yang add daily tests with travis CI -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-142) Upstream CI/CD pipeline
Weiwei Yang created YUNIKORN-142: Summary: Upstream CI/CD pipeline Key: YUNIKORN-142 URL: https://issues.apache.org/jira/browse/YUNIKORN-142 Project: Apache YuniKorn Issue Type: New Feature Components: build Reporter: Weiwei Yang Umbrella to track all efforts for CI/CD -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-143) Add travis integration for k8shim repo
Weiwei Yang created YUNIKORN-143: Summary: Add travis integration for k8shim repo Key: YUNIKORN-143 URL: https://issues.apache.org/jira/browse/YUNIKORN-143 Project: Apache YuniKorn Issue Type: Sub-task Components: shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang Travis CI integration -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-129) Add license headers for missing files in web
[
https://issues.apache.org/jira/browse/YUNIKORN-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-129.
--
Fix Version/s: 0.9
Resolution: Fixed
> Add license headers for missing files in web
>
>
> Key: YUNIKORN-129
> URL: https://issues.apache.org/jira/browse/YUNIKORN-129
> Project: Apache YuniKorn
> Issue Type: Task
> Components: webapp
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.9
>
>
> Add missing license headers in the core repo
> Update make file and add a check-license target which uses the include list
> that covers at least:
> {code:java}
> --include=*. {sh,md,yaml,yml,js,ts,html,js,scss}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-136) build of image works differently in shim and web
[ https://issues.apache.org/jira/browse/YUNIKORN-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-136. -- Fix Version/s: 0.9 Resolution: Fixed > build of image works differently in shim and web > > > Key: YUNIKORN-136 > URL: https://issues.apache.org/jira/browse/YUNIKORN-136 > Project: Apache YuniKorn > Issue Type: Improvement > Components: build, webapp >Reporter: Wilfred Spiegelenburg >Assignee: Wilfred Spiegelenburg >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > > The webapp uses TAG to set both the repo and the image name. > The shim uses REPOSITORY to just set the repo and always uses the same image > name. > For consistency the web build should do the build setup that is used by the > shim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Reminder: Community meeting is happening 10:00 am PST
Hi all We'll have the community meeting at 10:00 am PST. Please join us if you are interested. Today, we are going to talk about the new 0.8.0 release, and priorities for the upcoming new release. Thanks
Today's community meeting (EN) is postponed to May 7
Hi all A lot of us have conflicts for tomorrow's community meeting. So we decided to postpone it to May 7, 10:00 am - 10:30 am PST. At this meeting, we plan to discuss the 0.8.0 release, and upcoming feature prioritize, hope to see you there. Please subscribe to the calendar if you are interested: https://github.com/apache/incubator-yunikorn-core/blob/master/docs/community-sync-up.md Thanks
[jira] [Created] (YUNIKORN-134) Post 0.8.0 release - update roadmap
Weiwei Yang created YUNIKORN-134: Summary: Post 0.8.0 release - update roadmap Key: YUNIKORN-134 URL: https://issues.apache.org/jira/browse/YUNIKORN-134 Project: Apache YuniKorn Issue Type: Improvement Components: documentation Reporter: Weiwei Yang Assignee: Weiwei Yang Update the roadmap accordingly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-133) Performance improvement: optimize predicate function performance.
Weiwei Yang created YUNIKORN-133: Summary: Performance improvement: optimize predicate function performance. Key: YUNIKORN-133 URL: https://issues.apache.org/jira/browse/YUNIKORN-133 Project: Apache YuniKorn Issue Type: Improvement Components: shim - kubernetes Reporter: Weiwei Yang During the scalability test, we found the predicate functions are quite time-consuming. We need to investigate how we can optimize this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-132) Publish helm charts for convenient access
Weiwei Yang created YUNIKORN-132: Summary: Publish helm charts for convenient access Key: YUNIKORN-132 URL: https://issues.apache.org/jira/browse/YUNIKORN-132 Project: Apache YuniKorn Issue Type: New Feature Components: shim - kubernetes Reporter: Weiwei Yang It would be good if we can publish our helm charts to [https://hub.helm.sh/]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-131) Prometheus integration - phase 2
Weiwei Yang created YUNIKORN-131: Summary: Prometheus integration - phase 2 Key: YUNIKORN-131 URL: https://issues.apache.org/jira/browse/YUNIKORN-131 Project: Apache YuniKorn Issue Type: New Feature Components: metrics Reporter: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-121) The admission controller should use hostNetwork
[ https://issues.apache.org/jira/browse/YUNIKORN-121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-121. -- Resolution: Fixed > The admission controller should use hostNetwork > --- > > Key: YUNIKORN-121 > URL: https://issues.apache.org/jira/browse/YUNIKORN-121 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Labels: pull-request-available > > When running on EKS, with custom network plugin enabled (e.g calico). Without > using hostNetwork, the API server will not be able to connect to the webhook. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-121) The admission controller should use hostNetwork
Weiwei Yang created YUNIKORN-121: Summary: The admission controller should use hostNetwork Key: YUNIKORN-121 URL: https://issues.apache.org/jira/browse/YUNIKORN-121 Project: Apache YuniKorn Issue Type: Improvement Components: shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang When running on EKS, with custom network plugin enabled (e.g calico). Without using hostNetwork, the API server will not be able to connect to the webhook. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-116) Typo in the rest api to get nodes info
[
https://issues.apache.org/jira/browse/YUNIKORN-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-116.
--
Fix Version/s: 0.9
Resolution: Fixed
Hi [~kmarton] Thanks for the quick fix!
> Typo in the rest api to get nodes info
> --
>
> Key: YUNIKORN-116
> URL: https://issues.apache.org/jira/browse/YUNIKORN-116
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: webapp
> Reporter: Weiwei Yang
>Assignee: Kinga Marton
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 0.9
>
>
> There is a typo in the output of the nodes API:
> {code}
> [
> {
> "partitionName": "[mycluster]default",
> "nodesInfo": [
> {
> "nodeID": "ip-10-116-72-66.us-west-2.compute.internal",
> "hostName": "ip-10-116-72-66.us-west-2.compute.internal",
> "*RackName*": "/rack-default",
> "capacity": "[attachable-volumes-aws-ebs:25
> ephemeral-storage:94477937300 hugepages-1Gi:0 hugepages-2Mi:0 memory:7463
> pods:29 vcore:1900]",
> "allocated": "[memory:3958 vcore:1600]",
> "occupied": "[vcore:110]",
> "available": "[attachable-volumes-aws-ebs:25
> ephemeral-storage:94477937300 hugepages-1Gi:0 hugepages-2Mi:0 memory:3505
> pods:29 vcore:190]",
> "allocations": [
> ...
> {code}
> RackName -> rackName
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
Today's community sync up (中文) has been canceled.
Hi, The Apache YuniKorn community is busy working on 0.8.0 release, this week's sync meeting will be canceled. Let's meet 2 weeks later. Hope to see you there!
[RESULT] [VOTE] Release Apache YuniKorn (Incubating) 0.8.0
Hi all The vote to release Apache YuniKorn (Incubating) 0.8.0 has passed. We got 6 binding votes - Felix Cheung - Sunil Govindan - Weiwei Yang - Jason Lowe - Wilfred Spiegelenburg - Tao Yang and 1 non-binding vote - Adam Antal Vote thread: https://lists.apache.org/thread.html/rcd434d0c35e53e162726f22e10b0972dbd633949a49c2bb3577cdf20%40%3Cdev.yunikorn.apache.org%3E Thank you for all the above members to help us to verify and vote for the 0.8.0 release. We will move to IPMC voting shortly. Thanks
Re: [VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc4
Hi all Thank you. 72h passed and we got 6 binding and 1 non-binding +1. The voting has passed. I will send a result notice shortly and then create a thread for IPMC vote. Weiwei On Tue, Apr 28, 2020 at 2:31 AM Adam Antal wrote: > Weiwei, thanks for driving the release. > > +1 (non-binding) > > - Verified GPG signature > - Verified SHA512 hash > - Core, shim and web build and UT suite passes > - Interface's build passes > - Built Docker images successfully with ./build-docker-images.sh > > One minor issue that I found is that build-docker-images.sh script is not > executable in the archive: > > apache-yunikorn-0.8.0-incubating-src adamantal$ ls -la > build-docker-images.sh > -rw-r--r--@ 1 adamantal staff 2052 Apr 27 02:19 build-docker-images.sh > > I don't think that should be fixed in a new RC, but we can chmod +x the > script in some later releases. > > Thanks, > Adam > > On Tue, Apr 28, 2020 at 6:00 AM Tao Yang wrote: > > > Thanks weiwei for driving this release. > > > > +1 binding. > > > > - check checksum & signature > > - build images from source > > - install with helm charts > > - run some examples > > > > Thanks, > > Tao > > > > Wilfred Spiegelenburg 于2020年4月28日周二 上午11:23写道： > > > > > Weiwei, > > > > > > And a +1 for me also: > > > - signature > > > - checksum > > > - content (DISCLAIMER, NOTICE) is good > > > - Build of the code passes > > > > > > To add to the points Felix has made: point 6 the website update is in > > > progress. All pages will have the correct logo etc as part of the > footer > > > added. > > > > > > Wilfred > > > > > > On Tue, 28 Apr 2020 at 07:30, Jason Lowe wrote: > > > > > > > Thanks for putting this release together, Weiwei! > > > > > > > > +1 > > > > > > > > - verified release file meets incubator naming requirements > > > > - verified ASF LICENSE and NOTICE files > > > > - verified signature and digest > > > > - verified disclaimer meets incubator requirements > > > > - verified source repository tag matches release artifact contents > > > > > > > > Jason > > > > > > > > On Sun, Apr 26, 2020 at 7:57 PM Weiwei Yang wrote: > > > > > > > > > Hi all > > > > > > > > > > I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc4. > > > > > > > > > > All release artifacts, including source code package, signatures, > and > > > > > checksum, etc, can be found at: > > > > > > http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc4/ > > > > > > > > > > this release has been signed with PGP > > > > > key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to > > > > > w...@apache.org. You can find the KEYS file here: > > > > > http://people.apache.org/~wwei/KEYS. > > > > > > > > > > The release has been tagged with "v0.8.0" in all our git repos. > > > > > The JIRA issues have been resolved for this release can be found: > > > > > https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742 > . > > > > > > > > > > Please review and vote. Note, the only diff between RC4 and RC3 is > > the > > > > > DISCLAIMER file is replaced with DISCLAIMER-WIP. Past voting > threads: > > > RC1 > > > > > < > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/r22d8784b09ceead38f470534cae41e503c796b47144f62994e8a2115%40%3Cdev.yunikorn.apache.org%3E > > > > > >, > > > > > RC2 > > > > > < > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/rec0286566f4169563408c698c2ac02cdb4db0f4c587ac3a375a042ee%40%3Cdev.yunikorn.apache.org%3E > > > > > >, > > > > > RC3 > > > > > < > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/r09317bd6af041a9aa49fe1a23eb5a355e3a6768afc82e0bdbb06fea3%40%3Cdev.yunikorn.apache.org%3E > > > > > > > > > > > > > > > > [ ] +1 approve > > > > > [ ] +0 no opinion > > > > > [ ] -1 disapprove (and the reason why) > > > > > > > > > > > > > > >
Re: [ANNOUNCE] Updated web site deployed
Looks nice, thank you!! Weiwei On Tue, Apr 28, 2020 at 11:29 PM Sunil Govindan wrote: > Thanks Wilfred!! Appreciate it > > On Wed, Apr 29, 2020 at 10:34 AM Wilfred Spiegelenburg < > wilfr...@apache.org> > wrote: > > > Hi all, > > > > In preparation of the release and the requirement around site branding I > > have deployed an updated site. > > The changes include: > > - branding update > > - preparation of release download page > > - removal of unneeded files and references in the source > > - removal of hidden and unreferenced files in the deployement > > - updated build instructions > > > > The site is managed in the git repo: > > https://github.com/apache/incubator-yunikorn-site > > > > If you see anything incorrect or missing open a jira and add the label > > *website* to it. > > > > Thanks, > > Wilfred > > >
Re: [VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc4
Please ignore this and do *NOT* reply to this thread. The link was wrong, I will send another one shortly. Sorry for the confusion!! On Sun, Apr 26, 2020 at 5:47 PM Weiwei Yang wrote: > Hi all > > I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc4. > > All release artifacts, including source code package, signatures, and > checksum, etc, can be found at: > http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc3/ > > this release has been signed with PGP > key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to > w...@apache.org. You can find the KEYS file here: > http://people.apache.org/~wwei/KEYS. > > The release has been tagged with "*v0.8.0*" in all our git repos. > The JIRA issues have been resolved for this release can be found: > https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. > > Please review and vote. Note, the only diff between RC4 and RC3 is the > DISCLAIMER file is replaced with DISCLAIMER-WIP. > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and the reason why) >
[jira] [Created] (YUNIKORN-112) Fix release issues
Weiwei Yang created YUNIKORN-112: Summary: Fix release issues Key: YUNIKORN-112 URL: https://issues.apache.org/jira/browse/YUNIKORN-112 Project: Apache YuniKorn Issue Type: Bug Reporter: Weiwei Yang Felix gives us some feedback on our first release, we should make sure they are fixed in the next release, items include: - incubating in name - signature and hash fine - DISCLAIMER is ok - as above - LICENSE and NOTICE are fine - Ok with binary files - see note below - See note below on ASF headers - didn't compile from source 1. SHOULD: instead of a personal directory, release SHOULD be staged on [https://dist.apache.org/repos/dist/dev/incubator/yunikorn]. [https://incubator.apache.org/guides/releasemanagement.html#podling_constraints] this include the signing KEYS file that should be kept as a project KEYS file instead of a personal one/location 2. it is a bit strange that you have 5 LICENSE files, you can exclude the one in subdirectories when you build the source tarball? 3. might want to check the license for the image files: png, jpg - where are they from? 4. SHOULD: a large number of files should also have ASF headers, eg. .mod, .yaml, .md, .go 5. nice to have: more details on go setup for dev environment (maybe just a quick link), go setup info without GoLand IDE. It could be helpful to know more on how to build (is this the build step? [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/developer-guide.md#core-component-build]) 6. SHOULD: put Apache Incubator logo on the websitehttps://[incubator.apache.org/guides/press-kit.html|http://incubator.apache.org/guides/press-kit.html] [http://yunikorn.apache.org/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-100) FIFO scheduling prototype
[ https://issues.apache.org/jira/browse/YUNIKORN-100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-100. -- Fix Version/s: 0.9 Resolution: Fixed > FIFO scheduling prototype > - > > Key: YUNIKORN-100 > URL: https://issues.apache.org/jira/browse/YUNIKORN-100 > Project: Apache YuniKorn > Issue Type: Sub-task > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Re: [VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc3
Just a correction: The voting ends on *Sunday Monday** Apr 26, 2020, 20:00 PST.* Weiwei On Fri, Apr 24, 2020 at 6:29 AM Sunil Govindan wrote: > +1 (binding) > > - checksum & signature is fine. > - Basic build is fine > - ran sample spark job, all looks good. > > Thanks > Sunil > > On Fri, Apr 24, 2020 at 6:55 AM Weiwei Yang wrote: > > > Hi all > > > > I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc3. > (Note, > > the only diff between RC3 and RC2 is the "-rc2" suffix has been removed > > from the tarball/folder name) > > > > All release artifacts, including source code package, signatures, and > > checksum, etc, can be found at: > > http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc3/ > > > > this release has been signed with PGP > > key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to > > w...@apache.org. You can find the KEYS file here: > > http://people.apache.org/~wwei/KEYS. > > > > The release has been tagged with "*v0.8.0*" in all our git repos. > > The JIRA issues have been resolved for this release can be found: > > https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. > > > > Please review and vote. The vote will be open for at least 72 hours (that > > ends on *Monday, Apr 26, 2020, 20:00 PST*). > > > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove (and the reason why) > > >
[VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc3
Hi all I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc3. (Note, the only diff between RC3 and RC2 is the "-rc2" suffix has been removed from the tarball/folder name) All release artifacts, including source code package, signatures, and checksum, etc, can be found at: http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc3/ this release has been signed with PGP key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to w...@apache.org. You can find the KEYS file here: http://people.apache.org/~wwei/KEYS. The release has been tagged with "*v0.8.0*" in all our git repos. The JIRA issues have been resolved for this release can be found: https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. Please review and vote. The vote will be open for at least 72 hours (that ends on *Monday, Apr 26, 2020, 20:00 PST*). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason why)
Re: [VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc2
Hi all Thanks for voting for RC2, unfortunately, due to the issue Sunil has raised, we have to roll out RC3 to get this fixed. I will send another voting thread shortly. Sorry for the inconvenience. Thanks Weiwei On Thu, Apr 23, 2020 at 8:41 AM Sunil Govindan wrote: > Thanks Wilfred for clarifying this. > > +1 from side. > - checksum & signature is fine. > - Basic build is fine > - ran sample spark job, all looks good. > > Thanks +1 (binding) > > Sunil > > On Thu, Apr 23, 2020 at 7:31 PM Wilfred Spiegelenburg > > wrote: > > > Hi Weiwei, > > > > checked: > > - signature > > - checksum > > - content (DISCLAIMER, NOTICE) is good > > - The instructions in README.md should be extended to contain more > details > > on how to build from source. I think we can leave that for the next > steps. > > - Build of the code passes > > > > +1 (binding) for RC2 > > > > @Sunil Govindan : we can rename the files without > > needing to re-generate the checksum, signature or content of the file > > before we push them to the site for distribution. This is the last step > and > > is part of the normal process we do not have to change this at this > point. > > What we do now is similar to other Apache projects. > > > > Wilfred > > > > On Wed, 22 Apr 2020 at 16:57, Weiwei Yang wrote: > > > >> Hi all > >> > >> I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc2. > >> All release artifacts, including source code package, signatures, and > >> checksum, etc, can be found at: > >> http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc2/ > >> > >> this release has been signed with PGP > >> key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to > >> w...@apache.org. You can find the KEYS file here: > >> http://people.apache.org/~wwei/KEYS. > >> > >> The release has been tagged with "*v0.8.0*" in all our git repos. > >> The JIRA issues have been resolved for this release can be found: > >> https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. > >> > >> Please review and vote. The vote will be open for at least 72 hours > (that > >> ends on *Monday, Apr 25, 2020, 17:00 PST*). > >> > >> [ ] +1 approve > >> [ ] +0 no opinion > >> [ ] -1 disapprove (and the reason why) > >> > > >
[jira] [Resolved] (YUNIKORN-106) Typo in si Makefile
[ https://issues.apache.org/jira/browse/YUNIKORN-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-106. -- Fix Version/s: 0.9 Resolution: Fixed > Typo in si Makefile > --- > > Key: YUNIKORN-106 > URL: https://issues.apache.org/jira/browse/YUNIKORN-106 > Project: Apache YuniKorn > Issue Type: Bug > Components: scheduler-interface >Reporter: Wanqiang Ji >Assignee: Wanqiang Ji >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > > In the scheduler-interface we use the make build tool to generate the > si.proto, and add the header to warning the developer don't edit directly. > Such as: _This code is generated by the buikd process._ So I think the > *buikd* is typo. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-65) Remove job references from the web app
[ https://issues.apache.org/jira/browse/YUNIKORN-65?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-65. - Fix Version/s: 0.9 Resolution: Fixed > Remove job references from the web app > -- > > Key: YUNIKORN-65 > URL: https://issues.apache.org/jira/browse/YUNIKORN-65 > Project: Apache YuniKorn > Issue Type: Improvement > Components: webapp >Reporter: Wilfred Spiegelenburg >Assignee: Akhil PB >Priority: Minor > Labels: pull-request-available > Fix For: 0.9 > > > In the shim and core we have moved away from job and are using application as > a generic term to talk about sets of related allocations. > The web code still uses job everywhere for internal objects. We should change > to application in the webapp to get consistency. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-86) Set proper resource request and limit for YuniKorn pods
[ https://issues.apache.org/jira/browse/YUNIKORN-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-86. - Fix Version/s: 0.9 Resolution: Fixed > Set proper resource request and limit for YuniKorn pods > --- > > Key: YUNIKORN-86 > URL: https://issues.apache.org/jira/browse/YUNIKORN-86 > Project: Apache YuniKorn > Issue Type: Task > Components: core - scheduler >Reporter: Adam Antal >Assignee: Adam Antal >Priority: Critical > Labels: pull-request-available > Fix For: 0.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Right now we don't specify the resource request and limit for neither the > scheduler container nor the web container. We should define it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc2
Hi all I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc2. All release artifacts, including source code package, signatures, and checksum, etc, can be found at: http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc2/ this release has been signed with PGP key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to w...@apache.org. You can find the KEYS file here: http://people.apache.org/~wwei/KEYS. The release has been tagged with "*v0.8.0*" in all our git repos. The JIRA issues have been resolved for this release can be found: https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. Please review and vote. The vote will be open for at least 72 hours (that ends on *Monday, Apr 25, 2020, 17:00 PST*). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and the reason why)
Re: Resource request and limit for YuniKorn pods
Hi Adam Thanks for the investigation. These data are useful, I think what you suggested looks good to me. - I think it makes sense to set a relatively small number, this is to ensure we are not getting into trouble on small envs. But for memory, can we request for 1G at least? Memory cannot be throttled, 1G is safer. - For the CPU, it looks like we probably won't go beyond 4~5 CPUs. So setting a limit to 4 makes sense to me. I also checked an EKS cluster, I saw usage around 1 CPU there. So I think we can do resources: requests: cpu: 200m memory: *1Gi* limits: cpu: 4 memory: 2Gi @taoy...@apache.org could you help to review this? Let us know if this makes sense to you. Weiwei On Mon, Apr 20, 2020 at 6:16 AM Adam Antal wrote: > Hi, > > I am working on [YUNIKORN-86] (Set proper resource request and limit for > YuniKorn pods) issues and I would like to hear your feedback on this issue. > Link:https://issues.apache.org/jira/browse/YUNIKORN-86 > > Currently I am working on finding a justified request and limit value for > the scheduler and shim pods in k8s. So far: > - Researched a bit on the k8s default scheduler (kube-scheduler) and even > though it can be configured, that are no defaults for that pod. On AKS > there is a default that I could find on the internet: > >>> > resources: > requests: > cpu: 100m > memory: 128Mi > limits: > cpu: 4 > memory: 2Gi > >>> > - Tried to obtain some values from a deployed k8s cluster, but had troubles > with the metrics-server (in some old version there was heapster deployed) > - Ran the yunikorn-core/pkg/scheduler/tests/scheduler_perf_tests.go and > monitored the cpu/memory consumption. I don't have the run's full graph, > but these are the approximate values that I experienced >- for most of the time cpu was on around 4 cores >- on peak time it was around 5.2 cores >- the memory consumption was moderate, it was around 4.3% of the 16Gb > memory of my laptop - which is around 700Mb when the perf test was ended. > The memory depends on the stored/currently running applications, so I think > 1Gb should be fine for this for general purpose. > Note that the cpu numbers are related to the peak usage, when the scheduler > was under pressure, but when only a small amount of pods are > scheduled, this is a much smaller number. > > I suggest the following (similar to AKS) number: > >>> > resources: > requests: > cpu: 200m > memory: 512Mi > limits: > cpu: 4 > memory: 2Gi > >>> > > One additional thing: there were some measurements of YuniKorn in kubemark. > Could you please share your outputs if there's something related to the > scheduler and the shim pod's resource usage? > > Regards, > Adam >
Re: [VOTE] Release Apache YuniKorn (incubating) 0.8.0-rc1
-interface, web, core, k8shim > 3. License > - yunikorn-web Dockerfile is missing ASF license header > > Thanks, > Sunil > > On Mon, Apr 20, 2020 at 7:39 PM Julia Kinga Marton > wrote: > > > Hi Weiwei, > > > > I did some testing on the RC as well, below you can find my notes: > > Tested the following things: > > - verified sha512 sum: OK > > - verified signature: OK > > - built docker images > > - installed Yunikorn both with Helm chart and manually > > > > *During the testing I found the following issues:* > > - the helm chart link is wrong > > - in the README.md *Run YuniKorn on an existing K8s cluster *I would > extend > > it with changing the image pull policy as well (It is very easy to forget > > about it and keep it as latest). > > - the documentation about how to uninstall it is not complete, because it > > describes a one step command, however there are further steps we had to > > perform in order to have it completely uninstalled (scale the cluster to > 0 > > if we want to uninstall the admission controller as well, than we have to > > delete the configmap manually). Opened YUNIKORN-102 > > <https://issues.apache.org/jira/browse/YUNIKORN-102> for fixing it. > > - In the webUI the pending and terminated pods are show as Running > > applications. I think this may cause misunderstandings in the future. > > Opened YUNIKORN-103 <https://issues.apache.org/jira/browse/YUNIKORN-103> > > for > > discussing and fixing this gap. > > - In case of manual installation using scheduler.yaml > > < > > > https://github.com/apache/incubator-yunikorn-k8shim/blob/master/deployments/scheduler/scheduler.yaml > > >descriptor > > the web UI is not working properly and the admission controller > > installation is missing the yunikorn-service as well. Opened YUNIKORN-104 > > <https://issues.apache.org/jira/browse/YUNIKORN-104> for fixing the > > deployment descriptor. > > > > Regards, > > Kinga > > > > > > On Mon, Apr 20, 2020 at 6:48 AM Wilfred Spiegelenburg < > wilfr...@apache.org > > > > > wrote: > > > > > Hi Weiwei, > > > > > > Thank you for the first RC for YuniKorn. > > > > > > There are some issues with the release which means that I have to give > a > > -1 > > > at this point. > > > > > > 1) NOTICE file contains a Copyright 2018-2020 message. The first commit > > is > > > from 2019 so that date is not correct.The code was donated to Apache in > > > 2020. > > > 2) DISCLAIMER text is not correct. The project name should contain > > > incubating, i.e. Apache Yunikorn (Incubating). > > > The text "name of Apache TLP sponsor." should be replaced by the text > > > " Apache Incubator PMC." > > > 3) README.md does not state the fact that there is a pre-requisite of > > > having a full build environment with tools installed. We need a link to > > the > > > build doc at least to show where the information can be found. There > are > > > some small grammar issues also but those are minor. > > > 4) The helm-charts directory in the root of the package points to a non > > > existing location. The link that was created is wrong. > > > 5) The tag that has been used is v0.8.0-incubating-rc1. That reference > is > > > in the go.mod files. That means we cannot use this source release and > > > promote it to the real release. We cannot have a pointer to the rc1 tag > > in > > > the files. > > > > > > I think we need to stop the vote and create a new RC to fix these > issues. > > > > > > Wilfred > > > > > > On Sat, 18 Apr 2020 at 06:43, Weiwei Yang wrote: > > > > > > > Hi all > > > > > > > > I'd like to call a vote for Apache YuniKorn (incubating) 0.8.0-rc1. > > > > All release artifacts, including source code package, signatures, and > > > > checksum, etc, can be found at: > > > > http://people.apache.org/~wwei/apache-yunikorn-incubating-0.8.0-rc1/ > > > > > > > > this release has been signed with PGP > > > > key 8D076B6491A66D7B94E94519F57176CE11856D1F, corresponding to > > > > w...@apache.org. You can find the KEYS file here: > > > > http://people.apache.org/~wwei/KEYS. > > > > > > > > The release has been tagged with "*v0.8.0-incubating-rc1*" in all our > > git > > > > repos. > > > > The JIRA issues have been resolved for this release can be found: > > > > https://issues.apache.org/jira/projects/YUNIKORN/versions/12347742. > > > > > > > > Please review and vote. The vote will be open for at least 72 hours > > (that > > > > ends on *Monday, Apr 20, 2020, 20:00 PST*). > > > > > > > > [ ] +1 approve > > > > [ ] +0 no opinion > > > > [ ] -1 disapprove (and the reason why) > > > > > > > > Thanks > > > > Weiwei > > > > > > > > > >
[jira] [Created] (YUNIKORN-105) yunikorn-web dockerfile misses license file
Weiwei Yang created YUNIKORN-105: Summary: yunikorn-web dockerfile misses license file Key: YUNIKORN-105 URL: https://issues.apache.org/jira/browse/YUNIKORN-105 Project: Apache YuniKorn Issue Type: Bug Components: webapp Reporter: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-36) Cleanup leaked resources when deleting the helm chart
[ https://issues.apache.org/jira/browse/YUNIKORN-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-36. - Fix Version/s: 0.9 Resolution: Fixed > Cleanup leaked resources when deleting the helm chart > - > > Key: YUNIKORN-36 > URL: https://issues.apache.org/jira/browse/YUNIKORN-36 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.9 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Several resources may be leaked after this helm chart is deleted(uninstalled): > - admission-controller > deployment/service/mutatingwebhook/validatingwebhook/secret. The reason is > that admission-controller resources are deleted via preStop hook of > yunikorn-scheduler pod, the ServiceAccount and ClusterRoleBinding resources > are probably already deleted then. > - yunikorn-configs ConfigMap. This one is configured as pre-install hook in > helm chart, it won't be deleted when uninstalling. > All resources are expected to be cleaned up after the helm chart is deleted, > to achieve that, this issue will take the ServiceAccount and > ClusterRoleBinding resources as pre-install hooks and create a cleanup pod to > release ConfigMap/ClusterRoleBinding/ServiceAccount resources as prometheus > operator did (refer to > https://github.com/helm/charts/blob/d58247103fafd5930792bb1fd3ac240547523200/stable/prometheus-operator/templates/prometheus-operator/cleanup-crds.yaml#L33). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-100) FIFO scheduling prototype
Weiwei Yang created YUNIKORN-100: Summary: FIFO scheduling prototype Key: YUNIKORN-100 URL: https://issues.apache.org/jira/browse/YUNIKORN-100 Project: Apache YuniKorn Issue Type: Sub-task Reporter: Weiwei Yang Assignee: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-97) Embed admission-controller should set to false by default
Weiwei Yang created YUNIKORN-97: --- Summary: Embed admission-controller should set to false by default Key: YUNIKORN-97 URL: https://issues.apache.org/jira/browse/YUNIKORN-97 Project: Apache YuniKorn Issue Type: Task Components: shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang It will be better to use false value here, in case people just want to try out yunikorn as a secondary scheduler. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Re: YuniKorn 0.8 release updates
Hi Adam It completely makes sense. Would you like to take a look at what we can do with github actions? We already have set up actions in yunikorn-core and yunikorn-k8shim to execute UTs, we need that to add web and interface repos too; also it will be good to know what it can help with the release, e.g automatically build source code tarball for us. Weiwei On Thu, Apr 16, 2020 at 5:39 AM Adam Antal wrote: > Hi everyone, > > Just my two cents here... > > I think the tendency nowadays is to add these tools to build the code to > the repository itself. See Gitlab yamls or Github action file that can > declare the compile+test phase for the CI. > > Gitlab docs perfectly sums up why I think it can be a better choice to > include it next to the code: > > > Because .gitlab-ci.yml is in the repository and is version controlled, > old > > versions still build successfully, forks can easily make use of CI, > > branches can have different pipelines and jobs, and you have a single > > source of truth for CI. You can read more about the reasons why we are > > using .gitlab-ci.yml in our blog about it > > < > https://about.gitlab.com/blog/2015/05/06/why-were-replacing-gitlab-ci-jobs-with-gitlab-ci-dot-yml/ > > > > . > > > Also I recommend reading the article. > > Regards, > Adam > > On Wed, Apr 15, 2020 at 11:59 PM Weiwei Yang wrote: > > > Hi Wilfred/Sunil > > > > Thanks for the feedback. > > For a multi-repo project like YuniKorn, I think it makes sense to have a > > separate one to manage them. > > Put release things (script, doc, files) in any of the source code repo > is a > > bit weird to me. I have already created > > https://github.com/apache/incubator-yunikorn-release/. *which is for the > > release managers only.* This can avoid further confusion and also help us > > to do future releases in minutes. > > The docker file approach, which provides a build env without pollute > > people's local env, this might be helpful in certain cases. Such as when > > you need to install a lot of binaries to make a build. For YuniKorn, > since > > we just rely on Go for the build env, I don't see how useful it is at > > present. I am open to adopting that approach as long as it helps. > > > > Weiwei > > > > > > On Wed, Apr 15, 2020 at 8:06 AM Sunil Govindan > wrote: > > > > > Thanks Weiwei and Wilfred. > > > > > > +0 on this approach. I am overall feeling fine with the approach > > suggested. > > > However I may have personally prefer to do this in a create-release.sh > > and > > > Dockerfile like hadoop model. > > > > > > As Weiwei pointed out, its used in Opwnwhisk pjct as well, may be we > can > > > try this out and find the positiveness. > > > > > > I think, we should make the scripts or configs in this repo more > dynamic > > in > > > nature agnostic to any release version. > > > RM can supply this version etc as a run time param to these scripts. > > > > > > Thanks > > > Sunil > > > > > > On Tue, Apr 14, 2020 at 12:26 PM Weiwei Yang > > wrote: > > > > > > > Hi Wilfred > > > > > > > > Thanks for the feedback. > > > > > > > > The reason to introduce a new repo is to have some tools/docs easier > > for > > > > release managers. I found this is a good practice when you have > > multiple > > > > repos. Please see Apache OpenWhisk as an example. > > > > > > > > If you look at the config file I shared, that helps to track source > > code > > > > revision of different repos, which is easy to assemble a release. > > > > > > > > I propose to do unified release, which means we alway release all > repos > > > > together. This is simpler to manage. > > > > > > > > If you look at the archive file, I should have put a script and a > > README > > > > about how to build docker images. Can you please take a look at that? > > If > > > > that is not there, pls see the script on github. > > > > > > > > Thanks for verifying the content. Could you please let me know what > > files > > > > are wrong? > > > > > > > > > > > > > > > > -- > > > > Weiwei > > > > On Apr 13, 2020, 11:30 PM -0700, Wilfred Spiegelenburg < > > > > wilfr...@apache.org>, wrote: > > > > > Hi Weiwei, > > > > > > > > > > Do
[jira] [Resolved] (YUNIKORN-95) YuniKorn does not emit FailedScheduling event to k8s if pod uses non-existing PVC
[
https://issues.apache.org/jira/browse/YUNIKORN-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-95.
-
Fix Version/s: 0.9
Resolution: Fixed
> YuniKorn does not emit FailedScheduling event to k8s if pod uses non-existing
> PVC
> -
>
> Key: YUNIKORN-95
> URL: https://issues.apache.org/jira/browse/YUNIKORN-95
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - yarn
>Reporter: Adam Antal
>Assignee: Adam Antal
>Priority: Critical
> Labels: pull-request-available
> Fix For: 0.9
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If a pod uses a non-existing PVC, the default-scheduler emit the following
> event to Kubernetes:
> {noformat}
> Events:
> Type ReasonAge From Message
> -- ---
> Warning FailedScheduling 45s default-scheduler persistentvolumeclaim
> "ebs-claim-auto" not found
> {noformat}
> YuniKorn does not emit such event, we only know what has happened by checking
> the logs:
> {noformat}
> 2020-04-10T00:09:06.620Z DEBUG cache/task.go:404 checking PVC
> {"name": "non-existing-pvc"}
> 2020-04-10T00:09:06.620Z DEBUG cache/application.go:277task is
> not ready for scheduling{"appID": "pod-with-non-existing-pvc",
> "taskID": "5c3961d1-7abf-11ea-b27d-02ea8179ac42", "error":
> "persistentvolumeclaim \"non-existing-pvc\" not found"}
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
Re: YuniKorn 0.8 release updates
Hi Wilfred/Sunil Thanks for the feedback. For a multi-repo project like YuniKorn, I think it makes sense to have a separate one to manage them. Put release things (script, doc, files) in any of the source code repo is a bit weird to me. I have already created https://github.com/apache/incubator-yunikorn-release/. *which is for the release managers only.* This can avoid further confusion and also help us to do future releases in minutes. The docker file approach, which provides a build env without pollute people's local env, this might be helpful in certain cases. Such as when you need to install a lot of binaries to make a build. For YuniKorn, since we just rely on Go for the build env, I don't see how useful it is at present. I am open to adopting that approach as long as it helps. Weiwei On Wed, Apr 15, 2020 at 8:06 AM Sunil Govindan wrote: > Thanks Weiwei and Wilfred. > > +0 on this approach. I am overall feeling fine with the approach suggested. > However I may have personally prefer to do this in a create-release.sh and > Dockerfile like hadoop model. > > As Weiwei pointed out, its used in Opwnwhisk pjct as well, may be we can > try this out and find the positiveness. > > I think, we should make the scripts or configs in this repo more dynamic in > nature agnostic to any release version. > RM can supply this version etc as a run time param to these scripts. > > Thanks > Sunil > > On Tue, Apr 14, 2020 at 12:26 PM Weiwei Yang wrote: > > > Hi Wilfred > > > > Thanks for the feedback. > > > > The reason to introduce a new repo is to have some tools/docs easier for > > release managers. I found this is a good practice when you have multiple > > repos. Please see Apache OpenWhisk as an example. > > > > If you look at the config file I shared, that helps to track source code > > revision of different repos, which is easy to assemble a release. > > > > I propose to do unified release, which means we alway release all repos > > together. This is simpler to manage. > > > > If you look at the archive file, I should have put a script and a README > > about how to build docker images. Can you please take a look at that? If > > that is not there, pls see the script on github. > > > > Thanks for verifying the content. Could you please let me know what files > > are wrong? > > > > > > > > -- > > Weiwei > > On Apr 13, 2020, 11:30 PM -0700, Wilfred Spiegelenburg < > > wilfr...@apache.org>, wrote: > > > Hi Weiwei, > > > > > > Do we really need another repo for three files? > > > We should sleep track of this in the core repo not in another repo > which > > we > > > need to release manage again. I think managing the release from the > core > > > repo will make it easier later on if we need to or want to change the > > build > > > process further. Now we need need to manage and track soo many repos > that > > > it becomes more and more difficult. > > > We also need to keep in mind that version information is in the module > > > files. There might thus be more that needs to change for a release. The > > > other thing is that we might not want to release a new version of one > of > > > the components while updating another component. That would means that > we > > > need to release manage 5 repositories for one release, including all > the > > > overhead. > > > > > > Apache releases are source releases. We still need to provide some kind > > of > > > make etc over the source code also. I agree with the fact that we need > to > > > provide one source archive that is signed. However with the current > build > > > process just the k8shim code is enough to build the docker image. The > > other > > > code repos will be pulled in from github. The mod file point there for > > all > > > go dependencies including the core and SI. It does not provide any > detail > > > on the how and what for any of the repos. We need to provide some build > > > instructions in the root of the source archive. I would not know how to > > > build from the source package if we just add the checked out code into > > it. > > > We need to provide some steps even if they are just pointers to > existing > > > docs. > > > I also don't think we have the correct files in the archive with the > > > current generated archive. > > > > > > Wilfred > > > > > > > > > > On Mon, 13 Apr 2020 at 14:01, Weiwei Yang wrote: > > > > > > > For 0.8 release, I did s
Re: Improve node sorting algorithm
Hi Tao Thank you! I will take a look as well. This is an important feature, having the flexibility to let users configure sorting policies based on different requirements is useful. Thanks for starting this work. On Wed, Apr 15, 2020 at 1:11 AM Sunil Govindan wrote: > Thanks Tao for the efforts. Appreciate it. > I will help to take a look on this. > > Thanks > Sunil > > On Wed, Apr 15, 2020 at 12:09 PM Tao Yang wrote: > > > Hello everyone > > > > I have uploaded a design doc to improve node sorting algorithm in > > https://issues.apache.org/jira/browse/YUNIKORN-21, we are trying to make > > node sorting algorithm more flexible and pluggable, have better > performance > > and can support preferred nodes requirements for specific requests. > > If you are interested on this, we are looking forward to your comments > and > > suggestions. > > > > Thanks, > > Tao Yang > > >
[jira] [Created] (YUNIKORN-92) YuniKorn web site should have incubator logo
Weiwei Yang created YUNIKORN-92: --- Summary: YuniKorn web site should have incubator logo Key: YUNIKORN-92 URL: https://issues.apache.org/jira/browse/YUNIKORN-92 Project: Apache YuniKorn Issue Type: Bug Components: website Reporter: Weiwei Yang Fix the logo in yunikorn web-site -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-91) Create yunikorn-release git repo for release management
[ https://issues.apache.org/jira/browse/YUNIKORN-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-91. - Resolution: Fixed [https://github.com/apache/incubator-yunikorn-release] created > Create yunikorn-release git repo for release management > --- > > Key: YUNIKORN-91 > URL: https://issues.apache.org/jira/browse/YUNIKORN-91 > Project: Apache YuniKorn > Issue Type: Task > Components: release > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Fix For: 0.8 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-91) Create yunikorn-release git repo for release management
Weiwei Yang created YUNIKORN-91: --- Summary: Create yunikorn-release git repo for release management Key: YUNIKORN-91 URL: https://issues.apache.org/jira/browse/YUNIKORN-91 Project: Apache YuniKorn Issue Type: Bug Components: release Reporter: Weiwei Yang -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
YuniKorn 0.8 release updates
For 0.8 release, I did some work and I wanted to share the latest status. I think we should target for *docker-image-based* release mode. I propose to release a unified open-source tarball, we don't release a binary tarball (not a must [1]). Things have been *DONE*: 1. I have created a repo for release mgmt: https://github.com/yangwwei/yunikorn-release, I think we need to move this to apache repo too. 2. This repo has the instructions and tools for a release. The tool loads configs from https://github.com/yangwwei/yunikorn-release/blob/master/tools/release-configs.json and downloads source code from certain repo/branch/hash to assemble the release artifacts 3. This repo contains a *build-docker-image.sh* to build yunikorn docker images (scheduler, admission-controller, and web) 4. I have created *branch-0.8* for all 4 repos 5. The generated tarball will also have the helm chart for user to install and run yunikorn on an existing K8s cluster 6. I tried to generate PGP key and sign the tarball Things *TODO* 1. Create a repo for yunikorn-release under ASF 2. IIUC, https://issues.apache.org/jira/browse/YUNIKORN-79 is a blocker for 0.8. Can we get this fixed ASAP? 3. Once #2 is done, create a tag for 0.8-rc1 and start the voting thread Thanks! [1] https://infra.apache.org/release-publishing.html *The Apache Software Foundation exists to create open source software, so the fundamental requirement for a release is that it has the necessary source code to build the project. A project may provide compiled binaries of each release for the convenience of users.*
[jira] [Resolved] (YUNIKORN-90) Wrong order of input parameters for strings.HasPrefix in utils.go
[
https://issues.apache.org/jira/browse/YUNIKORN-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-90.
-
Fix Version/s: 0.8
Resolution: Fixed
> Wrong order of input parameters for strings.HasPrefix in utils.go
> -
>
> Key: YUNIKORN-90
> URL: https://issues.apache.org/jira/browse/YUNIKORN-90
> Project: Apache YuniKorn
> Issue Type: Bug
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Minor
> Labels: pull-request-available
> Fix For: 0.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code} strings.HasPrefix("[", partitionName) {code} should be
> {code}strings.HasPrefix(partitionName, "[") {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-85) Improve recovery performance by querying all pods once instead of querying pods on specified node for many times
[ https://issues.apache.org/jira/browse/YUNIKORN-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-85. - Fix Version/s: 0.8 Resolution: Fixed > Improve recovery performance by querying all pods once instead of querying > pods on specified node for many times > > > Key: YUNIKORN-85 > URL: https://issues.apache.org/jira/browse/YUNIKORN-85 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Recently I did some performance tests on kubemark cluster with 1k hollow > nodes, and found the recovery phase will take almost 1 minute, querying pods > on specified node one by one is the most time-consuming logic inside. > Therefore I propose to query all pods once here to improve the recovery > performance, after this improvement, the cost time of recovery phase can > reduce to only 2 seconds. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-84) Always waiting in recovery phase as long as rejected node exists
[ https://issues.apache.org/jira/browse/YUNIKORN-84?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-84. - Fix Version/s: 0/8 Resolution: Fixed > Always waiting in recovery phase as long as rejected node exists > > > Key: YUNIKORN-84 > URL: https://issues.apache.org/jira/browse/YUNIKORN-84 > Project: Apache YuniKorn > Issue Type: Bug >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Major > Labels: pull-request-available > Fix For: 0/8 > > Time Spent: 20m > Remaining Estimate: 0h > > In recovery phase, scheduler keeps waiting until all nodes are recovered, but > not all nodes are considered according to Context#recover in > context_recovery.go, only nodes with Healthy or Draining state will be > counted, nodes with Rejected state should be counted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-72) data race in unit test: TestTaskStateTransitions
[ https://issues.apache.org/jira/browse/YUNIKORN-72?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-72. - Fix Version/s: 0.8 Resolution: Fixed > data race in unit test: TestTaskStateTransitions > > > Key: YUNIKORN-72 > URL: https://issues.apache.org/jira/browse/YUNIKORN-72 > Project: Apache YuniKorn > Issue Type: Test > Components: test - unit >Reporter: Wilfred Spiegelenburg > Assignee: Weiwei Yang >Priority: Minor > Labels: pull-request-available > Fix For: 0.8 > > Attachments: yunikorn-67_test_fail.txt > > Time Spent: 20m > Remaining Estimate: 0h > > Once in a while the TestTaskStateTransitions test fails in the shim with a > data race. > The test is flaky and needs to be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-82) Wrong revision in docker image
[
https://issues.apache.org/jira/browse/YUNIKORN-82?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-82.
-
Assignee: Tao Yang
Resolution: Fixed
> Wrong revision in docker image
> --
>
> Key: YUNIKORN-82
> URL: https://issues.apache.org/jira/browse/YUNIKORN-82
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Reporter: Weiwei Yang
>Assignee: Tao Yang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Reproduce steps
> # from k8shim, build the scheduler docker image, such as:
> yunikorn/yunikorn-scheduler-k8s:latest
> # then run "docker inspect yunikorn/yunikorn-scheduler-k8s:latest"
> # check Labels
> "Labels": \{ "BuildTimeStamp": "2020-04-07T10:17:10-0700", "Version":
> "latest", "yunikorn-core-revision": "20200403014952",
> "yunikorn-k8shim-revision": "0c83fd7f5ff5",
> "yunikorn-scheduler-interface-revision": "20200327234544" }
> Looks like the revision for {{yunikorn-core-revision}} and
> {{yunikorn-scheduler-interface-revision}} are incorrect. Code to set this is
> in Makefile
> {code:java}
> ...
> @coreSHA=$$(go list -m "github.com/apache/incubator-yunikorn-core" | cut -d
> "-" -f4) ; \
> siSHA=$$(go list -m
> "github.com/apache/incubator-yunikorn-scheduler-interface" | cut -d "-" -f5)
> ; \
> shimSHA=$$(git rev-parse --short=12 HEAD) ; \
> docker build ./deployments/image/configmap -t
> ${REGISTRY}/yunikorn-scheduler-k8s:${VERSION} \
> --label "yunikorn-core-revision=$${coreSHA}" \
> --label "yunikorn-scheduler-interface-revision=$${siSHA}" \
> --label "yunikorn-k8shim-revision=$${shimSHA}" \
> --label "BuildTimeStamp=${DATE}" \
> --label "Version=${VERSION}"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-82) Wrong revision in docker image
Weiwei Yang created YUNIKORN-82:
---
Summary: Wrong revision in docker image
Key: YUNIKORN-82
URL: https://issues.apache.org/jira/browse/YUNIKORN-82
Project: Apache YuniKorn
Issue Type: Bug
Components: shim - kubernetes
Reporter: Weiwei Yang
Reproduce steps
# from k8shim, build the scheduler docker image, such as:
yunikorn/yunikorn-scheduler-k8s:latest
# then run "docker inspect yunikorn/yunikorn-scheduler-k8s:latest"
# check Labels
"Labels": {
"BuildTimeStamp": "2020-04-07T10:17:10-0700",
"Version": "latest",
"yunikorn-core-revision": "20200403014952",
"yunikorn-k8shim-revision": "0c83fd7f5ff5",
"yunikorn-scheduler-interface-revision": "20200327234544"
}
Looks like the revision for \{{yunikorn-core-revision}} and
\{{yunikorn-scheduler-interface-revision}} are incorrect. Code to set this is
in Makefile
{code}
...
@coreSHA=$$(go list -m "github.com/apache/incubator-yunikorn-core" | cut -d
"-" -f4) ; \
siSHA=$$(go list -m "github.com/apache/incubator-yunikorn-scheduler-interface"
| cut -d "-" -f5) ; \
shimSHA=$$(git rev-parse --short=12 HEAD) ; \
docker build ./deployments/image/configmap -t
${REGISTRY}/yunikorn-scheduler-k8s:${VERSION} \
--label "yunikorn-core-revision=$${coreSHA}" \
--label "yunikorn-scheduler-interface-revision=$${siSHA}" \
--label "yunikorn-k8shim-revision=$${shimSHA}" \
--label "BuildTimeStamp=${DATE}" \
--label "Version=${VERSION}"
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-80) Add "global" toleration for admission controller
[ https://issues.apache.org/jira/browse/YUNIKORN-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-80. - Fix Version/s: 0.8 Resolution: Fixed > Add "global" toleration for admission controller > > > Key: YUNIKORN-80 > URL: https://issues.apache.org/jira/browse/YUNIKORN-80 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > In case of Helm chart installation, we can define tolerations for the > scheduler only, but it is not possible to the admission controller, so there > can be situations when some taints are assigned to a node; we add the > tolerations to the deployment descriptor for the scheduler, so it will be > assigned to the node, but since the admission controller will not come up > because itt does not tolerate the taints set for the node. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-68) YuniKorn admission-controller should skip pods that explicitly set schedulerName
[ https://issues.apache.org/jira/browse/YUNIKORN-68?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-68. - Fix Version/s: 0.8 Resolution: Won't Fix > YuniKorn admission-controller should skip pods that explicitly set > schedulerName > > > Key: YUNIKORN-68 > URL: https://issues.apache.org/jira/browse/YUNIKORN-68 > Project: Apache YuniKorn > Issue Type: Bug >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Fix For: 0.8 > > > Currently, as long as the admission-controller is enabled, it will mutate the > pod's spec and set schedulerName to Yunikorn no matter what. But in case some > user wants to leverage default scheduler, and they set schedulerName > explicitly, we should let it be that. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-78) Rename the label component=scheduler to component=yunikorn-scheduler
[ https://issues.apache.org/jira/browse/YUNIKORN-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-78. - Fix Version/s: 0.8 Resolution: Fixed > Rename the label component=scheduler to component=yunikorn-scheduler > > > Key: YUNIKORN-78 > URL: https://issues.apache.org/jira/browse/YUNIKORN-78 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > In YUNIKORN-76, we have added labels component=scheduler for pod-affinity. I > would suggest we rename the value to yunikorn-scheduler to make it more > unique. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-76) YuniKorn admission-controller should be installed on the same host as Yunikorn-scheduler
[ https://issues.apache.org/jira/browse/YUNIKORN-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-76. - Fix Version/s: 0.8 Resolution: Fixed > YuniKorn admission-controller should be installed on the same host as > Yunikorn-scheduler > > > Key: YUNIKORN-76 > URL: https://issues.apache.org/jira/browse/YUNIKORN-76 > Project: Apache YuniKorn > Issue Type: Bug > Components: shim - kubernetes >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > It would be good to have Yunikorn scheduler and admission controller > co-located. If a node goes down, it's better to have them all shut down and > bring up together. > let's fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-73) Temporarily disable the badge on github
[ https://issues.apache.org/jira/browse/YUNIKORN-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-73. - Fix Version/s: 0.8 Resolution: Fixed > Temporarily disable the badge on github > --- > > Key: YUNIKORN-73 > URL: https://issues.apache.org/jira/browse/YUNIKORN-73 > Project: Apache YuniKorn > Issue Type: Improvement > Components: documentation > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Seems like our UT is not very stable on github, let's disable the badge and > reenable it once we have a stable UT env. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-73) Temporarily disable the badge on github
Weiwei Yang created YUNIKORN-73: --- Summary: Temporarily disable the badge on github Key: YUNIKORN-73 URL: https://issues.apache.org/jira/browse/YUNIKORN-73 Project: Apache YuniKorn Issue Type: Improvement Components: documentation Reporter: Weiwei Yang Assignee: Weiwei Yang Seems like our UT is not very stable on github, let's disable the badge and reenable it once we have a stable UT env. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-44) Pass scheduler startup options from environment variables
[ https://issues.apache.org/jira/browse/YUNIKORN-44?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-44. - Fix Version/s: 0.8 Resolution: Fixed > Pass scheduler startup options from environment variables > - > > Key: YUNIKORN-44 > URL: https://issues.apache.org/jira/browse/YUNIKORN-44 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes > Reporter: Weiwei Yang >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > Currently, we have built the command line options directly into the docker > image, > [https://github.com/apache/incubator-yunikorn-k8shim/blob/master/deployments/image/configmap/Dockerfile]. > instead, we should pass them as environment variables, this allows us to > tweak some configurations without rebuilt the docker image. Please see > example here: > [https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-69) Design YUNIKORN logo
[ https://issues.apache.org/jira/browse/YUNIKORN-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-69. - Fix Version/s: 0.8 Resolution: Fixed > Design YUNIKORN logo > > > Key: YUNIKORN-69 > URL: https://issues.apache.org/jira/browse/YUNIKORN-69 > Project: Apache YuniKorn > Issue Type: Task >Reporter: Sunil G >Priority: Major > Fix For: 0.8 > > Attachments: yunikorn-log-gray.png, yunikorn-logo-black.png, > yunikorn-logo-blue (1).png, yunikorn-logo-main.png > > > New Yunikorn logo creation and using in all projects. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Closed] (YUNIKORN-63) Report occupied resources while registering a node
[ https://issues.apache.org/jira/browse/YUNIKORN-63?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang closed YUNIKORN-63. --- Fix Version/s: 0.8 Resolution: Fixed > Report occupied resources while registering a node > -- > > Key: YUNIKORN-63 > URL: https://issues.apache.org/jira/browse/YUNIKORN-63 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - scheduler, scheduler-interface, shim - kubernetes > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > We need to support this while registering a node, otherwise, it could be > race-condition that the shim reports an update but the node is not yet > registered. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-58) Create yunikorn website
[ https://issues.apache.org/jira/browse/YUNIKORN-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-58. - Fix Version/s: 0.8 Resolution: Fixed > Create yunikorn website > > > Key: YUNIKORN-58 > URL: https://issues.apache.org/jira/browse/YUNIKORN-58 > Project: Apache YuniKorn > Issue Type: Task > Components: website >Reporter: Wilfred Spiegelenburg >Assignee: Sunil G >Priority: Blocker > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 10m > Remaining Estimate: 0h > > A new repo has been created for the web site > Need to setup build process and content for the site > the website is needed for the first release of yunikorn -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-62) Fix time logging issue while doing reservation
[
https://issues.apache.org/jira/browse/YUNIKORN-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-62.
-
Fix Version/s: 0.8
Resolution: Fixed
> Fix time logging issue while doing reservation
> --
>
> Key: YUNIKORN-62
> URL: https://issues.apache.org/jira/browse/YUNIKORN-62
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler
> Reporter: Weiwei Yang
> Assignee: Weiwei Yang
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Met a weird logging issue with zap, looks like it is related to the time
> package.
> {code:java}
> Met a weird logging issueMet a weird logging issue
> ```2020-03-27T13:30:26.553-0700 DEBUG scheduler/scheduling_node.go:213
> running predicates failed {"allocationId":
> "c7ed4d67-7069-11ea-b2f0-0251", "nodeID": "docker-desktop", "error":
> "predicate CheckNodeUnschedulable cannot be satisified, reason [Predicate
> NodeUnknownCondition failed]"}panic: runtime error: invalid memory address or
> nil pointer dereference[signal SIGSEGV: segmentation violation code=0x1
> addr=0x0 pc=0x11ac568] goroutine 51
> [running]:go.uber.org/zap/zapcore.(*jsonEncoder).AppendDuration(0xc007eca390,
> 0x77370eba)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:239
> +0xc8go.uber.org/zap/zapcore.(*jsonEncoder).AddDuration(0xc007eca390,
> 0x2c60b0f, 0x6, 0x77370eba)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:123
> +0x65go.uber.org/zap/zapcore.Field.AddTo(0x2c60b0f, 0x6, 0x8, 0x77370eba,
> 0x0, 0x0, 0x0, 0x0, 0x2fd78c0, 0xc007eca390)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/field.go:126
> +0x558go.uber.org/zap/zapcore.addFields(0x2fd78c0, 0xc007eca390,
> 0xc00016e000, 0x4, 0x4)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/field.go:199
> +0x114go.uber.org/zap/zapcore.consoleEncoder.writeContext(0xcc1ec0,
> 0xc007e0ace0, 0xc00016e000, 0x4, 0x4)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/console_encoder.go:131
> +0xffgo.uber.org/zap/zapcore.consoleEncoder.EncodeEntry(0xcc1ec0, 0xff,
> 0xbf97b678a0fc80f8, 0x95cef19f3, 0x3f4a2a0, 0x0, 0x0, 0x2c72aac, 0x15, 0x1,
> ...)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/console_encoder.go:110
> +0x47ego.uber.org/zap/zapcore.(*ioCore).Write(0xcc1ef0, 0xff,
> 0xbf97b678a0fc80f8, 0x95cef19f3, 0x3f4a2a0, 0x0, 0x0, 0x2c72aac, 0x15, 0x1,
> ...) /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/core.go:86
> +0x107go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc007e52b00,
> 0xc00016e000, 0x4, 0x4)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/entry.go:216
> +0x1e8go.uber.org/zap.(*Logger).Debug(0xc0005265a0, 0x2c72aac, 0x15,
> 0xc00016e000, 0x4, 0x4)
> /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/logger.go:179
> +0x96github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).tryNodes(0xc000314360,
> 0xc006024000, 0x2f7e3e0, 0xc007d0d600, 0x0)
> /Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_application.go:529
>
> +0x7edgithub.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).tryAllocate(0xc000314360,
> 0xc00a2006e8, 0xc0001701c0, 0x0)
> /Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_application.go:399
>
> +0x176github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate(0xc000530e00,
> 0xc0001701c0, 0x1)
> /Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_queue.go:517
>
> +0x139github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate(0xc000170150,
> 0xc0001701c0, 0x0)
> /Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_queue.go:529
>
> +0x6c8github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*partitionSchedulingContext).tryAllocate(0xc0001701c0,
> 0xc00a2006d8)
> /Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_partition.go:366
>
> +0x94github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).schedule(0xce6050)
>
[jira] [Created] (YUNIKORN-63) Report occupied resources while registering a node
Weiwei Yang created YUNIKORN-63: --- Summary: Report occupied resources while registering a node Key: YUNIKORN-63 URL: https://issues.apache.org/jira/browse/YUNIKORN-63 Project: Apache YuniKorn Issue Type: Improvement Components: core - scheduler, scheduler-interface, shim - kubernetes Reporter: Weiwei Yang We need to support this while registering a node, otherwise, it could be race-condition that the shim reports an update but the node is not yet registered. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-62) Fix time logging issue while doing reservation
Weiwei Yang created YUNIKORN-62:
---
Summary: Fix time logging issue while doing reservation
Key: YUNIKORN-62
URL: https://issues.apache.org/jira/browse/YUNIKORN-62
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler
Reporter: Weiwei Yang
Met a weird logging issue with zap, looks like it is related to the time
package.
{code:java}
Met a weird logging issueMet a weird logging issue
```2020-03-27T13:30:26.553-0700 DEBUG scheduler/scheduling_node.go:213 running
predicates failed {"allocationId": "c7ed4d67-7069-11ea-b2f0-0251",
"nodeID": "docker-desktop", "error": "predicate CheckNodeUnschedulable cannot
be satisified, reason [Predicate NodeUnknownCondition failed]"}panic: runtime
error: invalid memory address or nil pointer dereference[signal SIGSEGV:
segmentation violation code=0x1 addr=0x0 pc=0x11ac568] goroutine 51
[running]:go.uber.org/zap/zapcore.(*jsonEncoder).AppendDuration(0xc007eca390,
0x77370eba)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:239
+0xc8go.uber.org/zap/zapcore.(*jsonEncoder).AddDuration(0xc007eca390,
0x2c60b0f, 0x6, 0x77370eba)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:123
+0x65go.uber.org/zap/zapcore.Field.AddTo(0x2c60b0f, 0x6, 0x8, 0x77370eba, 0x0,
0x0, 0x0, 0x0, 0x2fd78c0, 0xc007eca390)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/field.go:126
+0x558go.uber.org/zap/zapcore.addFields(0x2fd78c0, 0xc007eca390, 0xc00016e000,
0x4, 0x4) /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/field.go:199
+0x114go.uber.org/zap/zapcore.consoleEncoder.writeContext(0xcc1ec0,
0xc007e0ace0, 0xc00016e000, 0x4, 0x4)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/console_encoder.go:131
+0xffgo.uber.org/zap/zapcore.consoleEncoder.EncodeEntry(0xcc1ec0, 0xff,
0xbf97b678a0fc80f8, 0x95cef19f3, 0x3f4a2a0, 0x0, 0x0, 0x2c72aac, 0x15, 0x1,
...)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/console_encoder.go:110
+0x47ego.uber.org/zap/zapcore.(*ioCore).Write(0xcc1ef0, 0xff,
0xbf97b678a0fc80f8, 0x95cef19f3, 0x3f4a2a0, 0x0, 0x0, 0x2c72aac, 0x15, 0x1,
...) /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/core.go:86
+0x107go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc007e52b00, 0xc00016e000,
0x4, 0x4) /Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/zapcore/entry.go:216
+0x1e8go.uber.org/zap.(*Logger).Debug(0xc0005265a0, 0x2c72aac, 0x15,
0xc00016e000, 0x4, 0x4)
/Users/wyang/go/pkg/mod/go.uber.org/zap@v1.13.0/logger.go:179
+0x96github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).tryNodes(0xc000314360,
0xc006024000, 0x2f7e3e0, 0xc007d0d600, 0x0)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_application.go:529
+0x7edgithub.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingApplication).tryAllocate(0xc000314360,
0xc00a2006e8, 0xc0001701c0, 0x0)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_application.go:399
+0x176github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate(0xc000530e00,
0xc0001701c0, 0x1)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_queue.go:517
+0x139github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*SchedulingQueue).tryAllocate(0xc000170150,
0xc0001701c0, 0x0)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_queue.go:529
+0x6c8github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*partitionSchedulingContext).tryAllocate(0xc0001701c0,
0xc00a2006d8)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduling_partition.go:366
+0x94github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).schedule(0xce6050)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduler.go:581
+0x90bgithub.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).internalSchedule(0xce6050)
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduler.go:101
+0x39created by
github.com/apache/incubator-yunikorn-core/pkg/scheduler.(*Scheduler).StartService
/Users/wyang/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20200327185421-18d6381b4f86/pkg/scheduler/scheduler.go:74
+0x1b2make: *** [run] Error 2```
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
--
[jira] [Resolved] (YUNIKORN-60) Not working link pointing to build document in deployment.md
[ https://issues.apache.org/jira/browse/YUNIKORN-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-60. - Fix Version/s: 0.8 Resolution: Fixed > Not working link pointing to build document in deployment.md > > > Key: YUNIKORN-60 > URL: https://issues.apache.org/jira/browse/YUNIKORN-60 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > The link from > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/setup/deployment.md] > pointing to the build document is nor working. > It points to > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/setup/developer-guide.md] > instead of > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/developer-guide.md] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-61) Typo in user-guide.md sleep job example
[
https://issues.apache.org/jira/browse/YUNIKORN-61?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang resolved YUNIKORN-61.
-
Fix Version/s: 0.8
Resolution: Fixed
> Typo in user-guide.md sleep job example
> ---
>
> Key: YUNIKORN-61
> URL: https://issues.apache.org/jira/browse/YUNIKORN-61
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: documentation
>Reporter: Kinga Marton
>Assignee: Kinga Marton
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The provided example command is wrong:
> A simple sleep job example:
> {code:bash}
> kubectl create -f examples/sleep/sleeppods.xml{code}
> It should be
> {code:bash}
> kubectl create -f examples/sleep/sleeppods.yaml{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-59) Fix typo in the example document nignx -> nginx
Weiwei Yang created YUNIKORN-59: --- Summary: Fix typo in the example document nignx -> nginx Key: YUNIKORN-59 URL: https://issues.apache.org/jira/browse/YUNIKORN-59 Project: Apache YuniKorn Issue Type: Improvement Components: documentation Reporter: Weiwei Yang Assignee: Sunil G -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-57) Go mod tidy scheduler-core dependencies
Weiwei Yang created YUNIKORN-57: --- Summary: Go mod tidy scheduler-core dependencies Key: YUNIKORN-57 URL: https://issues.apache.org/jira/browse/YUNIKORN-57 Project: Apache YuniKorn Issue Type: Improvement Components: core - scheduler Reporter: Weiwei Yang Assignee: Weiwei Yang Cleanup unnecessary dependencies -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-56) Broken Scheduler configuration link in deployment.md
[ https://issues.apache.org/jira/browse/YUNIKORN-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-56. - Fix Version/s: 0.8 Resolution: Fixed PR merged, thanks [~kmarton]! > Broken Scheduler configuration link in deployment.md > > > Key: YUNIKORN-56 > URL: https://issues.apache.org/jira/browse/YUNIKORN-56 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > The link from deployment.md pointing to how to manage scheduler's > configuration via configmap is broken. > it points to > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/setup/setup/configure-scheduler.md] > and it should be > [https://github.com/apache/incubator-yunikorn-core/blob/master/docs/setup/configure-scheduler.md] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-55) Add pod labels and annotations to allocation ask attributes
Weiwei Yang created YUNIKORN-55: --- Summary: Add pod labels and annotations to allocation ask attributes Key: YUNIKORN-55 URL: https://issues.apache.org/jira/browse/YUNIKORN-55 Project: Apache YuniKorn Issue Type: Bug Components: core - scheduler, shim - kubernetes, webapp Reporter: Weiwei Yang In YUNIKORN-54, we simplify the way to generate application IDs. The side effect is when we look at info from web UI, we lose some info about the pod info such as pod name, namespace, etc. A proper way to handle this is to get this info from pod and add labels, annotations info to allocation ask as attributes, and then send to scheduler-core. Also on web rest API, we need to display these attributes too. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-54) admission-controller: the generated applicationID should not exceed 64 chars
Weiwei Yang created YUNIKORN-54: --- Summary: admission-controller: the generated applicationID should not exceed 64 chars Key: YUNIKORN-54 URL: https://issues.apache.org/jira/browse/YUNIKORN-54 Project: Apache YuniKorn Issue Type: Bug Components: shim - kubernetes Reporter: Weiwei Yang If a label exceeds 64 chars, it will be rejected by some K8s clients. Currently, we composite a generated name by -- which might exceed 64 chars. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-52) Update queue/app screenshots in README
Weiwei Yang created YUNIKORN-52: --- Summary: Update queue/app screenshots in README Key: YUNIKORN-52 URL: https://issues.apache.org/jira/browse/YUNIKORN-52 Project: Apache YuniKorn Issue Type: Bug Components: documentation Reporter: Weiwei Yang Assignee: Weiwei Yang We have some updates in UI, let's refresh the screenshots as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-49) K8shim should report occupied resources to scheduler core
Weiwei Yang created YUNIKORN-49: --- Summary: K8shim should report occupied resources to scheduler core Key: YUNIKORN-49 URL: https://issues.apache.org/jira/browse/YUNIKORN-49 Project: Apache YuniKorn Issue Type: Sub-task Components: shim - kubernetes Reporter: Weiwei Yang Assignee: Weiwei Yang Report occupied resource (resources used by other schedulers) to the scheduler core and keep them consistent. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-48) Add UPDATE action in order to report node resource updates
[ https://issues.apache.org/jira/browse/YUNIKORN-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-48. - Fix Version/s: 0.8 Assignee: Weiwei Yang Resolution: Fixed > Add UPDATE action in order to report node resource updates > -- > > Key: YUNIKORN-48 > URL: https://issues.apache.org/jira/browse/YUNIKORN-48 > Project: Apache YuniKorn > Issue Type: Improvement > Components: scheduler-interface > Reporter: Weiwei Yang > Assignee: Weiwei Yang >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 10m > Remaining Estimate: 0h > > We need a UPDATE action in scheduler-interface in order to sync with node > updates. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-48) Add UPDATE action in order to report node resource updates
Weiwei Yang created YUNIKORN-48: --- Summary: Add UPDATE action in order to report node resource updates Key: YUNIKORN-48 URL: https://issues.apache.org/jira/browse/YUNIKORN-48 Project: Apache YuniKorn Issue Type: Improvement Components: scheduler-interface Reporter: Weiwei Yang We need a UPDATE action in scheduler-interface in order to sync with node updates. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-47) Support node capacity update
Weiwei Yang created YUNIKORN-47: --- Summary: Support node capacity update Key: YUNIKORN-47 URL: https://issues.apache.org/jira/browse/YUNIKORN-47 Project: Apache YuniKorn Issue Type: Improvement Components: core - scheduler Reporter: Weiwei Yang We need to support the live updates for node capacity. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-35) Broken DefaultSchedulerPolicy link in predicates.md
[ https://issues.apache.org/jira/browse/YUNIKORN-35?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-35. - Fix Version/s: 0.8 Resolution: Fixed > Broken DefaultSchedulerPolicy link in predicates.md > > > Key: YUNIKORN-35 > URL: https://issues.apache.org/jira/browse/YUNIKORN-35 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation, shim - kubernetes >Reporter: Kinga Marton >Assignee: Kinga Marton >Priority: Minor > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > > The link for DefaultSchedulerPolicy in > [https://github.com/apache/incubator-yunikorn-k8shim/blob/master/docs/design/predicates.md] > is pointing to a wrong location > ([https://github.com/apache/incubator-yunikorn-k8shim/blob/master/pkg/predicates/predictor.go]). > It should be > [https://github.com/apache/incubator-yunikorn-k8shim/blob/master/pkg/plugin/predicates/predictor.go] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-46) Add a quick search box in app page
Weiwei Yang created YUNIKORN-46: --- Summary: Add a quick search box in app page Key: YUNIKORN-46 URL: https://issues.apache.org/jira/browse/YUNIKORN-46 Project: Apache YuniKorn Issue Type: Bug Components: webapp Reporter: Weiwei Yang It would be good to add a quick-search box in the app page, in order to locate to a certain app quickly. I found it is usually quite difficult when there is a lot of apps. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-45) Task is not properly released when they are removed from pending state
Weiwei Yang created YUNIKORN-45: --- Summary: Task is not properly released when they are removed from pending state Key: YUNIKORN-45 URL: https://issues.apache.org/jira/browse/YUNIKORN-45 Project: Apache YuniKorn Issue Type: Bug Components: shim - kubernetes Reporter: Weiwei Yang We've observed in some cases there are some resources leaked on the cluster, causing some inconsistent state; after narrowing down the issue, it seems we are not properly releasing task resources when they are deleted from the pending state. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-40) Add doc for YuniKorn Community Sync Up
[ https://issues.apache.org/jira/browse/YUNIKORN-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-40. - Resolution: Fixed > Add doc for YuniKorn Community Sync Up > -- > > Key: YUNIKORN-40 > URL: https://issues.apache.org/jira/browse/YUNIKORN-40 > Project: Apache YuniKorn > Issue Type: Task > Components: community, documentation >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-44) Pass scheduler startup options from environment variables
Weiwei Yang created YUNIKORN-44: --- Summary: Pass scheduler startup options from environment variables Key: YUNIKORN-44 URL: https://issues.apache.org/jira/browse/YUNIKORN-44 Project: Apache YuniKorn Issue Type: Improvement Components: shim - kubernetes Reporter: Weiwei Yang Currently, we have built the command line options directly into the docker image, [https://github.com/apache/incubator-yunikorn-k8shim/blob/master/deployments/image/configmap/Dockerfile]. instead, we should pass them as environment variables, this allows us to tweak some configurations without rebuilt the docker image. Please see example here: [https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-40) Add doc for YuniKorn Community Sync Up
[ https://issues.apache.org/jira/browse/YUNIKORN-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang resolved YUNIKORN-40. - Fix Version/s: 0.8 Resolution: Fixed > Add doc for YuniKorn Community Sync Up > -- > > Key: YUNIKORN-40 > URL: https://issues.apache.org/jira/browse/YUNIKORN-40 > Project: Apache YuniKorn > Issue Type: Task > Components: community, documentation >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Labels: pull-request-available > Fix For: 0.8 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
Prepare for 0.8 release
Hi folks I'd like to call out for starting the YuniKorn 0.8 release now (the first release under Apache incubator). In the past few months, we got tremendous help from community members to help the growth of the project, now I think it is a good time to cut for 0.8 release. This release will target for K8s users who have needs to run Big Data workloads on K8s. Features include: - Hirecharcy resource queues and cross-queue fairness - Job ordering support at queue level, FAIR & FIFO - Configurable node-sorting policies: FAIR/BIN-PACKING - Automated reservations to avoid starving jobs - Dynamical queue creation and management via user-defined placement rules - Support K8s predicates functions and volume bindings - Configuration hot-update - Stateful recovery We've been working really hard to roll out all these features, in order to make YuniKorn standout and ready for production use. I propose to do code freeze on *March 27*, and the target release date is *Apr 3*. Let me know your thoughts, thanks! Weiwei
Re: emails notification updates for JIRA and GitHub
Sounds good, thank you! On Tue, Mar 17, 2020 at 8:57 PM Wilfred Spiegelenburg wrote: > Based on the fact that I have not had any further feedback on the changes > have started the process to change the jira notifications. > All notifications will go to: issues@ (new, changes, comments etc) > Jira creation and closure will also be send to the dev@ list. > > I have asked the project mentors to create a reviews@ list for us and when > that is done I'll ask the notification list for github to also be updated. > > I will send out and update when the changes have completed. > > Wilfred > > On Wed, 11 Mar 2020 at 14:23, Wilfred Spiegelenburg > wrote: > > > Sunil, > > > > Yes we are going to use issues@ for JIRA. That is just around the > corner, > > really a no brainer. > > > > Reviews or GitHub mails is something different. Hadoop Ozone is one of > the > > reasons I am asking this. > > The dev list got spammed by GitHub messages from ozone and they moved to > a > > separate list there. > > I am trying to be ahead of the curve and move our GitHub messages too. > > > > There are a total of 12 projects that use a reviews@ mailing list for > > apache projects. > > Projects like Spark, impala, kudu & ambari to name a few. So I don't > think > > we are that out of sync. > > Most of these projects also use a issues@ for their jira tracking. > > I am also OK with mixing the GitHub and JIRA mails on one list (issues@) > > if that is the preferred option, that is why I am asking. > > > > I looked at some other projects and if we mix the GitHub messages in with > > the standard dev@ list. > > It can cause the dev list to blow out from 1-2 messages per day to 20+ > > messages a day. > > Off course you can always filter the messages and separate on receive but > > keeping them separate to start with is easier. > > > > Cheers, > > Wilfred > > > > On Wed, 11 Mar 2020 at 13:13, Sunil Govindan wrote: > > > >> Hi Weiwei and Wilfred, > >> > >> I have not seen a reviews@ list so far. In other projects, I have seen > >> issues@ list where all updates on jira will go. > >> In any case, I agree in removing such notifications from dev@ list. And > >> if possible, also having an issues@ list as well > >> > >> - Sunil > >> > >> On Tue, Mar 10, 2020 at 10:04 AM Wilfred Spiegelenburg < > >> wilfr...@apache.org> wrote: > >> > >>> Hi Weiwei, > >>> > >>> Currently all notifications for github go to dev@, that is a setting > on > >>> the > >>> github repos that can be changed. > >>> I propose to move all of them to the reviews@ list and remove dev@ > from > >>> the > >>> github notifications > >>> We should be using dev@ for mails that require discussions and votes > >>> not as > >>> a generic notification list I think. > >>> > >>> Wilfred > >>> > >>> On Tue, 10 Mar 2020 at 15:18, Weiwei Yang wrote: > >>> > >>> > Hi Wilfred > >>> > > >>> > Thanks for taking care of this. > >>> > For the second one, since we already have GitBox notification about > >>> PRs, > >>> > creating reviews@ mailing list would get us too many duplicated > >>> messages? > >>> > > >>> > Weiwei > >>> > > >>> > On Mon, Mar 9, 2020 at 9:11 PM Wilfred Spiegelenburg < > >>> wilfr...@apache.org> > >>> > wrote: > >>> > > >>> >> Hi all, > >>> >> > >>> >> I noticed last week when moving all issues into the JIRA that there > >>> is no > >>> >> email send for JIRAs at all. I checked the project settings and we > >>> have > >>> >> currently not linked up the JIRA notifications to the correct list. > >>> >> The original plan was to use issues@ as the mailing list. However > we > >>> seem > >>> >> to have missed that change. > >>> >> > >>> >> If there are no reservations to using this I will ask INFRA to > change > >>> the > >>> >> notification list to issues@ for the following events: > >>> >> * Created > >>> >> * Updated > >>> >> * Assigned > >>> >> * Resolved > >>> >> * Closed > >>&g
Re: emails notification updates for JIRA and GitHub
Hi Wilfred Thanks for taking care of this. For the second one, since we already have GitBox notification about PRs, creating reviews@ mailing list would get us too many duplicated messages? Weiwei On Mon, Mar 9, 2020 at 9:11 PM Wilfred Spiegelenburg wrote: > Hi all, > > I noticed last week when moving all issues into the JIRA that there is no > email send for JIRAs at all. I checked the project settings and we have > currently not linked up the JIRA notifications to the correct list. > The original plan was to use issues@ as the mailing list. However we seem > to have missed that change. > > If there are no reservations to using this I will ask INFRA to change the > notification list to issues@ for the following events: > * Created > * Updated > * Assigned > * Resolved > * Closed > * Commented > * Comment Edited > That will bring us in line with most of the other projects. > > Second point for mailing lists: commits@ was originally slated for GitHub > emails. The mailing list does not exist at the moment. Currently all the > git emails come in on the dev@ mailing list which clutters that list. > Instead of creating commits@ I would like to propose to change the name to > reviews@ as it better covers what we do on GitHub with the PRs and the > review on the code. > > Please respond to this mail if you are not OK with the changes. If there > are no objections I will start the changes on Monday 16 March 2020 (AEDT > +11). > > Thank you, > Wilfred > -- *Weiwei Yang* Software Engineer, Cloudera Create your own WiseStamp email signature <https://www.wisestamp.com/signature-in-email/?utm_source=promotion_medium=signature_campaign=create_your_own=6536956273754112>
Re: Added a project status page
Hi Holden Thank you! It seems all good on that page except 2 minor things 1. Description >YuniKorn is a standalone resource scheduler ... There seems to be an extra ">", can we remove it? 2. Committers We are missing two names: - Wilfred Spiegelenburg - Akhil PB Thanks Weiwei On Fri, Feb 14, 2020 at 12:02 PM Holden Karau wrote: > Hi folks, I added a project status page ( > https://incubator.apache.org/projects/yunikorn.html ) if you notice > anything wrong with it feel free to let me know and I'll be happy to update > it (or help someone else update it). > > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > -- *Weiwei Yang* Software Engineer, Cloudera Create your own email signature <https://www.wisestamp.com/signature-in-email/?utm_source=promotion_medium=signature_campaign=create_your_own=6536956273754112>
Design review for priority support
Hi team Please kindly review the design doc in https://issues.apache.org/jira/browse/YUNIKORN-1. This explains the idea of how to support the priority in YuniKorn. Thanks for Tao's effort to work on this! Weiwei
