Re: YuniKorn Metrics

2021-04-15 Thread Tao Yang
Hi, Chaoran Sorry to be late for this response. Yes, We have did some performance tests and found that the scheduling process is far from transparent at the beginning, just as you said, the internal metrics is not good enough for us to spot issues or locate bottlenecks. So we have tried to

[jira] [Resolved] (YUNIKORN-634) Update the website for v0.10

2021-04-09 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang resolved YUNIKORN-634. --- Resolution: Duplicate > Update the website for v0

[jira] [Created] (YUNIKORN-634) Update the website for v0.10

2021-04-09 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-634: - Summary: Update the website for v0.10 Key: YUNIKORN-634 URL: https://issues.apache.org/jira/browse/YUNIKORN-634 Project: Apache YuniKorn Issue Type: Sub-task

[jira] [Created] (YUNIKORN-633) Update index.yaml with new release v0.10.0

2021-04-08 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-633: - Summary: Update index.yaml with new release v0.10.0 Key: YUNIKORN-633 URL: https://issues.apache.org/jira/browse/YUNIKORN-633 Project: Apache YuniKorn Issue Type

[RESULT] [VOTE] Release Apache YuniKorn (incubating) 0.10.0

2021-04-02 Thread Tao Yang
Hi all The vote to release Apache YuniKorn (Incubating) 0.10.0 has passed. We got 4 binding votes - Weiwei Yang - Wilfred Spiegelenburg - Julia Kinga Marton - Jason Lowe and 1 non-binding vote - Chaoran Yu Vote thread:

Re: [VOTE] Release Apache YuniKorn (incubating) 0.10.0

2021-04-02 Thread Tao Yang
unit tests except web tests > > Jason > > On Mon, Mar 29, 2021 at 1:31 AM Tao Yang wrote: > > > Hi all > > > > This is the first release candidate for the Apache YuniKorn (incubating) > > 0.10.0 release. > > > > We have resolved 184 issues. Incl

Re: [VOTE] Release Apache YuniKorn (incubating) 0.10.0

2021-03-29 Thread Tao Yang
; > > > > > other than that, I've done the following verifications and they are all > > > good: > > > > > >1. Verified ASF LICENSE files > > >2. Build docker images from source successfully > > >3. Verified docker image SHAs m

[VOTE] Release Apache YuniKorn (incubating) 0.10.0

2021-03-29 Thread Tao Yang
Hi all This is the first release candidate for the Apache YuniKorn (incubating) 0.10.0 release. We have resolved 184 issues. Including gang-scheduling support, application lifecycle control, dependency upgrades (go 1.15 & kubernetes 1.16), improvements on web UI and web-site. The release

[jira] [Created] (YUNIKORN-612) Create 0.10.0 release notes

2021-03-25 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-612: - Summary: Create 0.10.0 release notes Key: YUNIKORN-612 URL: https://issues.apache.org/jira/browse/YUNIKORN-612 Project: Apache YuniKorn Issue Type: Sub-task

[jira] [Resolved] (YUNIKORN-602) tag release 0.10.0 and update go mod files

2021-03-25 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang resolved YUNIKORN-602. --- Resolution: Fixed > tag release 0.10.0 and update go mod fi

Re: Apache YuniKorn 0.10.0 Release Plan

2021-03-25 Thread Tao Yang
: > > > Thanks Tao. Hope we can release it faster/on-time since it's been delayed > > from the beginning of March. > > > > Look forward to the 0.10 release with gang scheduling in place. > > > > Thanks, > > Bowen > > > > On Wed, Mar 24, 2021

[jira] [Created] (YUNIKORN-604) Create 0.10.0 helm chart release

2021-03-25 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-604: - Summary: Create 0.10.0 helm chart release Key: YUNIKORN-604 URL: https://issues.apache.org/jira/browse/YUNIKORN-604 Project: Apache YuniKorn Issue Type: Task

[jira] [Created] (YUNIKORN-602) tag release 0.10.0 and update go mod files

2021-03-24 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-602: - Summary: tag release 0.10.0 and update go mod files Key: YUNIKORN-602 URL: https://issues.apache.org/jira/browse/YUNIKORN-602 Project: Apache YuniKorn Issue Type

Apache YuniKorn 0.10.0 Release Plan

2021-03-24 Thread Tao Yang
Hi all, Currently the jiras tracked for 0.10 [1] are almost done, I would like to start the release process for v0.10.0 according to the release procedure document [2] in the next few days. When this is done I will start a voting thread for the newly uploaded release candidate. Please let me

[jira] [Created] (YUNIKORN-541) Unexpected WARN log in node.go#refreshAvailableResource

2021-02-07 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-541: - Summary: Unexpected WARN log in node.go#refreshAvailableResource Key: YUNIKORN-541 URL: https://issues.apache.org/jira/browse/YUNIKORN-541 Project: Apache YuniKorn

Re: [DISCUSS] Release manager for 0.10

2021-01-29 Thread Tao Yang
8, 2021 at 7:15 PM Tao Yang wrote: > > > Hi, Wilfred. > > > > Thanks for your efforts to keep this work going. May I have a chance to > be > > the release manager for 0.10? > > I have no experience about this before, but I would like to study from > the >

[jira] [Created] (YUNIKORN-410) Pod state change may cause incorrect update on SchedulerNode#occupied

2020-09-11 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-410: - Summary: Pod state change may cause incorrect update on SchedulerNode#occupied Key: YUNIKORN-410 URL: https://issues.apache.org/jira/browse/YUNIKORN-410 Project: Apache

Re: Resource request and limit for YuniKorn pods

2020-04-21 Thread Tao Yang
Thanks Adam for this efforts! I have did some performance tests on kubemark cluster, if we can improve some key phases in the scheduling process, the scheduler pod can take 1~4 CPUs at most times and 7~9 CPUs at peak times, the memory seems may easily go beyond 1G after tens of thousands pods

Re: [Discussion] Add integration testing framework and cases

2020-04-20 Thread Tao Yang
s set up a > meeting to discuss these points in detail, as Weiwei suggested. > > Thanks > Ayub Khan > > On Sun, Apr 19, 2020 at 9:08 PM Tao Yang wrote: > > > Thanks Weiwei for the comments. > > > > Of course we should have some verification results in cases, m

Re: Improve node sorting algorithm

2020-04-19 Thread Tao Yang
ed on different requirements is useful. Thanks for > starting this work. > > On Wed, Apr 15, 2020 at 1:11 AM Sunil Govindan wrote: > > > Thanks Tao for the efforts. Appreciate it. > > I will help to take a look on this. > > > > Thanks > > Sunil > > > &

[Discussion] Add integration testing framework and cases

2020-04-17 Thread Tao Yang
Hello everyone We are planning to add integration testing framework and initial test cases in https://issues.apache.org/jira/browse/YUNIKORN-29, general thoughts are as follows. Basic testing framework includes: 1. AppInfo struct: define basic information, requests and status of an application

Improve node sorting algorithm

2020-04-15 Thread Tao Yang
. If you are interested on this, we are looking forward to your comments and suggestions. Thanks, Tao Yang

[jira] [Created] (YUNIKORN-90) Wrong order of input parameters for strings.HasPrefix in utils.go

2020-04-09 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-90: Summary: Wrong order of input parameters for strings.HasPrefix in utils.go Key: YUNIKORN-90 URL: https://issues.apache.org/jira/browse/YUNIKORN-90 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-85) Improve recovery performance by querying all pods once instead of querying pods on specified node for many times

2020-04-09 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-85: Summary: Improve recovery performance by querying all pods once instead of querying pods on specified node for many times Key: YUNIKORN-85 URL: https://issues.apache.org/jira/browse

[jira] [Created] (YUNIKORN-84) Always waiting in recovery phase as long as rejected node exists

2020-04-09 Thread Tao Yang (Jira)
Tao Yang created YUNIKORN-84: Summary: Always waiting in recovery phase as long as rejected node exists Key: YUNIKORN-84 URL: https://issues.apache.org/jira/browse/YUNIKORN-84 Project: Apache YuniKorn