Re: Discussion with K8s Sig Team

2021-11-04 Thread Chaoran Yu
Hi Sunil, Next Wednesday at the same time works for me. From what I understand in the meeting yesterday, we want to initiate a conversation with sig-scheduling without committing to any actions just yet. The information we want to gather from them include the following: * What's the overall

[jira] [Resolved] (YUNIKORN-625) Use v1 for CertificateSigningRequest instead of v1beta1

2021-10-25 Thread Chaoran Yu (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu resolved YUNIKORN-625. - Resolution: Done > Use v1 for CertificateSigningRequest instead of v1be

Re: [DISCUSS] release v1.0.0 planning

2021-10-24 Thread Chaoran Yu
to our website: > https://yunikorn.apache.org/community/download/. > > On Fri, Oct 22, 2021 at 4:02 PM Chaoran Yu wrote: > >> I saw that we already have a JIRA filter for 1.0 >> <https://issues.apache.org/jira/projects/YUNIKORN/versions/12350288>. >> Sunil, where

[jira] [Created] (YUNIKORN-908) [Umbrella] Kubernetes 1.21 support

2021-10-24 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-908: --- Summary: [Umbrella] Kubernetes 1.21 support Key: YUNIKORN-908 URL: https://issues.apache.org/jira/browse/YUNIKORN-908 Project: Apache YuniKorn Issue Type: New

[jira] [Created] (YUNIKORN-911) CLONE - Verify helm chart install on 1.20

2021-10-24 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-911: --- Summary: CLONE - Verify helm chart install on 1.20 Key: YUNIKORN-911 URL: https://issues.apache.org/jira/browse/YUNIKORN-911 Project: Apache YuniKorn Issue

[jira] [Created] (YUNIKORN-910) CLONE - Verify predicates functions for K8s 1.20

2021-10-24 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-910: --- Summary: CLONE - Verify predicates functions for K8s 1.20 Key: YUNIKORN-910 URL: https://issues.apache.org/jira/browse/YUNIKORN-910 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-909) CLONE - Add 1.20 to the e2e test support matrix

2021-10-24 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-909: --- Summary: CLONE - Add 1.20 to the e2e test support matrix Key: YUNIKORN-909 URL: https://issues.apache.org/jira/browse/YUNIKORN-909 Project: Apache YuniKorn

[jira] [Resolved] (YUNIKORN-813) The capacity of undefined resource should NOT be considered zero

2021-10-24 Thread Chaoran Yu (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu resolved YUNIKORN-813. - Resolution: Fixed > The capacity of undefined resource should NOT be considered z

[jira] [Resolved] (YUNIKORN-897) CLONE - CLONE - Update the copyright years in NOTICE file

2021-10-22 Thread Chaoran Yu (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu resolved YUNIKORN-897. - Assignee: Chaoran Yu (was: Tao Yang) Resolution: Not A Problem This Jira was generated

[jira] [Created] (YUNIKORN-903) CLONE - Cleanup release area

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-903: --- Summary: CLONE - Cleanup release area Key: YUNIKORN-903 URL: https://issues.apache.org/jira/browse/YUNIKORN-903 Project: Apache YuniKorn Issue Type: Sub-task

[jira] [Created] (YUNIKORN-904) CLONE - Update website announce bar

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-904: --- Summary: CLONE - Update website announce bar Key: YUNIKORN-904 URL: https://issues.apache.org/jira/browse/YUNIKORN-904 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-898) CLONE - Add 0.11.0 to the artifact hub

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-898: --- Summary: CLONE - Add 0.11.0 to the artifact hub Key: YUNIKORN-898 URL: https://issues.apache.org/jira/browse/YUNIKORN-898 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-901) CLONE - Update the website for v0.11

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-901: --- Summary: CLONE - Update the website for v0.11 Key: YUNIKORN-901 URL: https://issues.apache.org/jira/browse/YUNIKORN-901 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-894) CLONE - Create the docs release for v0.11

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-894: --- Summary: CLONE - Create the docs release for v0.11 Key: YUNIKORN-894 URL: https://issues.apache.org/jira/browse/YUNIKORN-894 Project: Apache YuniKorn Issue

[jira] [Created] (YUNIKORN-900) CLONE - Update helm index for v0.11

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-900: --- Summary: CLONE - Update helm index for v0.11 Key: YUNIKORN-900 URL: https://issues.apache.org/jira/browse/YUNIKORN-900 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-902) CLONE - Update the CHANGELOG and generate the release

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-902: --- Summary: CLONE - Update the CHANGELOG and generate the release Key: YUNIKORN-902 URL: https://issues.apache.org/jira/browse/YUNIKORN-902 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-893) CLONE - Create 0.11.0 release notes

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-893: --- Summary: CLONE - Create 0.11.0 release notes Key: YUNIKORN-893 URL: https://issues.apache.org/jira/browse/YUNIKORN-893 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-895) CLONE - Update shim and core dependencies on master branch

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-895: --- Summary: CLONE - Update shim and core dependencies on master branch Key: YUNIKORN-895 URL: https://issues.apache.org/jira/browse/YUNIKORN-895 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-899) CLONE - [Helm chart] Update supported K8s versions

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-899: --- Summary: CLONE - [Helm chart] Update supported K8s versions Key: YUNIKORN-899 URL: https://issues.apache.org/jira/browse/YUNIKORN-899 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-891) [Umbrella] YuniKorn 1.0 release-related efforts

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-891: --- Summary: [Umbrella] YuniKorn 1.0 release-related efforts Key: YUNIKORN-891 URL: https://issues.apache.org/jira/browse/YUNIKORN-891 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-897) CLONE - CLONE - Update the copyright years in NOTICE file

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-897: --- Summary: CLONE - CLONE - Update the copyright years in NOTICE file Key: YUNIKORN-897 URL: https://issues.apache.org/jira/browse/YUNIKORN-897 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-892) CLONE - Create 0.11.0 helm chart release

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-892: --- Summary: CLONE - Create 0.11.0 helm chart release Key: YUNIKORN-892 URL: https://issues.apache.org/jira/browse/YUNIKORN-892 Project: Apache YuniKorn Issue

[jira] [Created] (YUNIKORN-896) CLONE - Tag release 0.11.0 and update go mod files

2021-10-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-896: --- Summary: CLONE - Tag release 0.11.0 and update go mod files Key: YUNIKORN-896 URL: https://issues.apache.org/jira/browse/YUNIKORN-896 Project: Apache YuniKorn

Re: [DISCUSS] release v1.0.0 planning

2021-10-22 Thread Chaoran Yu
supported version matrix to include 1.18, > > 1.19, > > > > 1.20, and 1.21? Updating the e2e test matrix should be a one-line > > change, > > > > and as part of the rebuild against 1.20, I verified that 1.21 is > > > functional. > > > > >

Re: Broadcast YuniKorn to the Industry!

2021-10-14 Thread Chaoran Yu
are of that sort of sponsorship. We can at least setup virtual > meetings via Zoom. > A monthly recurring meeting might be a good start : ) > > On Thu, Oct 14, 2021 at 9:27 PM Chaoran Yu > wrote: > > > I second this proposal! > > > > Instead of a one-time event

Re: Broadcast YuniKorn to the Industry!

2021-10-14 Thread Chaoran Yu
I second this proposal! Instead of a one-time event, we can do a recurring meetup (monthly or bi-monthly) and invite YuniKorn developers and users to talk about their experience and insights. A recurring event is well suited for cultivating a community. I took a look at Meetup.com. They charge a

[jira] [Closed] (YUNIKORN-833) Utilization and used capacity of queues are not displayed

2021-09-03 Thread Chaoran Yu (Jira)
[ https://issues.apache.org/jira/browse/YUNIKORN-833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaoran Yu closed YUNIKORN-833. --- Resolution: Cannot Reproduce > Utilization and used capacity of queues are not displa

[jira] [Created] (YUNIKORN-833) Utilization and used capacity of queues are not displayed

2021-09-03 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-833: --- Summary: Utilization and used capacity of queues are not displayed Key: YUNIKORN-833 URL: https://issues.apache.org/jira/browse/YUNIKORN-833 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-829) Produce metrics on queue-level resource utilization

2021-08-26 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-829: --- Summary: Produce metrics on queue-level resource utilization Key: YUNIKORN-829 URL: https://issues.apache.org/jira/browse/YUNIKORN-829 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-796) Pod scheduling could disproportionately concentrate on one node

2021-08-12 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-796: --- Summary: Pod scheduling could disproportionately concentrate on one node Key: YUNIKORN-796 URL: https://issues.apache.org/jira/browse/YUNIKORN-796 Project: Apache

[jira] [Created] (YUNIKORN-788) Make the scheduler max QPS match the default Kubernetes API server max requests inflight

2021-08-08 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-788: --- Summary: Make the scheduler max QPS match the default Kubernetes API server max requests inflight Key: YUNIKORN-788 URL: https://issues.apache.org/jira/browse/YUNIKORN-788

Re: [VOTE] Release Apache YuniKorn (incubating) 0.11.0 RC2

2021-08-04 Thread Chaoran Yu
+1 (nonbinding). I haven't done any additional verifications since my vote on RC1. But since the changes are on licensing only, I'll be able to give my thumb up On Mon, Aug 2, 2021 at 3:05 PM Julia Kinga Marton wrote: > Hi all, > > Even though the first RC passed this vote here, the IPMC found

Re: [VOTE] Release Apache YuniKorn (incubating) 0.11.0

2021-07-25 Thread Chaoran Yu
+1 (nonbinding) * Built images from the source * Ran unit tests * Ran the scheduler in K8s 1.17 and 1.20. Verified that regular scheduling works and gang scheduling works for Spark. * Verified UI and REST APIs Filed https://issues.apache.org/jira/browse/YUNIKORN-764 to document release

[jira] [Created] (YUNIKORN-764) Document how to verify a release

2021-07-25 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-764: --- Summary: Document how to verify a release Key: YUNIKORN-764 URL: https://issues.apache.org/jira/browse/YUNIKORN-764 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-747) Document how to run Spark applications with Spark Operator

2021-07-14 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-747: --- Summary: Document how to run Spark applications with Spark Operator Key: YUNIKORN-747 URL: https://issues.apache.org/jira/browse/YUNIKORN-747 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-711) Add license file to the release repo

2021-06-21 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-711: --- Summary: Add license file to the release repo Key: YUNIKORN-711 URL: https://issues.apache.org/jira/browse/YUNIKORN-711 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-712) Add license file to the site repo

2021-06-21 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-712: --- Summary: Add license file to the site repo Key: YUNIKORN-712 URL: https://issues.apache.org/jira/browse/YUNIKORN-712 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-710) Document how to upgrade a scheduler Helm release

2021-06-20 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-710: --- Summary: Document how to upgrade a scheduler Helm release Key: YUNIKORN-710 URL: https://issues.apache.org/jira/browse/YUNIKORN-710 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-704) Scheduling of DaemonSet pods may fail

2021-06-12 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-704: --- Summary: Scheduling of DaemonSet pods may fail Key: YUNIKORN-704 URL: https://issues.apache.org/jira/browse/YUNIKORN-704 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-703) During recovery no nodes are added to the scheduler cache

2021-06-09 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-703: --- Summary: During recovery no nodes are added to the scheduler cache Key: YUNIKORN-703 URL: https://issues.apache.org/jira/browse/YUNIKORN-703 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-696) Helm chart does not support upgrade

2021-06-08 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-696: --- Summary: Helm chart does not support upgrade Key: YUNIKORN-696 URL: https://issues.apache.org/jira/browse/YUNIKORN-696 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-689) Pods could be scheduled on nodes that don't have enough CPUs

2021-05-25 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-689: --- Summary: Pods could be scheduled on nodes that don't have enough CPUs Key: YUNIKORN-689 URL: https://issues.apache.org/jira/browse/YUNIKORN-689 Project: Apache

[jira] [Created] (YUNIKORN-686) Placeholders for completed applications are created again during recovery

2021-05-22 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-686: --- Summary: Placeholders for completed applications are created again during recovery Key: YUNIKORN-686 URL: https://issues.apache.org/jira/browse/YUNIKORN-686 Project

[jira] [Created] (YUNIKORN-675) Pod status update could fail due to conflicts

2021-05-19 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-675: --- Summary: Pod status update could fail due to conflicts Key: YUNIKORN-675 URL: https://issues.apache.org/jira/browse/YUNIKORN-675 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-665) Make comments in code conform to Godoc conventions

2021-05-05 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-665: --- Summary: Make comments in code conform to Godoc conventions Key: YUNIKORN-665 URL: https://issues.apache.org/jira/browse/YUNIKORN-665 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-657) Expose reason of application failure to pods

2021-04-25 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-657: --- Summary: Expose reason of application failure to pods Key: YUNIKORN-657 URL: https://issues.apache.org/jira/browse/YUNIKORN-657 Project: Apache YuniKorn Issue

[jira] [Created] (YUNIKORN-645) metrics endpoint doesn't export any metrics

2021-04-19 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-645: --- Summary: metrics endpoint doesn't export any metrics Key: YUNIKORN-645 URL: https://issues.apache.org/jira/browse/YUNIKORN-645 Project: Apache YuniKorn Issue

Re: YuniKorn Metrics

2021-04-15 Thread Chaoran Yu
d, so that we can clearly see the performance details in any > processes. > > Hope this can help. Thanks. > > Regards, > Tao > > Chaoran Yu 于2021年4月15日周四 上午4:04写道: > >> Hello Tao, >> >> During our discussion with Wilfred yesterday, he mentioned that you folks

YuniKorn Metrics

2021-04-14 Thread Chaoran Yu
Hello Tao, During our discussion with Wilfred yesterday, he mentioned that you folks at Alibaba have been running YuniKorn at some decent scale. We are also trying some big workloads (Spark batch jobs) with YuniKorn and would like to have better visibility in terms of the scheduling performance,

[jira] [Created] (YUNIKORN-638) Make placeholder image configurable

2021-04-10 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-638: --- Summary: Make placeholder image configurable Key: YUNIKORN-638 URL: https://issues.apache.org/jira/browse/YUNIKORN-638 Project: Apache YuniKorn Issue Type

[jira] [Created] (YUNIKORN-627) The container history graph does not go down when pods are gone

2021-04-06 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-627: --- Summary: The container history graph does not go down when pods are gone Key: YUNIKORN-627 URL: https://issues.apache.org/jira/browse/YUNIKORN-627 Project: Apache

[jira] [Created] (YUNIKORN-626) The dashboard view has lags or stops responding after window resizing

2021-04-06 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-626: --- Summary: The dashboard view has lags or stops responding after window resizing Key: YUNIKORN-626 URL: https://issues.apache.org/jira/browse/YUNIKORN-626 Project

Re: [VOTE] Release Apache YuniKorn (incubating) 0.10.0

2021-03-29 Thread Chaoran Yu
+1 On my side, I verified the following areas: * Built the image from source (make push) * Installed YK (scheduler + admission controller) using the Helm chart * Fair policy: Done for various pods (not Spark) * StateAware policy without gang scheduling: Done for Spark jobs * Gang scheduling: Done

Re: [DISCUSS] Operators and applications

2021-03-29 Thread Chaoran Yu
> Based on testing that was performed around gang scheduling and the spark > > operator by Bowen Li and Chaoran Yu we found that the behaviour around > the > > operator was far from optimal. YUNIKORN-558 > > <https://issues.apache.org/jira/browse/YUNIKORN-558> wa

[jira] [Created] (YUNIKORN-600) Placeholder manager needs to initialize the orphan pods map

2021-03-24 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-600: --- Summary: Placeholder manager needs to initialize the orphan pods map Key: YUNIKORN-600 URL: https://issues.apache.org/jira/browse/YUNIKORN-600 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-589) Multiple app IDs could be generated for the same app

2021-03-21 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-589: --- Summary: Multiple app IDs could be generated for the same app Key: YUNIKORN-589 URL: https://issues.apache.org/jira/browse/YUNIKORN-589 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-588) Placeholder pods are not cleaned up timely when the Spark driver fails

2021-03-19 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-588: --- Summary: Placeholder pods are not cleaned up timely when the Spark driver fails Key: YUNIKORN-588 URL: https://issues.apache.org/jira/browse/YUNIKORN-588 Project

[jira] [Created] (YUNIKORN-587) Allocated resources on a node could become negative

2021-03-19 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-587: --- Summary: Allocated resources on a node could become negative Key: YUNIKORN-587 URL: https://issues.apache.org/jira/browse/YUNIKORN-587 Project: Apache YuniKorn

[jira] [Created] (YUNIKORN-584) The node information could become out of sync with the underlying cluster resources

2021-03-18 Thread Chaoran Yu (Jira)
Chaoran Yu created YUNIKORN-584: --- Summary: The node information could become out of sync with the underlying cluster resources Key: YUNIKORN-584 URL: https://issues.apache.org/jira/browse/YUNIKORN-584

<    1   2