[jira] [Resolved] (YUNIKORN-1523) add Chinese translation for release announcement 1.2

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1523.
--
Resolution: Done

> add Chinese translation for release announcement 1.2
> 
>
> Key: YUNIKORN-1523
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1523
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: documentation, release
>Reporter: Wilfred Spiegelenburg
>Assignee: Chen Yu Teng
>Priority: Major
>  Labels: pull-request-available
>
> Release announcement for the release 1.2 should be translated to zh-cn as 
> part of the release



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Closed] (YUNIKORN-1552) Emit metrics for resource usage at non-leaf queues

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu closed YUNIKORN-1552.

Resolution: Cannot Reproduce

> Emit metrics for resource usage at non-leaf queues
> --
>
> Key: YUNIKORN-1552
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1552
> Project: Apache YuniKorn
>  Issue Type: New Feature
>  Components: metrics
>        Reporter: Chaoran Yu
>Assignee: Qi Zhu
>Priority: Major
>
> YuniKorn currently has resource usage metrics for leaf queues. But sometimes 
> it will be useful to look at the resource usage at a non-leaf queue. For 
> example, for a parent queue, we want to know the total aggregate usage of all 
> its child queues and remaining quota for this parent queue. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1553) Queue max metrics are not available

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1553.
--
Resolution: Cannot Reproduce

> Queue max metrics are not available
> ---
>
> Key: YUNIKORN-1553
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1553
> Project: Apache YuniKorn
>  Issue Type: New Feature
>  Components: metrics
>        Reporter: Chaoran Yu
>Assignee: Qi Zhu
>Priority: Major
>
> We have metrics for real-time allocations of each leaf queue, but the max 
> capacity metrics are not available. Max capacity of a queue can be modified 
> via REST APIs, scheduler ConfigMap etc and the metric tracking the max value 
> should automatically reflect the change.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1553) Queue max metrics are not available

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1553:


 Summary: Queue max metrics are not available
 Key: YUNIKORN-1553
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1553
 Project: Apache YuniKorn
  Issue Type: New Feature
  Components: metrics
Reporter: Chaoran Yu


We have metrics for real-time allocations of each leaf queue, but the max 
capacity metrics are not available. Max capacity of a queue can be modified via 
REST APIs, scheduler ConfigMap etc and the metric tracking the max value should 
automatically reflect the change.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1552) Emit metrics for resource usage at non-leaf queues

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1552:


 Summary: Emit metrics for resource usage at non-leaf queues
 Key: YUNIKORN-1552
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1552
 Project: Apache YuniKorn
  Issue Type: New Feature
  Components: metrics
Reporter: Chaoran Yu


YuniKorn currently has resource usage metrics for leaf queues. But sometimes it 
will be useful to look at the resource usage at a non-leaf queue. For example, 
for a parent queue, we want to know the total aggregate usage of all its child 
queues and remaining quota for this parent queue. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1495) Update the copyright years in NOTICE files to 2023

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1495.
--
Resolution: Done

> Update the copyright years in NOTICE files to 2023
> --
>
> Key: YUNIKORN-1495
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1495
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Wilfred Spiegelenburg
>Assignee: Ting Yao,Huang
>Priority: Major
>  Labels: newbie, pull-request-available
>
> A new year is starting and we need to update the year in the NOTICE file from 
> 2022 to 2023:
>  * all 6 repos have a NOTICE file in the root
>  * release repo only: release-top-level-artifacts/NOTICE
> See YUNIKORN-1013 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1503) Chinese translation of troubleshooting

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1503.
--
Target Version: 1.2.0
Resolution: Done

> Chinese translation of troubleshooting
> --
>
> Key: YUNIKORN-1503
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1503
> Project: Apache YuniKorn
>  Issue Type: Task
>Reporter: Chen Yu Teng
>Assignee: Wu hsuang zong
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1313) update release procedure gpg key for helm signing

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1313.
--
Resolution: Fixed

> update release procedure gpg key for helm signing
> -
>
> Key: YUNIKORN-1313
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1313
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: release
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> The gpg secring requirement for the helm chart signing is not clear we should 
> pull in more of the doc from helm to explain what is needed and what to check.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1521) fix release announcement jira links

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1521.
--
Fix Version/s: 1.2.0
   Resolution: Fixed

> fix release announcement jira links
> ---
>
> Key: YUNIKORN-1521
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1521
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: documentation
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.2.0
>
>
> The releases 0.12.2 and 1.1.0 contain broken links for the jiras that are 
> fixed in the release.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1504) Chinese translation of Run MPI jobs in workload

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1504.
--
Resolution: Done

> Chinese translation of Run MPI jobs in workload
> ---
>
> Key: YUNIKORN-1504
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1504
> Project: Apache YuniKorn
>  Issue Type: Task
>Reporter: Chen Yu Teng
>Assignee: Wu hsuang zong
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1479) Yunikorn Dashboard fa-question link got broken

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1479.
--
Fix Version/s: 1.2.0
   Resolution: Fixed

> Yunikorn Dashboard fa-question link got broken
> --
>
> Key: YUNIKORN-1479
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1479
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: webapp
>Reporter: Jagadeesan A S
>Assignee: Jagadeesan A S
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.2.0
>
>
> Yunikorn Dashboard UI, Help icon link got broken
> It needs to redirect to https://yunikorn.apache.org/community/events
> but currently https://yunikorn.apache.org/community/events/sessions



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1444) contribution guideline update: jira signup procedure

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1444.
--
Resolution: Done

> contribution guideline update: jira signup procedure 
> -
>
> Key: YUNIKORN-1444
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1444
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> The public signup for Jira was disabled, this disables new people from 
> creating a Jira account, in order to prevent spamming around Jira.
> The Infra page with the details: 
> [https://infra.apache.org/jira-guidelines.html]
> Based on the email thread we'll use an email to private@ and the slack 
> channel to request a Jira account



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1355) Generic example of GPU scheduling with Yunikorn

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1355.
--
Resolution: Done

> Generic example of GPU scheduling with Yunikorn
> ---
>
> Key: YUNIKORN-1355
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1355
> Project: Apache YuniKorn
>  Issue Type: Improvement
>  Components: deployment
>Reporter: Chen, Kai-Chun
>Assignee: Chen, Kai-Chun
>Priority: Minor
>  Labels: pull-request-available
>
> [https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/gpu-sharing.html#testing-gpu-time-slicing-with-the-nvidia-gpu-operator]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1429) Support k8s ingress on helm chart for yunikorn ui

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1429.
--
Target Version: 1.2.0
Resolution: Done

> Support k8s ingress on helm chart for yunikorn ui
> -
>
> Key: YUNIKORN-1429
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1429
> Project: Apache YuniKorn
>  Issue Type: Improvement
>  Components: release
>Reporter: Wansoo Kim
>Assignee: Wansoo Kim
>Priority: Minor
>  Labels: pull-request-available
>
> Port-forwarding is required to access web ui from helm chart.  We need to 
> support `ingress` to make it accessible from outside the cluster.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1034) Add Chinese translation for performance documents

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1034.
--
Resolution: Done

> Add Chinese translation for performance documents
> -
>
> Key: YUNIKORN-1034
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1034
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: cdmikechen
>Assignee: Chen Yu Teng
>Priority: Major
>  Labels: pull-request-available
>
> Add Chinese translation for user guide documents, this is a sub task on 
> https://issues.apache.org/jira/browse/YUNIKORN-1029
> This issue include YuniKorn site performance documents.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1282) Resource metrics are not tracked correctly for all queue types

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1282:


 Summary: Resource metrics are not tracked correctly for all queue 
types
 Key: YUNIKORN-1282
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1282
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: core - common
Affects Versions: 1.0.0
Reporter: Chaoran Yu
Assignee: Chaoran Yu
 Fix For: 1.1.0


Based on the conversation on Slack: 
[https://yunikornworkspace.slack.com/archives/CL9CRJ1KM/p1660758189285209], 
currently metrics for resource usage have two issues:
* Metrics are only tracked for leaf queues. No metrics are exported for 
non-leaf queues.

* Metrics are not updated in all places when they should be updated: when 
config is updated and when nodes are updated in the cluster 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-868) Add doc to address that placeholder now supports affinity

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-868.
-
Resolution: Done

> Add doc to address that placeholder now supports affinity
> -
>
> Key: YUNIKORN-868
> URL: https://issues.apache.org/jira/browse/YUNIKORN-868
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: documentation
>Reporter: Weiwei Yang
>Assignee: Ted Lin
>Priority: Major
>  Labels: pull-request-available
>
> YUNIKORN-861 adds affinity support for GS placeholders, we need to update the 
> doc accordingly. 
> http://yunikorn.apache.org/docs/next/user_guide/gang_scheduling#how-to-define-task-groups.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1261) Helm upgrade does not detect changes in ConfigMap

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1261:


 Summary: Helm upgrade does not detect changes in ConfigMap
 Key: YUNIKORN-1261
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1261
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: release
Affects Versions: 1.0.0
Reporter: Chaoran Yu


For an existing Helm release of YuniKorn, when there's a change in the 
ConfigMap (e.g. new queue configurations) and the release is upgraded using the 
"helm upgrade" command, the ConfigMap doesn't get updated



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [DISCUSS] Building multi architecture images

[Chaoran Yu]

Hi Wilfred, thanks for sharing.
I second the proposal. We plan to run YuniKorn on ARM later this year. I’m sure 
other people in the community may be interested as well.
Providing official multi-architecture images is higher priority than enabling 
multi-stage builds.


> On Jun 15, 2022, at 18:01, Wilfred Spiegelenburg  wrote:
> 
> Hi,
> 
> By default we only publish images with the AMD64 architecture.
> YUNIKORN-725 was implemented to allow builds to at least succeed on
> both AMD64 and ARM architectures.
> 
> Building images is a bit more involved and it was briefly discussed as
> part of YUNIKORN-725, and then moved to YUNIKORN-1215 for
> implementation.
> Over the last week I have been working on getting a build working and
> generating both AMD (amd64) and ARM (arm64v8) images combined into one
> image tag.
> I have a solution that works seamlessly if:
> * Docker Desktop [1] is installed.
> * Rancher Desktop [2] is installed with "dockerd (moby)" as the
> container runtime
> The code impact will be in the Docker and Makefile for the web and
> k8shim repo. Plus a new tool in the release repo.
> 
> Both desktops are free for use in open source development. Docker
> desktop is only free within limits and might not work for some of us.
> Rancher Desktop is the always free option. The CPU impact of running
> Rancher Desktop on my local machine is slightly higher than docker
> desktop. Besides that it works the same.
> 
> The next step could be moving to multi stage docker images for the
> build [3]. That should also open up the possibility for nedrctl to be
> used for building the multi architecture images.
> 
> My proposal is to:
> * release multi architecture images for the 1.1.0 release
> * delay multi stage build to a later release
> 
> Any comments on this plan? I will raise the PRs against the different
> repos as part of YUNIKORN-1215 in the next day or so.
> 
> Wilfred
> 
> [1] https://www.docker.com/products/docker-desktop/
> [2] https://rancherdesktop.io
> [3] 
> https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> For additional commands, e-mail: dev-h...@yunikorn.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1236) add artifacthub-repo.yml file for helm

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1236.
--
Resolution: Fixed

> add artifacthub-repo.yml file for helm
> --
>
> Key: YUNIKORN-1236
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1236
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: release
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> To claim ownership of the helm charts again we need to have a 
> \{{artifacthub-repo.yml}} file. The file should contain the owner emails to 
> allow us to take back ownership of the repo.
> See [ownership 
> claim|https://github.com/artifacthub/hub/blob/master/docs/repositories.md#ownership-claim]
>  on the artifact hub.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1223) Add Chinese translation for 1.0.0 Release

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1223.
--
Resolution: Done

> Add Chinese translation for 1.0.0 Release
> -
>
> Key: YUNIKORN-1223
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1223
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: cdmikechen
>Assignee: cdmikechen
>Priority: Major
>  Labels: pull-request-available
>
> Documents changed:
>  # download.md
>  # roadmap.md
>  # 1.0.0.md



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1224) Failed to publish website due to incompatible node version

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1224:


 Summary: Failed to publish website due to incompatible node version
 Key: YUNIKORN-1224
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1224
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: website
Reporter: Chaoran Yu
Assignee: Chaoran Yu
 Fix For: 1.1.0


Website publish no longer works. Example: 
[https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:73.]
 

The error is:

 
yarn install v1.22.15 
[64|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:65]info
 No lockfile found. 
[65|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:66][1/4]
 Resolving packages... 
[66|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:67][2/4]
 Fetching packages... 
[67|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:68]error
 @docusaurus/core@2.0.0-beta.21: The engine "node" is incompatible with this 
module. Expected version ">=16.14". Got "16.13.0" 
[68|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:69]info
 Visit [https://yarnpkg.com/en/docs/cli/install] for documentation about this 
command. 
[69|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:70]error
 Found incompatible module. 
[70|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:71]The
 command '/bin/sh -c yarn install' returned a non-zero code: 1 
[71|https://github.com/apache/yunikorn-site/runs/6638395350?check_suite_focus=true#step:3:72]
 
Need to change the base image to use node 16.14 or higher



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [ANNOUNCE] Apache YuniKorn v1.0.0 release

[Chaoran Yu]

Congrats! Amazing milestone!


On Wed, May 4, 2022 at 1:45 PM Weiwei Yang  wrote:

> Congrats, and thanks for everyone's contribution!!
>
> On Wed, May 4, 2022 at 1:31 PM Wilfred Spiegelenburg 
> wrote:
>
> > Hi all,
> >
> > It gives me great pleasure to announce that the Apache YuniKorn community
> > has
> > voted to release Apache YuniKorn v1.0.0.
> >
> > Apache YuniKorn v1.0.0 is the first release for the project as an
> > Apache top level project. It also marks a major milestone as the first
> > major release.
> >
> > It contains 173 fixes and improvements. The release details, list of
> > major features and incompatible changes are on the v1.0.0 announcement
> > page [1].
> >
> > You can also download the release from the Downloads page [2],
> >
> > Many thanks to everyone who contributed to the release. This release
> > is a direct result of your great contributions.
> >
> > Wilfred
> >
> > [1] https://yunikorn.apache.org/release-announce/1.0.0
> > [2] https://yunikorn.apache.org/download
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> >
> >
>

[jira] [Resolved] (YUNIKORN-1174) Document changes from YUNIKORN-998

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1174.
--
Resolution: Done

> Document changes from YUNIKORN-998
> --
>
> Key: YUNIKORN-1174
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1174
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: documentation, website
>Reporter: Craig Condit
>Assignee: Craig Condit
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1173) Basic scheduling fails on an existing cluster

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1173.
--
Resolution: Cannot Reproduce

> Basic scheduling fails on an existing cluster
> -
>
> Key: YUNIKORN-1173
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1173
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: shim - kubernetes
>        Reporter: Chaoran Yu
>Priority: Blocker
> Attachments: logs.txt, statedump.txt
>
>
> Environment: EKS K8s 1.20. 
> K8shim built based on commit: 
> [https://github.com/apache/yunikorn-k8shim/commit/be3bb70d9757b27d0c40d446306b928c79c80a9f]
> Core version used: v0.0.0-20220325135453-73d55282f052
> After YuniKorn is deployed, I deleted one of the pods managed by K8s 
> deployment, but YK didn't schedule the new pod that's created: 
> *spo-og60-03-spark-operator-86cc7ff747-9vzxl* 
> is the name of the new pod. It's stuck in pending and its event said 
> "spark-operator/spo-og60-03-spark-operator-86cc7ff747-9vzxl is queued and 
> waiting for allocation"
> State dump and scheduler logs are attached



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1173) Basic scheduling fails on an existing cluster

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1173:


 Summary: Basic scheduling fails on an existing cluster
 Key: YUNIKORN-1173
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1173
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: shim - kubernetes
Reporter: Chaoran Yu
 Attachments: logs.txt, statedump.txt

Environment: EKS K8s 1.20. 
K8shim built based on commit: 
[https://github.com/apache/yunikorn-k8shim/commit/be3bb70d9757b27d0c40d446306b928c79c80a9f]

Core version used: v0.0.0-20220325135453-73d55282f052

After YuniKorn is deployed, I deleted one of the pods managed by K8s 
deployment, but YK didn't schedule the new pod that's created: 

*spo-og60-03-spark-operator-86cc7ff747-9vzxl* 

is the name of the new pod. It's stuck in pending and its event said 
"spark-operator/spo-og60-03-spark-operator-86cc7ff747-9vzxl is queued and 
waiting for allocation"

State dump and scheduler logs are attached



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1143) Create a troubleshooting guide

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1143:


 Summary: Create a troubleshooting guide
 Key: YUNIKORN-1143
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1143
 Project: Apache YuniKorn
  Issue Type: Task
  Components: website
Reporter: Chaoran Yu


We need to create a user-facing troubleshooting guide that includes info on:

* Common scheduling failure scenarios
* What to look for when there's an issue with the scheduler (logs, YuniKorn UI, 
etc)
* Information to persist: scheduler logs, UI screenshots, scheduler state dump, 
REST endpoint results etc)
* Next steps: common troubleshooting steps and what to do if those don't work 
(e.g. file a Jira and include the info recorded in the preceding step)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1138) rename the variables that end with infos and informations

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1138.
--
Resolution: Fixed

> rename the variables that end with infos and informations
> -
>
> Key: YUNIKORN-1138
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1138
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: core - scheduler
>Reporter: ted
>Assignee: ted
>Priority: Minor
>  Labels: newbie, pull-request-available
>
> {{informations}} and {{infos}} are not valid words.
> rename the variables that end with {{informations}} and {{infos}} by removing 
> the {{s.}}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1123) UpdateNode may cause the scheduler to crash

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1123.
--
Resolution: Fixed

> UpdateNode may cause the scheduler to crash
> ---
>
> Key: YUNIKORN-1123
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1123
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: core - scheduler
>        Reporter: Chaoran Yu
>Assignee: Manikandan R
>Priority: Critical
>  Labels: pull-request-available
>
> [https://github.com/apache/incubator-yunikorn-core/blob/master/pkg/rmproxy/rmproxy.go#L369]
>  may cause the scheduler to crash because the node.Attributes map could be 
> uninitialized. 
> Example:
>  
> {code:java}
> 2022-03-16T05:22:46.077Z INFO cache/nodes.go:216 report updated nodes to 
> scheduler {"request": 
> "nodes: action:DECOMISSION > rmID:\"mycluster\" "}
> panic: assignment to entry in nil map
> goroutine 395199 [running]:
> github.com/apache/incubator-yunikorn-core/pkg/rmproxy.(*RMProxy).UpdateNode.func1(0xc02d9fa320,
>  0xc00030)
>  
> /go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20220221055154-ff851af3b358/pkg/rmproxy/rmproxy.go:368
>  +0x11d
> created by 
> github.com/apache/incubator-yunikorn-core/pkg/rmproxy.(*RMProxy).UpdateNode
>  
> /go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20220221055154-ff851af3b358/pkg/rmproxy/rmproxy.go:364
>  +0x7a
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1123) UpdateNode may cause the scheduler to crash

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1123:


 Summary: UpdateNode may cause the scheduler to crash
 Key: YUNIKORN-1123
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1123
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: core - scheduler
Reporter: Chaoran Yu


[https://github.com/apache/incubator-yunikorn-core/blob/master/pkg/rmproxy/rmproxy.go#L369]
 may cause the scheduler to crash because the node.Attributes map could be 
uninitialized. 

Example:

 
{code:java}
2022-03-16T05:22:46.077Z INFO cache/nodes.go:216 report updated nodes to 
scheduler {"request": 
"nodes: rmID:\"mycluster\" "}
panic: assignment to entry in nil map
goroutine 395199 [running]:
github.com/apache/incubator-yunikorn-core/pkg/rmproxy.(*RMProxy).UpdateNode.func1(0xc02d9fa320,
 0xc00030)
 
/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20220221055154-ff851af3b358/pkg/rmproxy/rmproxy.go:368
 +0x11d
created by 
github.com/apache/incubator-yunikorn-core/pkg/rmproxy.(*RMProxy).UpdateNode
 
/go/pkg/mod/github.com/apache/incubator-yunikorn-core@v0.0.0-20220221055154-ff851af3b358/pkg/rmproxy/rmproxy.go:364
 +0x7a
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1096) Add Release Announcement v0.12.2 translation zh-cn

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1096.
--
Resolution: Done

> Add Release Announcement v0.12.2 translation zh-cn
> --
>
> Key: YUNIKORN-1096
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1096
> Project: Apache YuniKorn
>  Issue Type: Improvement
>  Components: website
>Reporter: cdmikechen
>Priority: Major
>  Labels: pull-request-available
>
> Add Release Announcement v0.12.2 translation zh-cn



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-1085) DaemonSet pods may fail to be scheduled on new nodes added during autoscaling

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1085:


 Summary: DaemonSet pods may fail to be scheduled on new nodes 
added during autoscaling
 Key: YUNIKORN-1085
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1085
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: shim - kubernetes
Affects Versions: 0.12.2
 Environment: Amazon EKS, K8s 1.20, Cluster Autoscaler
Reporter: Chaoran Yu


After YUNIKORN-704 was done, YuniKorn should have the same mechanism as the 
default scheduler when it comes to scheduling DaemonSet pods. That's the case 
most times in our deployments. But recently we have found that DaemonSet 
scheduling became problematic again: When K8s Cluster Autoscaler adds new nodes 
in response to pending pods in the cluster, EKS will automatically create a CNI 
DaemonSet (Amazon's container networking module), one pod on each newly created 
node. But YuniKorn could not schedule these pods successfully. There's no 
informative error messages. The default queue that these pods belong to have 
available resources too. Because they couldn't be scheduled, EKS refuses to 
mark the new nodes as ready, they then get stuck in NotReady state. This issue 
is not always reproducible, but it has happened a few times. The root cause 
needs to be further researched



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1082) Helm should honor imagePullSecrets for admission controller

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1082.
--
Resolution: Fixed

> Helm should honor imagePullSecrets for admission controller
> ---
>
> Key: YUNIKORN-1082
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1082
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: release
>Reporter: Craig Condit
>Assignee: Craig Condit
>Priority: Major
>  Labels: pull-request-available
>
> Currently, imagePullSecrets are only used for the scheduler deployment, not 
> the admission controller.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1081) Don't expose admission controller externally by default

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1081.
--
Resolution: Fixed

> Don't expose admission controller externally by default
> ---
>
> Key: YUNIKORN-1081
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1081
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: release
>Reporter: Craig Condit
>Assignee: Craig Condit
>Priority: Major
>  Labels: pull-request-available
>
> In our helm charts, we currently define the admission controller service as a 
> LoadBalancer. This should be ClusterIP instead.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1046) Fix website footer

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1046.
--
Resolution: Fixed

> Fix website footer
> --
>
> Key: YUNIKORN-1046
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1046
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: newbie, pull-request-available
>
> The footer of the website contains a couple of issues:
>  * links to only 4 of the 6 repositories (release and web site)
>  * points to old blogs (release of 0.8)
>  * points to roster while we have a people page
>  * blog should be replaced with a link to the events under community 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1079) fix internal links in the release procedure

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1079.
--
Resolution: Fixed

> fix internal links in the release procedure
> ---
>
> Key: YUNIKORN-1079
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1079
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> Internal links in the release procedure document do not work after moving the 
> doc from the github repo to the website.
> The internal links in the markdown document use mixed lower and upper case 
> they should all be lower case.
> There is a difference between how the github rendering and the website 
> rendering works. The mixed case works in the github rendering not in the 
> website.
> example: 
>  * 
> [https://yunikorn.apache.org/community/release_procedure/#Create-a-Release] 
> fail
>  * 
> [https://yunikorn.apache.org/community/release_procedure/#create-a-release] 
> works
>  * 
> [https://github.com/apache/incubator-yunikorn-release/blob/branch-0.11/docs/release-procedure.md#Create-a-Release]
>  works
>  * 
> [https://github.com/apache/incubator-yunikorn-release/blob/branch-0.11/docs/release-procedure.md#create-a-release]
>  works



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1071) Update website for YuniKorn Meetup in Feb 2022

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1071.
--
Resolution: Done

> Update website for YuniKorn Meetup in Feb 2022
> --
>
> Key: YUNIKORN-1071
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1071
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Chenya Zhang
>Assignee: Chenya Zhang
>Priority: Major
>  Labels: pull-request-available
>
> Update the "Events" section of the website.
> https://yunikorn.apache.org/community/events



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1047) Move release procedure documentation from git repo to the website

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1047.
--
Resolution: Done

> Move release procedure documentation from git repo to the website
> -
>
> Key: YUNIKORN-1047
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1047
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> We have a clearly document release procedure. This procedure should be part 
> of the website and is documentation under the community not hidden in the 
> release repo.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1059) Add Chinese translation for get started documents

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1059.
--
Resolution: Done

> Add Chinese translation for get started documents
> -
>
> Key: YUNIKORN-1059
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1059
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: cdmikechen
>Assignee: cdmikechen
>Priority: Major
>  Labels: pull-request-available
>
> Add Chinese translation for get started documents, this is a sub task on 
> https://issues.apache.org/jira/browse/YUNIKORN-1029
> This issue include YuniKorn site get started documents.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: Apache YuniKorn (Incubating) - Community Graduation Vote

[Chaoran Yu]

+1.

Thanks Weiwei for putting together all the evidence showing that the
project is in good shape for graduation.

On Tue, Jan 25, 2022 at 9:43 PM Chenya Zhang 
wrote:

> +1 to graduate YuniKorn from the ASF
> incubator and become a top-level Apache project!
>
> Best,
> Chenya
>
> On Tue, Jan 25, 2022 at 9:09 PM Weiwei Yang  wrote:
>
>> Hi YuniKorn community and mentors
>>
>> Based on the discussion thread [1], after 2 years time of incubating, it
>> is
>> considered that now is a good time to graduate YuniKorn from the ASF
>> incubator and become a top-level Apache project. We have reviewed the ASF
>> project maturity model [2] and provided some assessment of the project's
>> maturity based on the guidelines. Details are included as the following. I
>> have enough reasons to believe the project has done sustainable
>> development
>> successfully in the Apache way. Please read this and add your vote by
>> replying to this email, your feedback will be much appreciated!!! Note,
>> this vote is not just for committers or PPMC members, we welcome anyone in
>> the community to vote, thanks!
>>
>> *Code, License, and Copyright*
>>
>> All code is maintained on github, under Apache 2.0 license. We have
>> reviewed all the dependencies and ensured they do not bring any license
>> issues. All the status files, license headers, and copyright are up to
>> date.
>>
>> *Release*
>>
>> The community has released 5 releases in the past 2 years, i.e v0.8, v0.9,
>> v0.10, v0,11, and v0.12. These releases were done by 5 different release
>> managers [3] and indicate the community can create releases independently.
>> We have also a well-documented release process, automated tools to help
>> new
>> release managers with the process.
>>
>> *Quality*
>>
>> The community has developed a comprehensive CI/CD pipeline as a guard of
>> the code quality. The pipeline runs per-commit license check, code-format
>> check, code-coverage check, UT, and end-to-end tests. All these are built
>> as automated github actions, new contributors can easily trigger and view
>> results when submitting patches.
>>
>> *Community*
>>
>> The community has developed an easy-to-read homepage for the project [4],
>> the website hosts all the materials related to the project including
>> versioned documentation, user docs, developer docs, design docs,
>> performance docs. It provides the top-level navigation to the software
>> download page, where links to all our previous releases. It also has the
>> pages for the new contributors on-boarding with the project, such as how
>> to
>> join community meetings, events links, etc.
>>
>> The community shows appreciation to all contributors and welcomes all
>> kinds
>> of contributions (not just for code). We have built an open, diverse
>> community and gathered many people to work together. With that, we have 41
>> unique code contributors and some non-code contributors as well. Many of
>> them have becoming to be committers and PPMC members while working with
>> the
>> community. There were 2 new mentors, 8 new committers, 4 new PPMC from 6
>> different organizations [5] added in the incubating phase. And in total,
>> the project has 6 mentors, 23 PPMC, and 29 committers from at least 14
>> different organizations. All the info are generally available on the
>> project website, including some guidelines to help people become
>> committer/PPMC member [6]. Community collaboration was done in a
>> wide-public, open manner, we leverage regular bi-weekly/weekly community
>> meetings for 2 different timezones [7] and dev/user slack channels,
>> mailing
>> lists for offline discussions.
>>
>> *Independence*
>>
>> The project was initially donated by Cloudera, but with a diverse open
>> source community, it has been operated as an independent project since it
>> entered into ASF incubator. The committers and PPMC members are a group of
>> passionate people from at least 14 different organizations, such as
>> Alibaba, Apple, Cloudera, Databricks, LinkedIn, Microsoft, Snowflake, etc.
>> The project's success is not depending on any single entity.
>>
>> [1] https://lists.apache.org/thread/dno411y59g2pcy1d3kd7s3kdjz9jw65n
>> [2]
>> https://community.apache.org/apache-way/apache-project-maturity-model.html
>>
>> [3] https://yunikorn.apache.org/community/download
>> [4] https://yunikorn.apache.org/
>> [5] https://incubator.apache.org/projects/yunikorn.html
>> [6] https://yunikorn.apache.org/community/people
>>
>> [6]
>>
>> https://docs.google.com/document/d/165gzC7uhcKc5XDWiMYSRKBiPQBy2tDtXADUPuhGlUa0
>>
>

[jira] [Created] (YUNIKORN-1042) "Why YuniKorn" is not rendered properly on mobile

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1042:


 Summary: "Why YuniKorn" is not rendered properly on mobile
 Key: YUNIKORN-1042
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1042
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: website
Reporter: Chaoran Yu
 Attachments: IMG_499E342B4680-1.jpeg

Likely due to YUNIKORN-1036, now the Why YuniKorn section is not rendered 
properly when viewed on mobile. See the attached screenshot



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-1036) Fix why yunikorn doc format on the home page

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1036.
--
Resolution: Fixed

> Fix why yunikorn doc format on the home page
> 
>
> Key: YUNIKORN-1036
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1036
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
>  Labels: pull-request-available
>
> Currently, this is an image file, not flexible to provide different language 
> versions.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE} Release Apache YuniKorn (incubating) 0.12.2 RC3

[Chaoran Yu]

+1 (binding)

- Built images from source
- Deployed using the Helm chart on a k8s cluster
- Verified basic pod scheduling and Spark scheduling
- Verified that the last-minute bug in RC2 has been fixed.

Thanks Craig for the quick effort!
Chaoran


On Fri, Jan 21, 2022 at 12:12 PM Sunil Govindan  wrote:

> +1 (binding)
>
> - Built from source
> - Deployed on local k8s cluster
> - Ran basic jobs
> - Verified checksum and signature
> - UI looks good.
>
> Thanks
> Sunil
>
> On Fri, Jan 21, 2022 at 10:23 AM Craig Condit 
> wrote:
>
> > Hello everyone,
> >
> > I’d like to call a vote for releasing Apache YuniKorn (incubating) 0.12.2
> > RC3.
> >
> > The release artifacts have been uploaded here:
> > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.2-rc3/ <
> > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.2-rc3/>
> >
> > My public key is located here:
> > https://downloads.apache.org/incubator/yunikorn/KEYS <
> > https://downloads.apache.org/incubator/yunikorn/KEYS>
> >
> > JIRA issues that have been resolved in this release:
> > https://issues.apache.org/jira/issues/?filter=12351270 <
> > https://issues.apache.org/jira/issues/?filter=12351270>
> >
> > Git tags for each component are as follows:
> >
> > incubator-yunikorn-scheduler-interface: v0.12.2-1
> > incubator-yunikorn-core: v0.12.2-1
> > incubator-yunikorn-k8shim: v0.12.2-3
> > incubator-yunikorn-web: v0.12.2-1
> > https://github.com/apache/incubator-yunikorn-release: v0.12.2-3
> >
> > One the release is voted on and approved, all repos will be tagged 0.12.2
> > for consistency.
> >
> > Please review and vote. The vote will be open for at least 72 hours and
> > closes on Tuesday, January 25 2022, 1pm PDT.
> >
> > [ ] +1 Approve
> > [ ] +0 No opinion
> > [ ] -1 Disapprove (and the reason why)
> >
> >
> > Thank you,
> > Craig
>

Re: [RESULT] [VOTE] Release Apache YuniKorn (incubating) 0.12.2 RC2

[Chaoran Yu]

Yes, Helm install and upgrade both work.
The failure scenario is as follows:

1. Both the admission controller and the scheduler pods are running
2. The scheduler pod is restarted for some reason (e.g. deleted, evicted, or 
crashed)
3. The new scheduler pod will be stuck in the pending state because it’s 
intercepted by the admission controller (The schedulerName field is yunikorn).

I think this bug is critical because if the scheduler pod fails for any reason, 
someone has to manually redeploy the whole thing.


> On Jan 20, 2022, at 21:45, Weiwei Yang  wrote:
> 
> Hmmm. that is a bug. But during the release verification, I have tried the
> helm install, and that works as expected. I am guessing that is because the
> scheduler always gets started first. Maybe the same for the upgrade? In
> this case, maybe this can work as long as people are using helm charts to
> deploy yunikorn? Craig, could you please look into this and let us know if
> we need to revoke the vote for 0.12.2 and have a 0.12.3?
> 
> Thank you Chaoran to raise this up. Much appreciated!
> 
> On Thu, Jan 20, 2022 at 5:00 PM Chaoran Yu  wrote:
> 
>> I just spotted a bug https://issues.apache.org/jira/browse/YUNIKORN-1038.
>> which is critical and worth porting back into branch 0.12
>> 
>> On Thu, Jan 20, 2022 at 12:12 PM Sunil Govindan  wrote:
>> 
>>> A late +1 (binding) from me.
>>> 
>>> I build this from source
>>> - Ran basic spark job
>>> - Verified UI
>>> - Checked signature.
>>> - Checked the images.
>>> 
>>> Thanks
>>> Sunil
>>> 
>>> On Wed, Jan 19, 2022 at 8:44 AM Craig Condit 
>>> wrote:
>>> 
>>>> Hi all,
>>>> 
>>>> The vote to Release Apache YuniKorn (incubating) 0.12.2 RC2 has passed
>>>> with 3 binding +1 votes and 3 non-binding +1 votes.
>>>> 
>>>> Vote thread:
>>>> https://lists.apache.org/thread/1gw0k0g5fy86r8ljnjttdco04w7z5j4j <
>>>> https://lists.apache.org/thread/1gw0k0g5fy86r8ljnjttdco04w7z5j4j>
>>>> 
>>>> Thank you to all the members who helped verify this release. We will
>> move
>>>> to IPMC voting shortly.
>>>> 
>>>> 
>>>> Thanks,
>>>> Craig
>>>> 
>>>> 
>>>> 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [RESULT] [VOTE] Release Apache YuniKorn (incubating) 0.12.2 RC2

[Chaoran Yu]

I just spotted a bug https://issues.apache.org/jira/browse/YUNIKORN-1038.
which is critical and worth porting back into branch 0.12

On Thu, Jan 20, 2022 at 12:12 PM Sunil Govindan  wrote:

> A late +1 (binding) from me.
>
> I build this from source
> - Ran basic spark job
> - Verified UI
> - Checked signature.
> - Checked the images.
>
> Thanks
> Sunil
>
> On Wed, Jan 19, 2022 at 8:44 AM Craig Condit 
> wrote:
>
> > Hi all,
> >
> > The vote to Release Apache YuniKorn (incubating) 0.12.2 RC2 has passed
> > with 3 binding +1 votes and 3 non-binding +1 votes.
> >
> > Vote thread:
> > https://lists.apache.org/thread/1gw0k0g5fy86r8ljnjttdco04w7z5j4j <
> > https://lists.apache.org/thread/1gw0k0g5fy86r8ljnjttdco04w7z5j4j>
> >
> > Thank you to all the members who helped verify this release. We will move
> > to IPMC voting shortly.
> >
> >
> > Thanks,
> > Craig
> >
> >
> >
>

[jira] [Created] (YUNIKORN-1038) Admission controller does not ignore the YuniKorn scheduler pod

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-1038:


 Summary: Admission controller does not ignore the YuniKorn 
scheduler pod
 Key: YUNIKORN-1038
 URL: https://issues.apache.org/jira/browse/YUNIKORN-1038
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: shim - kubernetes
Affects Versions: 0.12.2
Reporter: Chaoran Yu
 Fix For: 1.0.0


The admission controller currently intercepts the YuniKorn scheduler pod just 
like other pods. This shouldn't happen because YuniKorn won't there to schedule 
itself when it's restarting. This is caused by not returning the value at this 
line 
[https://github.com/apache/incubator-yunikorn-k8shim/blob/v0.12.2-1/pkg/plugin/admissioncontrollers/webhook/admission_controller.go#L127]
 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.2 RC2

[Chaoran Yu]

+1 (binding)

Checked the following:
- Ran unit tests
- Built images from source
- Installed using the Helm chart
- Ran Spark jobs
- Checked scheduler logs
- Checked UI


On Sat, Jan 15, 2022 at 11:38 AM 陳宥騰  wrote:

> +1
> - Run unit test(core 、 k8shim)
> - Make image(core、k8shim、web)
> - Deploy yunikorn 0.12.2 on k8s 1.20.10
> - Check web UI
> - Check log
> - Run some examples in k8shim/deployments like sleep、namespace、nginx
> - Check pprof function like debug/pprof/profile、heap
>
>
> Chenya Zhang  於 2022年1月15日 週六 上午1:51寫道：
>
> > +1
> >
> > - Build from source
> > - Run unit tests
> > - Run sample workloads
> > - Checked logs and K8s messages
> > - Verified the Web UI
> > - Verified REST endpoints
> > - Checked no unexpected files
> >
> > Best,
> > Chenya
> >
> > On Fri, Jan 14, 2022 at 1:00 AM Peter Bacsko  wrote:
> >
> > > +1
> > >
> > > - Verified signature & hash
> > > - Built images with make
> > > - Installed to KIND cluster
> > > - Executed batch sleep job
> > > - Inspected logs
> > > - Checked some REST endpoints
> > >
> > > Cheers,
> > > Peter
> > >
> > > On Fri, Jan 14, 2022 at 7:49 AM Weiwei Yang  wrote:
> > >
> > > > +1
> > > >
> > > > Verified the following things:
> > > >
> > > >- Checked NOTICE, DISCLAIMER, README, and CHANGELOG
> > > >- Verified signature
> > > >- Build from source using make
> > > >- Run unit test using make test
> > > >- Run some sample workloads, roughly went over the logs and k8s
> > > messages
> > > >- Verified the basic UI functionalities
> > > >- Inspected the docker images and verified the commit SHAs
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Thu, Jan 13, 2022 at 4:53 PM Wilfred Spiegelenburg <
> > > wilfr...@apache.org
> > > > >
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Checked the following:
> > > > > * no unexpected binaries or files
> > > > > * signature confirmed
> > > > > * hash confirmed
> > > > > * build from source
> > > > > * startup and run the tests
> > > > > * deployed into kind 1.22.4 using helm charts and local build
> > > > > * checked REST calls: logLevel,  healthcheck, and statedump
> > > > > * checked Web UI
> > > > >
> > > > > On Fri, 14 Jan 2022 at 05:45, Craig Condit  >
> > > > wrote:
> > > > > >
> > > > > > Hello everyone,
> > > > > >
> > > > > > I’d like to call a vote for releasing Apache YuniKorn
> (incubating)
> > > > > 0.12.2 RC2.
> > > > > >
> > > > > > The release artifacts have been uploaded here:
> > > > >
> > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.2-rc2/
> > > <
> > > > >
> > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.2-rc2/>
> > > > > >
> > > > > > My public key is located here:
> > > > > https://downloads.apache.org/incubator/yunikorn/KEYS <
> > > > > https://downloads.apache.org/incubator/yunikorn/KEYS>
> > > > > >
> > > > > > JIRA issues that have been resolved in this release:
> > > > > https://issues.apache.org/jira/issues/?filter=12351270 <
> > > > > https://issues.apache.org/jira/issues/?filter=12351270>
> > > > > >
> > > > > > Since only changes to the release repo were necessary for RC2,
> tags
> > > > > remain at 0.12.2-1 in code, only the release repo has been tagged
> > > > 0.12.2-2.
> > > > > Post-release, all repos will be tagged 0.12.2 for consistency.
> > > > > >
> > > > > > Please review and vote. The vote will be open for at least 72
> hours
> > > and
> > > > > closes on Tuesday, January 18 2022, 1pm PDT.
> > > > > >
> > > > > > [ ] +1 Approve
> > > > > > [ ] +0 No opinion
> > > > > > [ ] -1 Disapprove (and the reason why)
> > > > > >
> > > > > >
> > > > > > Thank you,
> > > > > > Craig
> > > > > >
> > > > >
> > > > >
> -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New committer: Yu Teng Chen

[Chaoran Yu]

Congrats! Look forward to seeing more amazing work from you!


On Wed, Jan 12, 2022 at 5:35 PM Wei-Chiu Chuang  wrote:

> Great! Congrats!
>
> Weiwei Yang 於 2022年1月13日 週四，上午9:28寫道：
>
> > Yu Teng, congrats!
> > Hope to work with you more : )
> >
> > On Wed, Jan 12, 2022 at 4:40 PM Craig Condit 
> > wrote:
> >
> > > Congratulations Yu Teng, and welcome to the team!
> > >
> > > Craig
> > >
> > >
> > > > On Jan 12, 2022, at 6:35 PM, Wilfred Spiegelenburg <
> > wilfr...@apache.org>
> > > wrote:
> > > >
> > > > The Project Management Committee (PMC) for Apache YuniKorn has
> invited
> > > > Yu Teng to
> > > > become a committer and we are pleased to announce that he has
> accepted.
> > > > Please join me in congratulating him.
> > > >
> > > > Congratulations & Welcome aboard Yu Teng!
> > > >
> > > > Wilfred
> > > > on behalf of The Apache YuniKorn PPMC
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > >
> > >
> >
>

Re: [DISCUSS] v0.12.2 release to allow K8s 1.22 and K8s 1.23 deployments

[Chaoran Yu]

+1 because it unblocks some users.

On Tue, Jan 11, 2022 at 4:35 PM Sunil Govindan  wrote:

> I think this will help our customers who are stuck with K8s 1.22 issue
> before 1.0 is out.
>
> +1 to the proposal
>
> Thanks
> Sunil
>
> On Tue, Jan 11, 2022 at 4:11 PM Wilfred Spiegelenburg  >
> wrote:
>
> > Over the last weeks Craig and I have been working on getting the
> > admission controller deployment updated. These changes were committed
> > yesterday as YUNIKORN-941. The main goal was to move away from old K8s
> > objects and API calls and remove the scripts for certificate creation.
> >
> > With the changes committed deployments and e2e tests against K8s1.22
> > and K8s 1.23 are no longer failing.
> > I would like to propose that we release v0.12.2 with just the changes
> > for the admission controller to allow deployments on the latest
> > versions of K8s. Since Craig is the person that has been driving this
> > code change I would like to propose him as the release manager.
> >
> > This is the list of changes related to YUNIKORN-941. It includes a
> > number of other jiras as they are all fixed by the new deployment:
> > * YUNIKORN-995
> > * YUNIKORN-938
> > * YUNIKORN-947
> > * YUNIKORN-625
> > * YUNIKORN-726
> > It has one jira it dependents on: YUNIKORN-928
> >
> > Wilfred
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> >
> >
>

[jira] [Resolved] (YUNIKORN-1006) Address QU 30, 40, 50

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-1006.
--
Resolution: Done

> Address QU 30, 40, 50
> -
>
> Key: YUNIKORN-1006
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1006
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Major
>  Labels: pull-request-available
>
> QU30
> The project provides a well-documented, secure and private channel to report 
> security issues, along with a documented way of responding to them.
> QU40
> The project puts a high priority on backwards compatibility and aims to 
> document any incompatible changes and provide tools and documentation to help 
> users transition to new features.
> CO50
> The project documents how contributors can earn more rights such as commit 
> access or decision power, and applies these principles consistently.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: Apache YuniKorn (Incubating) - Community Graduation Vote

[Chaoran Yu]

I agree. It makes PR reviews really hard.
Is this the JIRA for it https://issues.apache.org/jira/browse/YUNIKORN-583?

On Mon, Jan 10, 2022 at 7:04 PM Sunil Govindan  wrote:

> Yes, Chenya.
> I found that very troubling as well.
>
> We have to clean that up. Thanks for chiming in.
>
> Thanks
> Sunil
>
> On Mon, Jan 10, 2022 at 6:57 PM Chenya Zhang 
> wrote:
>
>> 
>>
>> Another thing in my mind is our PR code check... :D
>>
>> The "check warning" from Codecov is everywhere which makes our PR review
>> inconvenient. I'm not sure if the ASF board will randomly check our PRs and
>> find it not ideal.
>>
>> If there is no ticket tracking this, I could help to create one and do
>> some investigation there.
>>
>> Thanks,
>> Chenya
>>
>> On Mon, Jan 10, 2022 at 12:49 PM Sunil Govindan 
>> wrote:
>>
>>> I created YUNIKORN-1007
>>> 
>>>
>>> We have to look at LC20 more closely given the public artifacts that we
>>> have.
>>>
>>> + @Wilfred Spiegelenburg 
>>>
>>> Thanks
>>> Sunil
>>>
>>> On Mon, Jan 10, 2022 at 11:59 AM Chenya Zhang <
>>> chenyazhangche...@gmail.com>
>>> wrote:
>>>
>>> > Thanks Weiwei for creating the tickets! I will help to work on
>>> > https://issues.apache.org/jira/browse/YUNIKORN-1006 to address QU 30,
>>> 40,
>>> > 50.
>>> >
>>> > On Mon, Jan 10, 2022 at 10:21 AM Weiwei Yang  wrote:
>>> >
>>> >> I think we can add some documents to clearly address QU 30, 40, 50.
>>> >> I have created a task under YUNIKORN-1005
>>> >>  to address
>>> them.
>>> >> Thank you Chenya, Holden for your feedback, please comment more if
>>> there
>>> >> is
>>> >> anything else outstanding.
>>> >>
>>> >> On Mon, Jan 10, 2022 at 9:45 AM Holden Karau 
>>> >> wrote:
>>> >>
>>> >> > For "The project provides a well-documented, secure and private
>>> channel
>>> >> to
>>> >> > report security issues, along with a documented way of responding to
>>> >> them.'
>>> >> > the standard that I've seen used is to tell people to e-mail
>>> private@
>>> >> > when they think they might have a security related issue. I think
>>> that
>>> >> > would probably work well for Yunikorn too.
>>> >> >
>>> >> >
>>> >> > On Mon, Jan 10, 2022 at 7:04 AM Chenya Zhang <
>>> >> chenyazhangche...@gmail.com>
>>> >> > wrote:
>>> >> >
>>> >> >> Hi Weiwei,
>>> >> >>
>>> >> >> Thanks for driving this! The evaluation is quite comprehensive
>>> >> overall. I
>>> >> >> checked our Apache project maturity guidelines and noticed the
>>> below
>>> >> three
>>> >> >> items. Not sure if we already have them but they are not blockers
>>> to
>>> >> our
>>> >> >> graduation. We could think more about them along the way.
>>> >> >>
>>> >> >> QU30
>>> >> >>
>>> >> >> The project provides a well-documented, secure and private channel
>>> to
>>> >> >> report security issues, along with a documented way of responding
>>> to
>>> >> them.
>>> >> >>
>>> >> >> QU40
>>> >> >>
>>> >> >> The project puts a high priority on backwards compatibility and
>>> aims to
>>> >> >> document any incompatible changes and provide tools and
>>> documentation
>>> >> to
>>> >> >> help users transition to new features.
>>> >> >>
>>> >> >> CO50
>>> >> >>
>>> >> >> The project documents how contributors can earn more rights such as
>>> >> >> commit access or decision power, and applies these principles
>>> >> consistently.
>>> >> >>
>>> >> >>
>>> >> >> Thanks,
>>> >> >>
>>> >> >> Chenya
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Mon, Jan 10, 2022 at 12:00 AM Weiwei Yang 
>>> wrote:
>>> >> >>
>>> >> >>> Hi YuniKorn community and mentors
>>> >> >>>
>>> >> >>> Based on the discussion thread [1], after 2 years time of
>>> incubating,
>>> >> it
>>> >> >>> is
>>> >> >>> considered that now is a good time to graduate YuniKorn from the
>>> ASF
>>> >> >>> incubator and become a top-level Apache project. We have reviewed
>>> the
>>> >> ASF
>>> >> >>> project maturity model [2] and provided some assessment of the
>>> >> project's
>>> >> >>> maturity based on the guidelines. Details are included as the
>>> >> following.
>>> >> >>> Please read this and share your thoughts by replying to this
>>> email,
>>> >> your
>>> >> >>> feedback will be much appreciated!!!
>>> >> >>>
>>> >> >>> *Code, License, and Copyright*
>>> >> >>>
>>> >> >>> All code is maintained on github, under Apache 2.0 license. We
>>> have
>>> >> >>> reviewed all the dependencies and ensured they do not bring any
>>> >> license
>>> >> >>> issues. All the status files, license headers, and copyright are
>>> up to
>>> >> >>> date.
>>> >> >>>
>>> >> >>> *Release*
>>> >> >>>
>>> >> >>> The community has released 5 releases in the past 2 years, i.e
>>> v0.8,
>>> >> >>> v0.9,
>>> >> >>> v0.10, v0,11, and v0.12. These releases were done by 5 different
>>> >> release
>>> >> >>> managers [3] and indicate the community can create releases
>>> >> >>> independently.
>>> >> >>> We have also a well-documented release process, automated tools to
>>> >> 

[jira] [Resolved] (YUNIKORN-988) The YuniKorn logo doesn't render correctly on mobile

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-988.
-
Resolution: Fixed

> The YuniKorn logo doesn't render correctly on mobile
> 
>
> Key: YUNIKORN-988
> URL: https://issues.apache.org/jira/browse/YUNIKORN-988
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: website
>        Reporter: Chaoran Yu
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: newbie, pull-request-available
> Attachments: IMG_2178.jpg, IMG_390AF0ABB151-1.jpeg, 
> site_screenshot_darkmode.jpeg
>
>
> The YuniKorn logo appears to be distorted when viewing the website from 
> mobile. Tried with multiple mobile browsers and it's the same. Please see the 
> attached screenshot.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-996) update roadmap with v1.0 plan

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-996.
-
Resolution: Done

> update roadmap with v1.0 plan
> -
>
> Key: YUNIKORN-996
> URL: https://issues.apache.org/jira/browse/YUNIKORN-996
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> update the roadmap with the outcome of this discussion:
> https://lists.apache.org/thread/b1rr6jj1sjkow53fcb0pxpq0nkqgbngh



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: Form a process of YuniKorn Improvement Proposal (YIP)

[Chaoran Yu]

This will be a great initiative to introduce more structure to how we
handle larger-scale features and improvements.
So far we have been doing things in an ad-hoc way, which tends to get less
effective as the community grows.
+1 from me.

On Wed, Jan 5, 2022 at 4:26 PM Weiwei Yang  wrote:

> Oops, got it wrong, you mean broader review, not ASF board.
> Sorry, please ignore my last comment : )
>
> On Wed, Jan 5, 2022 at 4:25 PM Weiwei Yang  wrote:
>
> > Hi Bowen
> >
> > +1
> > Having a formal process will definitely help the cross-org communication.
> > Do we need the ASF board to review this? I am not sure, usually, each
> > project committee is able to decide what is the best for the project.
> >
> > Thanks
> >
> > On Wed, Jan 5, 2022 at 4:07 PM Bowen Li  wrote:
> >
> >> Hi all,
> >>
> >> I'd like to start conversation of building a formal process of YuniKorn
> >> Improvement Proposal (YIP).
> >>
> >> (X)IP is a common approach to propose, discuss, collaborate on and
> tackle
> >> major or important changes in open source projects and communities.
> Within
> >> Apache projects, there're successful examples and adoptions like Spark
> >> (SPIP), Flink (FLIP), Kafka (KIP).
> >>
> >> Similarly, a YIP will define the following parts, including but not
> >> limited
> >> to:
> >> - what's considered a "major change" that needs a YIP
> >> - what should be included in a YIP (e.g. motivation/business
> >> justifications, use case requirements, proposed changes, API changes,
> >> migration/compatibility, rejected alternatives, etc)
> >> - who should initiate or be involved in a YIP
> >> - end-to-end process
> >>
> >> YK community has been growing, and we've seen cases where such a process
> >> can help to better facilitate communications, understanding, and
> >> collaborations within YK community.
> >>
> >> Please share your thoughts, or +1/-1. If we get a consensus this is
> good,
> >> I
> >> will submit a draft for YIP for broader review.
> >>
> >> Thanks,
> >> Bowen
> >>
> >
>

Re: [DISCUSS] Graduate YuniKorn from Apache Incubator

[Chaoran Yu]

+1 from me. I agree we can get started on the graduation process now.


> On Jan 4, 2022, at 08:39, Weiwei Yang  wrote:
> 
> Hi all
> 
> Thanks for the feedback!
> Aiming for the next board meeting sounds good to me. This thread is to
> collect feedback, so please +1 if you think we are good to get started.
> Thank you Wilfred for putting an eye on the name search and fixing the
> whimsy.
> 
> On Tue, Jan 4, 2022 at 2:14 AM Wilfred Spiegelenburg 
> wrote:
> 
>> I think that is cutting it too close. The board meeting is in 2 weeks.
>> The incubator needs to file its report a week before that on the 12th.
>> That leaves us with just a week to finish up everything.
>> We have not done all the work yet  that we need to do (administrative)
>> and we still need to go through the two voting rounds also.
>> 
>> I would feel more comfortable aiming for the next meeting which is in
>> mid February.
>> 
>> Wilfred
>> 
>> On Tue, 4 Jan 2022 at 17:27, Wei-Chiu Chuang  wrote:
>>> 
>>> I think it's the time.
>>> Do we want to propose it as part of the January board meeting report?
>>> Time's running out.
>>> 
>>> On Tue, Jan 4, 2022 at 2:08 PM Wilfred Spiegelenburg <
>> wilfr...@apache.org>
>>> wrote:
>>> 
 Graduation is not linked to any release. It is purely a project status.
 There are still a number of things that need to be completed before we
 can graduate.
 But I think we should start the discussion and the process to graduate.
 
 One of the points I cleared up today is the podling name search [1].
 I fixed the whimsy scans that we should pass before graduation via [2].
 
 Wilfred
 
 [1] https://issues.apache.org/jira/browse/PODLINGNAMESEARCH-193
 [2] https://issues.apache.org/jira/browse/YUNIKORN-856
 
 On Tue, 4 Jan 2022 at 16:45, Chenya Zhang >> 
 wrote:
> 
> Thanks Weiwei for initiating this discussion!
> 
> So happy to see all the milestones achieved by YuniKorn in the past
>> two
> years. Big Kudos to the whole community!
> 
> A few follow-up questions, do we plan to release the plug-in
>> framework
> before graduating from Apache Incubator? Any potential timeline is
>> needed
> for 1.00 release?
> 
> Thanks,
> Chenya
> 
> On Mon, Jan 3, 2022 at 9:20 PM Weiwei Yang  wrote:
> 
>> Hi, yunikorn community
>> 
>> 
>> Happy new year!! I hope you all have an excellent start in 2022.
>> 
>> 
>> YuniKorn started incubating in ASF on Jan/21/2020, and the project
>> has
 been
>> incredibly well managed and maintained by the community in almost 2
 years
>> under ASF now. There were 5 releases published in a good cadence,
>> i.e
 v0.8,
>> v0.9, v0.10, v0,11, and v0.12. The community works closely
>> together to
>> build better open-source software and is committed to success
>> together.
>> Since incubating in ASF, the project has:
>> 
>> 
>>   - Added 2 new mentors, 8 new committers, 2 new PPMC from 6
>> different
>>   organizations [1].
>>   - Published 5 releases by 5 different release managers [2]
>>   - Regular bi-weekly/weekly community meetings for 2 different
 timezones,
>>   meeting minutes recorded here [3]
>>   - Leveraged mailing list and slack channel for various offline
>>   discussions, i.e user/dev groups
>>   - The community co-operates the YouTube channel, with many
 demo/intro
>>   videos published [4]
>> 
>> Now the project has 6 mentors, 21 PPMC, and 27 committers from 14
 different
>> organizations. I have enough reasons to believe the project has
>> done
>> sustainable development successfully in the Apache way. Thanks to
>> all
 the
>> mentors, committers, release managers, contributors, and
 evangelists!!! I
>> have reviewed the Apache maturity doc, I don’t see anything that
 blocks us
>> from graduating from the incubator. Therefore, I want to raise this
>> discussion in the community, appreciate your feedback!!
>> 
>> 
>> [1] https://incubator.apache.org/projects/yunikorn.html
>> 
>> [2] https://yunikorn.apache.org/community/download
>> 
>> [3]
>> 
>> 
 
>> https://docs.google.com/document/d/165gzC7uhcKc5XDWiMYSRKBiPQBy2tDtXADUPuhGlUa0
>> 
>> [4] https://www.youtube.com/channel/UCDSJ2z-lEZcjdK27tTj_hGw
>> 
>> 
>> Thanks!
>> 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
 For additional commands, e-mail: dev-h...@yunikorn.apache.org
 
 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
>> For additional commands, e-mail: dev-h...@yunikorn.apache.org
>> 
>> 


-
To unsubscribe, e-mail: 

Re: [ANNOUNCE] New committer: Peter Bacsko

[Chaoran Yu]

Congrats Peter. Welcome aboard!


> On Jan 4, 2022, at 08:23, Weiwei Yang  wrote:
> 
> Hi Peter
> 
> Thank you! Have fun!!
> 
> On Tue, Jan 4, 2022 at 8:07 AM Peter Bacsko  wrote:
> 
>> Hi everyone,
>> 
>> thanks everyone for inviting as a committer, I'm sure we'll have some good
>> time together with YK! :)
>> 
>> Cheers,
>> Peter
>> 
>> On Tue, Jan 4, 2022 at 11:37 AM Wilfred Spiegelenburg >> 
>> wrote:
>> 
>>> The Project Management Committee (PMC) for Apache YuniKorn has invited
>>> Peter  to
>>> become a committer and we are pleased to announce that he has accepted.
>>> Please join me in congratulating him.
>>> 
>>> Congratulations & Welcome aboard Peter!
>>> 
>>> Wilfred
>>> on behalf of The Apache YuniKorn PPMC
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
>>> For additional commands, e-mail: dev-h...@yunikorn.apache.org
>>> 
>>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-828) Add YuniKorn core's queue-level capacity (guaranteed, max) metrics

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-828.
-
Resolution: Done

> Add YuniKorn core's queue-level capacity (guaranteed, max) metrics
> --
>
> Key: YUNIKORN-828
> URL: https://issues.apache.org/jira/browse/YUNIKORN-828
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: core - common
>Reporter: Chenya Zhang
>Assignee: Chenya Zhang
>Priority: Major
>  Labels: pull-request-available
>
> Queue-level capacity metrics (guaranteed resource, max resource) are not 
> implemented in code.
> Users need to adjust the capacity threshold manually from any monitoring 
> dashboard if their queue capacity changes.
> It is hard for users to evaluate and demonstrate historical usage trend. Not 
> a small amount of manual work is needed by users or dev ops.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-989) Update the copyright years in NOTICE file to 2022

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-989:
---

 Summary: Update the copyright years in NOTICE file to 2022
 Key: YUNIKORN-989
 URL: https://issues.apache.org/jira/browse/YUNIKORN-989
 Project: Apache YuniKorn
  Issue Type: Task
Reporter: Chaoran Yu
 Fix For: 1.0.0


Similar to https://issues.apache.org/jira/browse/YUNIKORN-617, we need to do it 
again but for 2022 this time



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-988) The YuniKorn logo doesn't render correctly on mobile

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-988:
---

 Summary: The YuniKorn logo doesn't render correctly on mobile
 Key: YUNIKORN-988
 URL: https://issues.apache.org/jira/browse/YUNIKORN-988
 Project: Apache YuniKorn
  Issue Type: Bug
  Components: website
Reporter: Chaoran Yu
 Attachments: IMG_2178.jpg

The YuniKorn logo appears to be distorted when viewing the website from mobile. 
Tried with multiple mobile browsers and it's the same. Please see the attached 
screenshot.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-987) remove references to Cloudera jiras

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-987.
-
Resolution: Fixed

> remove references to Cloudera jiras
> ---
>
> Key: YUNIKORN-987
> URL: https://issues.apache.org/jira/browse/YUNIKORN-987
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: test - e2e
>Reporter: Wilfred Spiegelenburg
>Assignee: Ryan Lo
>Priority: Major
>  Labels: newbie, pull-request-available
>
> Two references for internal Cloudera jiras have slipped into the Apache code 
> base.
> Both references should be removed.
> COMPX-4042 in predicates_suite_test.go (twice):
> Link to jira should be removed, comment text explains it all.
> COMPX-4041 in drip_feed_schedule_test.go:
> The test is not disabled and the comments referencing YUNIKORN-317 and 
> COMPX-4041 should be removed as they are not relevant.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: Apache YuniKorn (Incubating) 0.12.1 Released

[Chaoran Yu]

I forgot to mention that the source code and convenience images can be found at 
the Downloads page: https://yunikorn.apache.org/community/download 
<https://yunikorn.apache.org/community/download> 

The Artifact Hub https://artifacthub.io/packages/helm/yunikorn/yunikorn 
<https://artifacthub.io/packages/helm/yunikorn/yunikorn> has been updated with 
the latest Helm chart.


> On Dec 26, 2021, at 22:09, Chaoran Yu  wrote:
> 
> Hi all,
> 
> The IPMC has voted in approval of the v0.12.1 release candidate. The voting 
> result thread can be found at 
> https://lists.apache.org/thread/1h3s77jo977qjf8c3w34flj6lj307yks 
> <https://lists.apache.org/thread/1h3s77jo977qjf8c3w34flj6lj307yks>.
> 
> Now we are officially announcing that v0.12.1 is released. The release notes 
> can be found at https://yunikorn.apache.org/release-announce/0.12.1 
> <https://yunikorn.apache.org/release-announce/0.12.1>. Thanks to all the 
> folks who contributed to this release! 
> 
> Hope everyone is having a great holiday break. Let’s regroup in the new year 
> to achieve even bigger goals, among them the 1.0 release and graduation from 
> the Apache incubator!
> 
> Cheers,
> Chaoran
> 

Apache YuniKorn (Incubating) 0.12.1 Released

[Chaoran Yu]

Hi all,

The IPMC has voted in approval of the v0.12.1 release candidate. The voting 
result thread can be found at 
https://lists.apache.org/thread/1h3s77jo977qjf8c3w34flj6lj307yks 
.

Now we are officially announcing that v0.12.1 is released. The release notes 
can be found at https://yunikorn.apache.org/release-announce/0.12.1 
. Thanks to all the folks 
who contributed to this release! 

Hope everyone is having a great holiday break. Let’s regroup in the new year to 
achieve even bigger goals, among them the 1.0 release and graduation from the 
Apache incubator!

Cheers,
Chaoran

[jira] [Resolved] (YUNIKORN-891) [Umbrella] YuniKorn 0.12 release-related efforts

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-891.
-
Resolution: Done

> [Umbrella] YuniKorn 0.12 release-related efforts
> 
>
> Key: YUNIKORN-891
> URL: https://issues.apache.org/jira/browse/YUNIKORN-891
> Project: Apache YuniKorn
>  Issue Type: Task
>    Reporter: Chaoran Yu
>        Assignee: Chaoran Yu
>Priority: Major
>
> This umbrella is to track the work items needed for the 0.12.0 release.
> Release manager: Chaoran Yu.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-903) Clean up the release area on the download site

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-903.
-
Resolution: Done

> Clean up the release area on the download site
> --
>
> Key: YUNIKORN-903
> URL: https://issues.apache.org/jira/browse/YUNIKORN-903
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>    Reporter: Chaoran Yu
>        Assignee: Chaoran Yu
>Priority: Major
>
> This step should be performed after the website updates have been made as the 
> download links change.
> There should only be one release, the latest, in the release area. Any 
> release that has been in the release area will be automatically copied to the 
> archive. Older releases should be downloaded from the archive directly, not 
> from the release area.
> The releases need to clean up in two locations:
>  * Remove the newly released version from the _dev_ area by removing the old 
> release candidate directory. For the location see [release candidate 
> location|https://github.com/apache/incubator-yunikorn-release/blob/master/docs/release-procedure.md#Upload-Release-Candidate-Artefacts]
>  * Remove the non-current release from the _release_ area by removing the old 
> release directory. For the location see [release 
> location|https://github.com/apache/incubator-yunikorn-release/blob/master/docs/release-procedure.md#Publish-the-Release]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-901) Update the website for v0.12

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-901.
-
Resolution: Done

> Update the website for v0.12
> 
>
> Key: YUNIKORN-901
> URL: https://issues.apache.org/jira/browse/YUNIKORN-901
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: website
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Major
>  Labels: pull-request-available
>
> Updates include:
> * Create a new documentation version.
> * Create the release announcement.
> * Update the download page of the website.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-721) Improve YuniKorn core's queue-level and scheduler metrics

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-721.
-
Resolution: Done

> Improve YuniKorn core's queue-level and scheduler metrics
> -
>
> Key: YUNIKORN-721
> URL: https://issues.apache.org/jira/browse/YUNIKORN-721
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: core - common
>Reporter: Chenya Zhang
>Assignee: Chenya Zhang
>Priority: Major
>  Labels: pull-request-available
>
> To improve YuniKorn core's queue-level and scheduler metrics:
>  - Differentiate queue level metrics with scheduler metrics, e.g. using 
> "IncQueueApplicationsAccepted"
>  - Refactor related queue and scheduler metrics definitions
>  - Refactor related queue and scheduler metrics operation functions
>  - Update metrics naming and help messages 
>  - Update in-line comments and documentations



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: yunikorn contributor

[Chaoran Yu]

Welcome! Forking the project and starting to use it will be a good start.
Also you are welcome to attend the bi-weekly community meeting.
See https://yunikorn.apache.org/community/get_involved and
https://yunikorn.apache.org/community/how_to_contribute for details.
If you spot any bugs or missing features, filing JIRAs will be a nice
contribution. You can also get your hands dirty with open JIRAs that are
marked as newbie

.

On Thu, Dec 23, 2021 at 9:40 PM 曾喜煌 <9501...@gmail.com> wrote:

> I want to contribute to yunikorn
> apache jira user name: Tseng Hsi-Huang
>

[jira] [Resolved] (YUNIKORN-717) Refactor YuniKorn core's scheduler metrics for node number and sorting latency

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-717.
-
Resolution: Done

> Refactor YuniKorn core's scheduler metrics for node number and sorting latency
> --
>
> Key: YUNIKORN-717
> URL: https://issues.apache.org/jira/browse/YUNIKORN-717
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: core - common
>Reporter: Chenya Zhang
>Assignee: Chenya Zhang
>Priority: Major
>  Labels: pull-request-available
>
> Defining "totalNodeActive" and "totalNodeFailed" to initialize scheduler 
> metrics is redundant in code.
>  * It can be combined into "node"
>  * It can use a "prometheus.CounterVec" with different Prometheus labels
> Defining "nodeSortingLatency", "appSortingLatency", and "queueSortingLatency" 
> to initialize scheduler metrics is redundant in code.
>  * It can be combined into "sortingLatency"
>  * It can use a "prometheus.CounterVec" with different Prometheus labels



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: Observability of actually cpu/memory usage

[Chaoran Yu]

Previously when doing research on this topic, I saw that the metrics-server
documentation says:"*Metrics Server is not meant for non-autoscaling
purposes. For example, don't use it to forward metrics to monitoring
solutions, or as a source of monitoring solution metrics. In such cases
please collect metrics from Kubelet /metrics/resource endpoint directly*."
But the Kubelet APIs
that
the statement refers to are not documented, meaning they are hidden APIs
that can change or be deprecated at any future Kubernetes release.
Integrating with these APIs doesn't sound promising. But besides Kubelet,
the actual utilization info of workloads is not readily available anywhere
else. We'll need to explore other ideas.

On Tue, Dec 21, 2021 at 12:51 PM Weiwei Yang  wrote:

> Thank you Bowen to raise this up, this is an interesting topic. Bear with
> me this long reply : )
>
> Like Wilfred mentioned, YK doesn't know about the actual used resources in
> terms of CPU and memory for each pod, or application, at least not today. I
> understand the requirements about tracking this info in order to give users
> some feedback or even recommendations on how to tune their jobs more
> properly. It would be good to have something in our view as "Allocated" vs
> "Used" for each app/queue. We could further introduce some penalties if
> people keep over-requesting resources.
>
> However, most likely we will need to do this outside of YK. The major
> reason is all data YK is consuming are from api-server, backed by etcd. Non
> of such metrics will be stored in etcd, as per design in metrics-server
> . Second, YK doesn't
> have any per-node agent running that we can facilitate to collect actual
> resource usages, we still need to leverage a 3rd party tool to do so. Maybe
> we can do some integration with metrics-server, aggregating app/queue used
> info from those fragmented metrics, and then plug that into our
> yunikorn-web UI. We have the flexibility to do this I believe, which could
> be an option.
>
> On Mon, Dec 20, 2021 at 10:28 PM Wilfred Spiegelenburg <
> wilfr...@apache.org>
> wrote:
>
> > Hi Bowen,
> >
> > Maybe a strange question but what do you consider "actually
> > used" resources? Anything the scheduler sees is used. The scheduler has
> no
> > information on what the container really occupies: it asked for 100GB but
> > it only allocated 50GB etc. If you need that YuniKorn cannot help you. If
> > it is just a looking at allocation over time YuniKorn is capable of
> giving
> > you the information.
> >
> > Second point to make is that normally applications do not provide any
> > information on what they expect to use before they use it. Let's take a
> > Spark application. The driver creates pods as it needs new executors. The
> > Spark config drives those requests and the limitations. The scheduler
> only
> > sees the pods that are really requested. It does not know, and should not
> > know, if that is limited by what is configured or that the job uses only
> > part or more than what is configured.
> >
> > The only time the scheduler would have any idea about a "maximum" is
> when a
> > gang request is made. For gang scheduling we can track if the gang
> > request is completely used or not. We could add metrics for it on an
> > application. We can also track the number of containers allocated for an
> > application or queue, the time from start to finish for containers etc.
> We
> > could even track the maximum resource allocation for an application or a
> > queue over a time interval. Prometheus should give us a number of
> > possibilities, we just need to hook them into the scheduling cycle.
> >
> > As far as I know we currently do not have application metrics but that
> can
> > always be added. Some queue metrics are there already. I think one of
> those
> > is what you are looking for to fill a number of the gaps that you see. I
> > have added YUNIKORN-829 as a subtask to YUNIKORN-720 [1] which is already
> > referencing a number of metrics to improve. With the release of v0.12.1 I
> > moved that jira to v1.0.0. A major improvement to the metrics would be a
> > nice addition for the v1.0.0.
> >
> > I would not see anything that is blocking enhancing metrics: it is a part
> > that can be improved without a major impact on other functionality. We do
> > need to make sure that we measure the impact on performance and memory
> > usage.
> >
> > Wilfred
> >
> > [1] https://issues.apache.org/jira/browse/YUNIKORN-720
> >
> > On Tue, 21 Dec 2021 at 16:18, Bowen Li  wrote:
> >
> > > Hi community,
> > >
> > > Reviving https://issues.apache.org/jira/browse/YUNIKORN-829 . We are
> > > running Spark on YuniKorn, and have a requirement to provide more
> > > observability of *actual* resource usage for our customers, data
> > > engineers/scientists who wrote Spark jobs who may not have deep
> 

[jira] [Resolved] (YUNIKORN-796) Pod scheduling could disproportionately concentrate on one node

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-796.
-
  Assignee: Chaoran Yu
Resolution: Cannot Reproduce

Resolving as Cannot Reproduce for now because I haven't seen this symptom for 
some time now. I'll either re-open it or create another one with more details 
if it happens again

> Pod scheduling could disproportionately concentrate on one node
> ---
>
> Key: YUNIKORN-796
> URL: https://issues.apache.org/jira/browse/YUNIKORN-796
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: core - scheduler
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Major
>
> We've observed a few cases when YK keeps scheduling incoming pods to just one 
> node, even though the cluster has many other nodes that have abundant 
> resources. Not sure what could be causing this behavior because we don't see 
> it in every K8s cluster we have. Maybe somewhere in the node sorting 
> algorithm, a node could be favored over all other nodes under some conditions?



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-755) Update the shim repo dependencies

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-755.
-
Resolution: Invalid

Closing as Invalid because the repo has been updated by other PRs

> Update the shim repo dependencies
> -
>
> Key: YUNIKORN-755
> URL: https://issues.apache.org/jira/browse/YUNIKORN-755
> Project: Apache YuniKorn
>  Issue Type: Task
>  Components: shim - kubernetes
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
>Priority: Trivial
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[RESULT] [VOTE] Release Apache YuniKorn (incubating) 0.12.1 RC1

[Chaoran Yu]

Hi all,

The vote to release Apache YuniKorn (Incubating) 0.12.1 RC1 has passed.

We got 3 binding votes
- Weiwei Yang
- Wilfred Spiegelenburg
- Sunil Govindan

and 4 non-binding votes
- Craig Condit
- Peter Bacsko
- Chenya Zhang
- Yuteng Chen

Vote thread: https://lists.apache.org/thread/1yr4y7tftth5bfxlxfy4l8gmo3ypk4d2 
 

Thank you to all the above members to help us to verify and vote for this RC of 
the 0.12.1 release. We will move to IPMC voting shortly.

Regards,
Chaoran

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.1

[Chaoran Yu]

Thank you everyone for voting! The deadline (12/17 1pm PDT) is almost here
but we still need one more binding vote from a member of the PPMC to pass
the release.
To give more time to the PPMC members who haven't voted yet and given
the upcoming weekend, I'm extending the deadline.
Now the vote will close on *Monday 12/20 5pm PDT*.

On Wed, Dec 15, 2021 at 7:42 PM Chenya Zhang 
wrote:

> +1
>
> Verified the following:
> - Built from source
> - Docker image
> - Signature
> - Installed with Helm Charts
> - REST API, Web UI
> - Triggered simple jobs and checked scheduling, events, logs
> - Unit tests
>
> On Wed, Dec 15, 2021 at 10:13 AM Peter Bacsko  wrote:
>
> > +1 (non-binding)
> >
> > Verified the following:
> > * Signature
> > * Hash
> > * Built image locally
> > * Installed YK with helm on Minikube
> > * Run sleep batch job
> > * Checked some REST endpoints (nodes/queues/apps/statedump/loglevel)
> > * Checked YK logs
> >
> > BR,
> > Peter
> >
> > On Wed, Dec 15, 2021 at 6:58 PM Craig Condit 
> > wrote:
> >
> > > +1.
> > >
> > > Verified the following:
> > >
> > >
> > > - Verified SHA256 hash
> > > - Verified GPG signature
> > > - Ran unit and e2e tests
> > > - Built docker images
> > > - Installed via helm chart into local Minikube cluster
> > > - Tested basic functionality including gang scheduling
> > >
> > >
> > > Craig
> > >
> > >
> > > > On Dec 14, 2021, at 1:46 PM, Chaoran Yu 
> > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I'd like to call a vote for a release candidate for Apache YuniKorn
> > > (incubating) 0.12.1 release.
> > > >
> > > > The release artifacts have been uploaded to
> > > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/ <
> > > https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/>
> > > >
> > > > My public key is located here
> > > https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS <
> > > https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS>
> > > >
> > > > The release has been tagged with "v0.12.1" in all our git
> repositories.
> > > >
> > > > The JIRA issues that have been resolved in this release can be found
> > > here <
> https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843
> > >.
> > > >
> > > > Please review and vote. The vote will be open for at least 72 hours
> and
> > > > closes on Friday, December 17 2021, 1pm PDT.
> > > >
> > > > [ ] +1 approve
> > > > [ ] +0 no opinion
> > > > [ ] -1 disapprove (and the reason why)
> > > >
> > > >
> > > > Thank you,
> > > > Chaoran
> > > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> > >
> > >
> >
>

[jira] [Resolved] (YUNIKORN-900) Update helm index for v0.12

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-900.
-
Resolution: Done

> Update helm index for v0.12
> ---
>
> Key: YUNIKORN-900
> URL: https://issues.apache.org/jira/browse/YUNIKORN-900
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: release
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Major
>  Labels: pull-request-available
>




--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-898) Add 0.12 to the artifact hub

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-898.
-
Resolution: Duplicate

> Add 0.12 to the artifact hub
> 
>
> Key: YUNIKORN-898
> URL: https://issues.apache.org/jira/browse/YUNIKORN-898
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>    Reporter: Chaoran Yu
>        Assignee: Chaoran Yu
>Priority: Major
>
> We need to update our gh-pages in the release repository, so the artifacthub 
> will have the new release as well: 
> https://artifacthub.io/packages/helm/yunikorn/yunikorn



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-892) Create 0.12 helm chart release

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-892.
-
Resolution: Done

> Create 0.12 helm chart release
> --
>
> Key: YUNIKORN-892
> URL: https://issues.apache.org/jira/browse/YUNIKORN-892
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: release
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Major
>  Labels: pull-request-available
>
> Release the helm charts in yunikorn-release repository with following steps:
>  * create a new branch: branch-0.12
>  * update version to 0.12 in helm-charts/yunikorn/Chart.yaml
>  * tag v0.12.0 (after the release notes have been written)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-893) Create 0.12 release notes

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-893.
-
Resolution: Done

> Create 0.12 release notes
> -
>
> Key: YUNIKORN-893
> URL: https://issues.apache.org/jira/browse/YUNIKORN-893
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: release
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Major
>
> Release notes should be updated in CHANGELOG file which will be added into 
> the release artifacts.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.1

[Chaoran Yu]

Thanks guys for the feedback. Not sure how the public key and files like 
hpa.yaml ended up in the chart. I suspect I did something wrong with SVN.
The README issue was because a commit that went into master didn’t go into 
branch-0.12. I’ve fixed that.
Now reported issues should have been fixed. Since no actual code changes 
occurred, let’s keep voting on the same RC. Please review again. 

Thanks,
Chaoran



> On Dec 14, 2021, at 16:35, Wilfred Spiegelenburg  wrote:
> 
> -1 based on the helm chart issues:
> * unknown files in the helm chart (publickey and yaml files)
> * helm README content is not correct
> 
> I was able to:
> * check signature
> * check hash
> * build from source
> * startup and run the tests
> * manually deployed image
> 
> On Wed, 15 Dec 2021 at 10:34, Craig Condit  wrote:
> 
>> Seeing a few issues with the archive:
>> 
>> 1) there’s a stale public.key file in the helm-charts dir, which should
>> probably not be in the archive.
>> 2) The helm charts that are present don’t seem to correspond with any
>> known version I can see. The differ significantly with both master and
>> branch-0.12.1 upstream.
>> 3) helm-charts/yunikorn/README.md seems to be missing the updated K8s
>> versions (probably a missing commit somewhere).
>> 
>> 
>> Craig
>> 
>>> On Dec 14, 2021, at 4:05 PM, Craig Condit 
>> wrote:
>>> 
>>> Was able to verify this time. However, I think the archive needs to be
>> regenerated as the helm chart key is still the old one (not @apache.org).
>>> 
>>> 
>>>> On Dec 14, 2021, at 3:40 PM, Chaoran Yu 
>> wrote:
>>>> 
>>>> Thanks Craig for catching that! I’ve re-signed all the release
>> artifacts using the key corresponding to my Apache email. New artifacts
>> have been uploaded to the same location. My key has been uploaded to the
>> MIT key server as well.
>>>> 
>>>> 
>>>>> On Dec 14, 2021, at 12:45, Craig Condit > <mailto:apa...@craigcondit.com>> wrote:
>>>>> 
>>>>> Hi Chaoran,
>>>>> 
>>>>> Thanks for putting all this together.
>>>>> 
>>>>> I’m trying to verify the release, but am running into some issues with
>> the signing key. Did you upload it to pgpkeys.mit.edu <
>> http://pgpkeys.mit.edu/><http://pgpkeys.mit.edu/ <http://pgpkeys.mit.edu/>>?
>> Also, it seems that it is not an @apache.org address. AFAIK, that is
>> required?
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Craig
>>>>> 
>>>>> 
>>>>>> On Dec 14, 2021, at 1:58 PM, Chaoran Yu 
>> wrote:
>>>>>> 
>>>>>> My public key is at
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS <
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS>. Sorry
>> about the wrong link
>>>>>> 
>>>>>> 
>>>>>>> On Dec 14, 2021, at 11:46, Chaoran Yu 
>> wrote:
>>>>>>> 
>>>>>>> Hi all,
>>>>>>> 
>>>>>>> I'd like to call a vote for a release candidate for Apache YuniKorn
>> (incubating) 0.12.1 release.
>>>>>>> 
>>>>>>> The release artifacts have been uploaded to
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/ <
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/>
>>>>>>> 
>>>>>>> My public key is located here
>> https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS <
>> https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS>
>>>>>>> 
>>>>>>> The release has been tagged with "v0.12.1" in all our git
>> repositories.
>>>>>>> 
>>>>>>> The JIRA issues that have been resolved in this release can be found
>> here <https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843>.
>>>>>>> 
>>>>>>> Please review and vote. The vote will be open for at least 72 hours
>> and
>>>>>>> closes on Friday, December 17 2021, 1pm PDT.
>>>>>>> 
>>>>>>> [ ] +1 approve
>>>>>>> [ ] +0 no opinion
>>>>>>> [ ] -1 disapprove (and the reason why)
>>>>>>> 
>>>>>>> 
>>>>>>> Thank you,
>>>>>>> Chaoran
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> -
>>>> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org > dev-unsubscr...@yunikorn.apache.org>
>>>> For additional commands, e-mail: dev-h...@yunikorn.apache.org > dev-h...@yunikorn.apache.org>
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
>> For additional commands, e-mail: dev-h...@yunikorn.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.1

[Chaoran Yu]

Thanks Craig for catching that! I’ve re-signed all the release artifacts using 
the key corresponding to my Apache email. New artifacts have been uploaded to 
the same location. My key has been uploaded to the MIT key server as well.


> On Dec 14, 2021, at 12:45, Craig Condit  wrote:
> 
> Hi Chaoran,
> 
> Thanks for putting all this together.
> 
> I’m trying to verify the release, but am running into some issues with the 
> signing key. Did you upload it to pgpkeys.mit.edu <http://pgpkeys.mit.edu/>? 
> Also, it seems that it is not an @apache.org address. AFAIK, that is required?
> 
> 
> Thanks,
> 
> Craig
> 
> 
>> On Dec 14, 2021, at 1:58 PM, Chaoran Yu  wrote:
>> 
>> My public key is at 
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS 
>> <https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS>. Sorry 
>> about the wrong link
>> 
>> 
>>> On Dec 14, 2021, at 11:46, Chaoran Yu  wrote:
>>> 
>>> Hi all,
>>> 
>>> I'd like to call a vote for a release candidate for Apache YuniKorn 
>>> (incubating) 0.12.1 release.
>>> 
>>> The release artifacts have been uploaded to 
>>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/ 
>>> <https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/>
>>> 
>>> My public key is located here 
>>> https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS 
>>> <https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS>
>>> 
>>> The release has been tagged with "v0.12.1" in all our git repositories.
>>> 
>>> The JIRA issues that have been resolved in this release can be found here 
>>> <https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843>.
>>> 
>>> Please review and vote. The vote will be open for at least 72 hours and
>>> closes on Friday, December 17 2021, 1pm PDT.
>>> 
>>> [ ] +1 approve
>>> [ ] +0 no opinion
>>> [ ] -1 disapprove (and the reason why)
>>> 
>>> 
>>> Thank you,
>>> Chaoran
>>> 
>> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.1

[Chaoran Yu]

My public key is at 
https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS 
<https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS>. Sorry about 
the wrong link


> On Dec 14, 2021, at 11:46, Chaoran Yu  wrote:
> 
> Hi all,
> 
> I'd like to call a vote for a release candidate for Apache YuniKorn 
> (incubating) 0.12.1 release.
> 
> The release artifacts have been uploaded to 
> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/ 
> <https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12.1/>
> 
> My public key is located here 
> https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS 
> <https://dist.apache.org/repos/dist/release/incubator/yunikorn/KEYS>
> 
> The release has been tagged with "v0.12.1" in all our git repositories.
> 
> The JIRA issues that have been resolved in this release can be found here 
> <https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843>.
> 
> Please review and vote. The vote will be open for at least 72 hours and
> closes on Friday, December 17 2021, 1pm PDT.
> 
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and the reason why)
> 
> 
> Thank you,
> Chaoran
> 

Re: Renaming v0.12.0 to v0.12.1

[Chaoran Yu]

Now that we have reached a consensus, I’ll go ahead with a 0.12.1 release. I’ve 
created YUNIKORN-980 <https://issues.apache.org/jira/browse/YUNIKORN-980> to 
track release process improvement that will be used for 1.0.0.
When I make the website changes to announce 0.12.1, I’ll briefly explain the 
reason we skipped 0.12.0


> On Dec 13, 2021, at 17:20, Weiwei Yang  wrote:
> 
> Just had a discussion with Craig about this. Given this is not fixable, I
> am +1 with having a 0.12.1 release.
> We somehow need to explain why 0.12 isn't there on our download page and
> remind people not to use this tag.
> 
> On Mon, Dec 13, 2021 at 4:54 PM Craig Condit  wrote:
> 
>> Weiwei,
>> 
>> 0.12.0 is burned.
>> 
>> If a user attempts to depend on core@v0.12.0 at this point, they are now
>> getting bits that have a critical deadlock in them, and broken code. Worse,
>> this is invisible to them, as the GitHub tag does not reflect the same bits
>> that they will get when compiling.
>> 
>> We cannot use 0.12.0 any more at this point.
>> 
>> 
>> Craig
>> 
>> 
>>> On Dec 13, 2021, at 6:49 PM, Weiwei Yang  wrote:
>>> 
>>> We should not have a 0.12.1 without a 0.12.0 release, I think that is
>>> against the release convention.
>>> I don't think we need to worry too much about the tags, we can tag it
>> after
>>> the RC passed all the votes.
>>> Because the dependency in our release binary is locally resolved, what
>> was
>>> in the go mod file did not really matter that much. If you look at the
>> tool
>>> script here
>>> <
>> https://github.com/apache/incubator-yunikorn-release/blob/d689df755035a5b9f7b61bb31d5c8726f28dfd8a/tools/build-release.py#L189-L195
>>> ,
>>> because we do a "replace" to make sure the dependencies to the (SI, core)
>>> are pointing to the local files.
>>> 
>>> On Mon, Dec 13, 2021 at 4:47 PM Wilfred Spiegelenburg <
>> wilfr...@apache.org>
>>> wrote:
>>> 
>>>> Hi Chaoran,
>>>> 
>>>> Craig and I have been discussing the same thing while looking at the
>> build.
>>>> Tags are considered immutable when they are created as per [1]
>>>> When you look at the sum database info at [2] it is almost instant. So
>> yes
>>>> we get to a new schema. I also think that option 3 is the best option
>> to go
>>>> to in the future.
>>>> 
>>>> For this release I think we need to move to v0.12.1
>>>> 
>>>> Wilfred
>>>> 
>>>> [1] https://github.com/golang/go/issues/33969
>>>> [2] https://sum.golang.org
>>>> 
>>>> On Tue, 14 Dec 2021 at 11:20, Craig Condit 
>> wrote:
>>>> 
>>>>> +1. I also think we need to have a discussion on how to handle RC
>> builds
>>>>> in the future given the existence of sum.golang.org <
>>>>> http://sum.golang.org/>.
>>>>> 
>>>>> Some possible approaches (using 1.0.0 as an example version):
>>>>> 
>>>>> 1) Re-tag everything on rc build, and just ship RCs as 1.0.0, 1.0.1,
>>>>> 1.0.2, etc.
>>>>> Pro: Reproducible builds
>>>>> Con: Users may be confused about the stability of any given build.
>>>>> 
>>>>> 2) Tag builds with 1.0.0-rc0, 1.0.0-rc1, etc. Once voting completes,
>>>>> re-tag each component as 1.0.0 (but do NOT update go.mod/go.sum, since
>>>> the
>>>>> binaries cannot be changed per Apache policy).
>>>>> Pro: Reproducible, still provides “final” tag for end user use
>>>>> Con: Builds will contain references to -rc{x} dependencies
>>>>> 
>>>>> 3) Same as #2, but use a build number scheme (1.0.0-1, 1.0.0-2, etc.),
>>>> and
>>>>> then re-tag as 1.0.0 after release.
>>>>> Pro: Reproducible, still providing final tag for end user use
>>>>> Con: Technically, these are still considered pre-release versions
>>>>> according to https://semver.org/ <https://semver.org/>
>>>>> 
>>>>> I don’t see a perfect solution, but #3 seems the least bad to me.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> 
>>>>>> On Dec 13, 2021, at 6:13 PM, Chaoran Yu 
>>>> wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> I’m suggesting to rename th

[jira] [Created] (YUNIKORN-980) Improve the release procedure

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-980:
---

 Summary: Improve the release procedure
 Key: YUNIKORN-980
 URL: https://issues.apache.org/jira/browse/YUNIKORN-980
 Project: Apache YuniKorn
  Issue Type: Improvement
  Components: documentation, release
Reporter: Chaoran Yu


Given the discussions at 
[https://mail-archives.apache.org/mod_mbox/yunikorn-dev/202112.mbox/%3CE828F0F4-F625-4D76-81C9-DA0ED337B744%40craigcondit.com%3E],
 finalize a new release procedure and update 
[https://github.com/apache/incubator-yunikorn-release/blob/master/docs/release-procedure.md]
   



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Renaming v0.12.0 to v0.12.1

[Chaoran Yu]

Hi all,

I’m suggesting to rename the 0.12.0 release to 0.12.1.

Last week we had a release candidate for 0.12.0 that had issues and didn’t pass 
the voting process. Now thanks to Craig Condit who promptly resolved all the 
blockers, we are ready to start the release process again. Due to YUNIKORN-896 
 that was done last week, 
v0.12.0 was a tag that was already used in the core repo. To resolve a blocker, 
a commit was rolled back from the core’s branch-0.12 branch. Now the core repo 
needs to be re-tagged and then the shim needs to point to the updated tag in 
the core. But due to a design decision 
 in the Go 
Modules system (again thanks to Craig who found it), the go.sum checksum 
records can’t be updated to point to a new commit that re-uses a previous tag. 
The consequence is that we can’t re-tag the core with v0.12.0 anymore. This is 
the reason why I’m proposing the rename the upcoming release 0.12.1.

Let me know if you have any concerns or other suggestions.

Thanks,
Chaoran

Re: [VOTE] Release Apache YuniKorn (incubating) 0.12.0

[Chaoran Yu]

Thanks Craig for summarizing the blockers. 
Given their severities, let’s resolve them first. I’ll create another set of 
artifacts after they are fixed and then call a vote again.


> On Dec 9, 2021, at 14:03, Craig Condit  wrote:
> 
> I’m -1 on this one.
> 
> I pulled down the source archive, built it and attempted to run e2e tests. 
> However, I’m seeing several issues (even after the revert of YUNIKORN-941 
> from the release repo):
> 
> 
> - Verify_Non_Matching_NodeSelector_Respected e2e test times out (cause 
> unknown)
> - Admission controller logs several TLS cert errors (seems to be caused by 
> the v1beta -> v1 CSR changes)
> - Scheduler shuts down after 10 minutes (possibly related to the health 
> checker changes)
> 
> I think we should consider each of these release blockers and re-release 0.12 
> once they are fixed. I’ve created 
> https://issues.apache.org/jira/browse/YUNIKORN-974 
> <https://issues.apache.org/jira/browse/YUNIKORN-974> to track the blockers.
> 
> Thanks,
> 
> Craig
> 
>> On Dec 9, 2021, at 1:58 AM, Chaoran Yu  wrote:
>> 
>> Hi all,
>> 
>> We now have the first release candidate for the Apache YuniKorn (incubating) 
>> 0.12.0 release.
>> 
>> The release artifacts have been uploaded to: 
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12/ 
>> <https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12/>
>> 
>> My public key has been appended to 
>> https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS 
>> <https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS>
>> 
>> The release has been tagged with "v0.12.0" in our git repos.
>> 
>> The JIRA issues that have been resolved in this release can be found at 
>> https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843 
>> <https://issues.apache.org/jira/projects/YUNIKORN/versions/12350025>
>> 
>> Please review and vote. The vote will be open for at least 72 hours and 
>> closes on Sunday, December 12 2021, 12:00 PDT).
>> [ ] +1 approve
>> [ ] +0 no opinion
>> [ ] -1 disapprove (and the reason why)
>> 
>> Thank you,
>> Chaoran
>> 
>> 
>> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[VOTE] Release Apache YuniKorn (incubating) 0.12.0

[Chaoran Yu]

Hi all,

We now have the first release candidate for the Apache YuniKorn (incubating) 
0.12.0 release.

The release artifacts have been uploaded to: 
https://dist.apache.org/repos/dist/dev/incubator/yunikorn/0.12/ 


My public key has been appended to 
https://dist.apache.org/repos/dist/dev/incubator/yunikorn/KEYS 


The release has been tagged with "v0.12.0" in our git repos.

The JIRA issues that have been resolved in this release can be found at 
https://issues.apache.org/jira/projects/YUNIKORN/versions/12350843 


Please review and vote. The vote will be open for at least 72 hours and closes 
on Sunday, December 12 2021, 12:00 PDT).
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and the reason why)

Thank you,
Chaoran

[jira] [Resolved] (YUNIKORN-896) Tag release 0.12 and update go mod files

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-896.
-
Resolution: Done

> Tag release 0.12 and update go mod files
> 
>
> Key: YUNIKORN-896
> URL: https://issues.apache.org/jira/browse/YUNIKORN-896
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: core - common, shim - kubernetes
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Blocker
>  Labels: pull-request-available
>
> After tagging for the 0.12 release the dependencies for the core and shim 
> need to be updated to point to the new tag.
> Order:
>  * tag si and web
>  * tag core and go mod update (si ref)
>  * tag shim and go mod update (core and si ref)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-902) Update the CHANGELOG and generate the 0.12 release

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-902.
-
Resolution: Done

> Update the CHANGELOG and generate the 0.12 release
> --
>
> Key: YUNIKORN-902
> URL: https://issues.apache.org/jira/browse/YUNIKORN-902
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>    Reporter: Chaoran Yu
>        Assignee: Chaoran Yu
>Priority: Major
>  Labels: pull-request-available
>
> * Go to the [releases page in 
> jira|https://issues.apache.org/jira/projects/YUNIKORN?selectedItem=com.atlassian.jira.jira-projects-plugin%3Arelease-page=released-unreleased]
>  * Click on the version that is about to be released, i.e. 0.12
>  * Click on the {{Release Notes}} link on the top of the page
>  * Click the button {{Configure Release Notes}}
>  * Select the style {{Text}} and click {{create}}
>  * Scroll to the bottom of the page and copy the content of the text area and 
> update the 
> [CHANGELOG|https://github.com/apache/incubator-yunikorn-release/blob/master/release-top-level-artifacts/CHANGELOG]
>  file.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-895) Update shim and core dependencies on master branch

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-895.
-
Resolution: Done

> Update shim and core dependencies on master branch
> --
>
> Key: YUNIKORN-895
> URL: https://issues.apache.org/jira/browse/YUNIKORN-895
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: shim - kubernetes
>        Reporter: Chaoran Yu
>    Assignee: Chaoran Yu
>Priority: Blocker
>  Labels: pull-request-available
>
> The current scheduler is building with an older release of the core.
> There are two changes that need to be made:
>  * In preparation for the v0.12 release we need to update the dependency in 
> branch-0.12 to start pointing to a core version in branch-0.12
>  * The master of the shim needs an update to point to the latest version of 
> the core master



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-899) [Helm chart] Update supported K8s versions

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-899.
-
Resolution: Done

> [Helm chart] Update supported K8s versions
> --
>
> Key: YUNIKORN-899
> URL: https://issues.apache.org/jira/browse/YUNIKORN-899
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>    Reporter: Chaoran Yu
>        Assignee: Chaoran Yu
>Priority: Major
>  Labels: newbie, pull-request-available
>
> Right now the README in the Helm chart has the following info related the 
> supported K8s versions:
> supported K8s versions
> |K8s Version|Support?|
> |-|:-:|
> |1.16.x (or earlier)|X|
> |1.17.x|√|
> |1.18.x|√|
> |1.19.x|√|
> This is not true anymore, since the shim already uses 1.20 version, and the 
> tests are running with 1.18 1.19 and 1.20.
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: 0.12 release preparation

[Chaoran Yu]

Ok that makes sense. I’ll start the process tomorrow morning my time (PDT)


> On Dec 7, 2021, at 21:56, Wilfred Spiegelenburg  wrote:
> 
> We can start the process. Create branches, prepare the jiras etc. all that 
> work will take some time and then we can still progress.
> We can work on getting this last thing in even when that is done by just 
> cherry picking into the branch.
> 
> Wilfred
> 
> On Wed, 8 Dec 2021 at 15:25, Chaoran Yu  <mailto:yuchaoran2...@gmail.com>> wrote:
> Hi Wilfred, I was planning to kick off the release after YUNIKORN-941 
> <https://issues.apache.org/jira/browse/YUNIKORN-941> is done. I marked it as 
> a blocker because the installation procedure has been revamped and a risk 
> with upgrading the scheduler addressed. 
> Do you think it’s good to leave it until 1.0?
> 
> 
>> On Dec 7, 2021, at 19:19, Wilfred Spiegelenburg > <mailto:wilfr...@apache.org>> wrote:
>> 
>> Hi Chaoran,
>> 
>> Has the release been kicked off yet? I have not seen the branches created 
>> yet.
>> If we want to get anything voted on and released just after Christmas we 
>> need to start the process of the release handling today. I do not think we 
>> can make it anymore before Christmas.
>> We need to do branch and some other preparation, move and or close jiras so 
>> we have a clean slate and build. After that we need the two vote rounds 
>> (dev@ and the IPMC) which is 2 times almost a week.
>> There might also be changes that people are holding back. Changes that 
>> should not go into v0.12.0 but are for the next release v1.0.0.
>> 
>> If you need help kicking the release off let me know.
>> Wilfred
>> 
>> On Tue, 30 Nov 2021 at 03:11, Julia Kinga Marton > <mailto:kmar...@apache.org>> wrote:
>> Hi Chaoran,
>> 
>> That sounds great, thank you!
>> 
>> Regards,
>> Kinga
>> 
>> On Mon, Nov 29, 2021 at 5:00 PM Chaoran Yu > <mailto:yuchaoran2...@gmail.com>> wrote:
>> 
>> > Hi Kinga, I agree that we don't have blockers and I can start the release
>> > process soon.
>> > I'll target a release candidate to be voted by the incubator PMC by Monday
>> > Dec 6, unless people have other ideas on the timeline, in which case let me
>> > know
>> >
>> > On Mon, Nov 29, 2021 at 4:25 AM Julia Kinga Marton > > <mailto:kmar...@apache.org>>
>> > wrote:
>> >
>> > > Hi all,
>> > >
>> > > As far as I know, we are planning to release the 0.12 version this year.
>> > In
>> > > order to meet the deadline I think we should start the preparations soon.
>> > > Most of the issues targeted for this release are already fixed. There are
>> > > some open and in progress Jiras, what we need to think if we want to
>> > > include or not, however I think none of them are blockers.
>> > >
>> > > @yuchaoran2...@gmail.com <mailto:yuchaoran2...@gmail.com> 
>> > > mailto:yuchaoran2...@gmail.com>> what do you 
>> > > think?
>> > > Can we start branching?
>> > >
>> > > Regards,
>> > > Kinga Marton
>> > >
>> >
> 

Re: 0.12 release preparation

[Chaoran Yu]

Hi Wilfred, I was planning to kick off the release after YUNIKORN-941 
<https://issues.apache.org/jira/browse/YUNIKORN-941> is done. I marked it as a 
blocker because the installation procedure has been revamped and a risk with 
upgrading the scheduler addressed. 
Do you think it’s good to leave it until 1.0?


> On Dec 7, 2021, at 19:19, Wilfred Spiegelenburg  wrote:
> 
> Hi Chaoran,
> 
> Has the release been kicked off yet? I have not seen the branches created yet.
> If we want to get anything voted on and released just after Christmas we need 
> to start the process of the release handling today. I do not think we can 
> make it anymore before Christmas.
> We need to do branch and some other preparation, move and or close jiras so 
> we have a clean slate and build. After that we need the two vote rounds (dev@ 
> and the IPMC) which is 2 times almost a week.
> There might also be changes that people are holding back. Changes that should 
> not go into v0.12.0 but are for the next release v1.0.0.
> 
> If you need help kicking the release off let me know.
> Wilfred
> 
> On Tue, 30 Nov 2021 at 03:11, Julia Kinga Marton  <mailto:kmar...@apache.org>> wrote:
> Hi Chaoran,
> 
> That sounds great, thank you!
> 
> Regards,
> Kinga
> 
> On Mon, Nov 29, 2021 at 5:00 PM Chaoran Yu  <mailto:yuchaoran2...@gmail.com>> wrote:
> 
> > Hi Kinga, I agree that we don't have blockers and I can start the release
> > process soon.
> > I'll target a release candidate to be voted by the incubator PMC by Monday
> > Dec 6, unless people have other ideas on the timeline, in which case let me
> > know
> >
> > On Mon, Nov 29, 2021 at 4:25 AM Julia Kinga Marton  > <mailto:kmar...@apache.org>>
> > wrote:
> >
> > > Hi all,
> > >
> > > As far as I know, we are planning to release the 0.12 version this year.
> > In
> > > order to meet the deadline I think we should start the preparations soon.
> > > Most of the issues targeted for this release are already fixed. There are
> > > some open and in progress Jiras, what we need to think if we want to
> > > include or not, however I think none of them are blockers.
> > >
> > > @yuchaoran2...@gmail.com <mailto:yuchaoran2...@gmail.com> 
> > > mailto:yuchaoran2...@gmail.com>> what do you 
> > > think?
> > > Can we start branching?
> > >
> > > Regards,
> > > Kinga Marton
> > >
> >

[jira] [Created] (YUNIKORN-966) Retrieve the username from the SparkApp CRD

[Chaoran Yu (Jira)]

Chaoran Yu created YUNIKORN-966:
---

 Summary: Retrieve the username from the SparkApp CRD
 Key: YUNIKORN-966
 URL: https://issues.apache.org/jira/browse/YUNIKORN-966
 Project: Apache YuniKorn
  Issue Type: Sub-task
  Components: shim - kubernetes
Reporter: Chaoran Yu


Currently the shim only looks at the pods to get the value of the label 
yunikorn.apache.org/username. When the Spark operator plugin is enabled, we 
should look at the SparkApp CRD for the label.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-964) Fix vulnerabilities reported by artifacthub

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-964.
-
Resolution: Fixed

> Fix vulnerabilities reported by artifacthub
> ---
>
> Key: YUNIKORN-964
> URL: https://issues.apache.org/jira/browse/YUNIKORN-964
> Project: Apache YuniKorn
>  Issue Type: Bug
>Reporter: Kinga Marton
>Assignee: Wilfred Spiegelenburg
>Priority: Blocker
>  Labels: pull-request-available
> Fix For: 0.12
>
>
> Artifacthub has a security report for each image. 
> We need to check and fix the reported vulnerabilities: 
> [https://artifacthub.io/packages/helm/yunikorn/yunikorn/0.11.0?modal=security-report]
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: 0.12 release preparation

[Chaoran Yu]

Hi Kinga, I agree that we don't have blockers and I can start the release
process soon.
I'll target a release candidate to be voted by the incubator PMC by Monday
Dec 6, unless people have other ideas on the timeline, in which case let me
know

On Mon, Nov 29, 2021 at 4:25 AM Julia Kinga Marton 
wrote:

> Hi all,
>
> As far as I know, we are planning to release the 0.12 version this year. In
> order to meet the deadline I think we should start the preparations soon.
> Most of the issues targeted for this release are already fixed. There are
> some open and in progress Jiras, what we need to think if we want to
> include or not, however I think none of them are blockers.
>
> @yuchaoran2...@gmail.com  what do you think?
> Can we start branching?
>
> Regards,
> Kinga Marton
>

[jira] [Resolved] (YUNIKORN-925) Add document to explain existing yunikorn metrics

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-925.
-
Resolution: Done

> Add document to explain existing yunikorn metrics
> -
>
> Key: YUNIKORN-925
> URL: https://issues.apache.org/jira/browse/YUNIKORN-925
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: core - scheduler, documentation
>Reporter: Weiwei Yang
>Assignee: Ting Yao,Huang
>Priority: Major
>  Labels: pull-request-available
>
> We need to add a section in 
> http://yunikorn.apache.org/docs/next/performance/metrics, to explain existing 
> metrics supported by yunikorn. We should list it as a table, and have all the 
> supported metrics in it, including some explanation about each of these 
> metrics.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-944) website deployment broken due to search

[Chaoran Yu (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chaoran Yu resolved YUNIKORN-944.
-
Resolution: Fixed

> website deployment broken due to search
> ---
>
> Key: YUNIKORN-944
> URL: https://issues.apache.org/jira/browse/YUNIKORN-944
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: website
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>  Labels: pull-request-available
>
> The changes for the website search were only added to the deployment build 
> and not to the local build script \{{./local-build.sh}}.
> Builds that pass locally do not pass when run through the git hub deploy 
> action.
> Major version changes are required to update the build process and get it to 
> pass with the search:
> {code:java}
> error @docusaurus/theme-search-algolia@2.0.0-beta.9: The engine "node" is 
> incompatible with this module. Expected version ">=14". Got "12.18.0"
> 104info Visit https://yarnpkg.com/en/docs/cli/install for documentation about 
> this command.
> 105error Found incompatible module. {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [ANNOUNCE] New committer: Craig Condit

[Chaoran Yu]

Congrats Craig. Well deserved!


On Sun, Nov 14, 2021 at 4:44 PM Wilfred Spiegelenburg 
wrote:

> The Project Management Committee (PMC) for Apache YuniKorn has invited
> Craig to
> become a committer and we are pleased to announce that he has accepted.
> Please join me in congratulating him.
>
> Congratulations & Welcome aboard Craig!
>
> Wilfred
> on behalf of The Apache YuniKorn PPMC
>

Re: Discussion with K8s Sig Team

[Chaoran Yu]

Maybe a more focused discussion with a smaller group of core sig-scheduling
folks will be better. We can consider asking them for such a meeting
earlier than 11/18, as Weiwei mentioned.
That avoids conflict with the meetup too.

On Thu, Nov 4, 2021 at 10:46 PM Weiwei Yang  wrote:

> I see.
> Let's discuss this in the next sync-up call.
> Thank you!
>
> On Thu, Nov 4, 2021 at 9:04 PM Chenya Zhang 
> wrote:
>
> > btw. We have the YK Meetup on Nov 18th 4:30 - 6:00pm. Hope it will not be
> > in conflict with the K8s Sig call.
> >
> > On Thu, Nov 4, 2021 at 5:33 PM Sunil Govindan  wrote:
> >
> > > Thanks, Chaoron.
> > >
> > >
> > > On Thu, Nov 4, 2021 at 5:23 PM Chaoran Yu 
> > wrote:
> > >
> > > > Sure, let's use my list as the agenda and finalize next Wednesday
> > > >
> > > > On Thu, Nov 4, 2021 at 3:58 PM Sunil Govindan 
> > wrote:
> > > >
> > > > > Hi Weiwei
> > > > >
> > > > > Yes. I suggest meeting next Wed inside of the YK community.
> > > > > Based on that we can take a call whether to join on 18th.
> > > > >
> > > > > Thanks, Chaoron for summing it up. And we could use that as agenda.
> > Is
> > > > that
> > > > > fine?
> > > > >
> > > > > Thanks
> > > > > Sunil
> > > > >
> > > > > On Thu, Nov 4, 2021 at 3:21 PM Weiwei Yang 
> wrote:
> > > > >
> > > > > > I think Chaoran covered most of the questions we discussed
> > yesterday.
> > > > > > Sunil, are you suggesting to meet next Wed inside of the YK
> > community
> > > > > > before setting up a call with sig-scheduling?
> > > > > > I noticed the next sig-scheduling meeting is Nov 18, we probably
> > can
> > > > try
> > > > > to
> > > > > > set up sometime earlier than that with a smaller group of people.
> > > > > >
> > > > > > On Thu, Nov 4, 2021 at 1:02 PM Chaoran Yu <
> yuchaoran2...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi Sunil,
> > > > > > >
> > > > > > > Next Wednesday at the same time works for me. From what I
> > > understand
> > > > in
> > > > > > the
> > > > > > > meeting yesterday, we want to initiate a conversation with
> > > > > sig-scheduling
> > > > > > > without committing to any actions just yet.
> > > > > > >
> > > > > > > The information we want to gather from them include the
> > following:
> > > > > > >
> > > > > > > * What's the overall process and timeline when contributing a
> > > plugin?
> > > > > > > * Besides contributing code directly in the plugins repo, can
> the
> > > > code
> > > > > > live
> > > > > > > in an outside repo (e.g. Apache repos)? Any other options?
> > > > > > > * The scheduler-plugins repo doesn't align with Kubernetes
> itself
> > > in
> > > > > > terms
> > > > > > > of release cadence nor does it match the latest k8s version.
> How
> > is
> > > > it
> > > > > > > maintained?
> > > > > > > * How does the default scheduler in the core K8s repo
> incorporate
> > > > > > > community-contributed plugins? What's the process of promoting
> a
> > > > > plugin?
> > > > > > >
> > > > > > > We can finalize the agenda when we meet next week
> > > > > > >
> > > > > > > On Thu, Nov 4, 2021 at 12:21 PM Sunil Govindan <
> > sun...@apache.org>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Folks,
> > > > > > > >
> > > > > > > > Thanks for joining our weekly sync call.
> > > > > > > >
> > > > > > > > We have an action item to discuss with the SIG team sooner.
> > Could
> > > > we
> > > > > > > > conclude the meeting agenda earlier within the YuniKorn
> > community
> > > > > > itself
> > > > > > > > before we have that call?
> > > > > > > > We can have one more YuniKorn community call next week to
> > > conclude
> > > > > > before
> > > > > > > > starting the discussion with K8s. I could help schedule this
> at
> > > the
> > > > > > same
> > > > > > > > time as yesterday (3.00 pm PST) if there are no objections.
> > > > > > > >
> > > > > > > > Please share your thoughts.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Sunil
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: YK Perf benchmark

[Chaoran Yu]

Thanks so much Tingyao and Yuteng! These results will certainly help
promote YuniKorn to newcomers and those who are hesitant in adopting it.

On Thu, Nov 4, 2021 at 10:45 PM Weiwei Yang  wrote:

> Hi all
>
> As you know or may not know, in the last 2 months, Tingyao and Yuteng are
> helping to perform a new round of perf-bench marking based on the latest
> codebase. They've done this from scratch, set up a simulated env using
> their lab's hardware, setup up metrics servers to observe and discover perf
> bottleneck, tuning the perf to a reasonably good result. I really
> appreciate their efforts to get this done.
>
> Now, they are summarizing the result and preparing to update the YK
> website with a bunch of good docs, including perf result, and benchmark
> tutorial. This will be done before the next release time frame.
>
> Again, thank you Tingyao, Yuteng, this is a great example of community
> collaboration! Well done!
>
> Thanks
> Weiwei
>

Re: Discussion with K8s Sig Team

[Chaoran Yu]

Sure, let's use my list as the agenda and finalize next Wednesday

On Thu, Nov 4, 2021 at 3:58 PM Sunil Govindan  wrote:

> Hi Weiwei
>
> Yes. I suggest meeting next Wed inside of the YK community.
> Based on that we can take a call whether to join on 18th.
>
> Thanks, Chaoron for summing it up. And we could use that as agenda. Is that
> fine?
>
> Thanks
> Sunil
>
> On Thu, Nov 4, 2021 at 3:21 PM Weiwei Yang  wrote:
>
> > I think Chaoran covered most of the questions we discussed yesterday.
> > Sunil, are you suggesting to meet next Wed inside of the YK community
> > before setting up a call with sig-scheduling?
> > I noticed the next sig-scheduling meeting is Nov 18, we probably can try
> to
> > set up sometime earlier than that with a smaller group of people.
> >
> > On Thu, Nov 4, 2021 at 1:02 PM Chaoran Yu 
> wrote:
> >
> > > Hi Sunil,
> > >
> > > Next Wednesday at the same time works for me. From what I understand in
> > the
> > > meeting yesterday, we want to initiate a conversation with
> sig-scheduling
> > > without committing to any actions just yet.
> > >
> > > The information we want to gather from them include the following:
> > >
> > > * What's the overall process and timeline when contributing a plugin?
> > > * Besides contributing code directly in the plugins repo, can the code
> > live
> > > in an outside repo (e.g. Apache repos)? Any other options?
> > > * The scheduler-plugins repo doesn't align with Kubernetes itself in
> > terms
> > > of release cadence nor does it match the latest k8s version. How is it
> > > maintained?
> > > * How does the default scheduler in the core K8s repo incorporate
> > > community-contributed plugins? What's the process of promoting a
> plugin?
> > >
> > > We can finalize the agenda when we meet next week
> > >
> > > On Thu, Nov 4, 2021 at 12:21 PM Sunil Govindan 
> > wrote:
> > >
> > > > Hi Folks,
> > > >
> > > > Thanks for joining our weekly sync call.
> > > >
> > > > We have an action item to discuss with the SIG team sooner. Could we
> > > > conclude the meeting agenda earlier within the YuniKorn community
> > itself
> > > > before we have that call?
> > > > We can have one more YuniKorn community call next week to conclude
> > before
> > > > starting the discussion with K8s. I could help schedule this at the
> > same
> > > > time as yesterday (3.00 pm PST) if there are no objections.
> > > >
> > > > Please share your thoughts.
> > > >
> > > > Thanks
> > > > Sunil
> > > >
> > >
> >
>