[jira] [Created] (YUNIKORN-2628) fix release announcement links

[Wilfred Spiegelenburg (Jira)]

Wilfred Spiegelenburg created YUNIKORN-2628:
---

 Summary: fix release announcement links
 Key: YUNIKORN-2628
 URL: https://issues.apache.org/jira/browse/YUNIKORN-2628
 Project: Apache YuniKorn
  Issue Type: Task
  Components: website
Reporter: Wilfred Spiegelenburg
Assignee: Wilfred Spiegelenburg


In YUNIKORN-2596 a regression snuck in breaking the links to the release 
announcements.

Need to reverse that path change for the release announcements.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-2627) Add K8s 1.30 to the e2e matrix

[Wilfred Spiegelenburg (Jira)]

Wilfred Spiegelenburg created YUNIKORN-2627:
---

 Summary: Add K8s 1.30 to the e2e matrix
 Key: YUNIKORN-2627
 URL: https://issues.apache.org/jira/browse/YUNIKORN-2627
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Wilfred Spiegelenburg


k8s 1.30 support in kind is now available as part of the [0.23 
release|https://github.com/kubernetes-sigs/kind/releases/tag/v0.23.0]

Need to add 1.30 to the matrix for the next release



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn 1.5.1 RC1

[Desai, Mit]

+1 (non-binding)


  *   Built release on MacOS Sonoma (arm64)
  *   Installed locally on Kind Cluster (1.28)
  *   Successfully ran make test
  *   Ran sample sleep jobs

Thank you, Peter, for your efforts in driving the release.

- Mit Desai

From: Peter Bacsko 
Date: Friday, May 10, 2024 at 1:41 AM
To: dev@yunikorn.apache.org 
Subject: [VOTE] Release Apache YuniKorn 1.5.1 RC1
Hello everyone,

I would like to call a vote for releasing Apache YuniKorn 1.5.1 RC1.
This is a minor release which contains only bugfixes.

The release artefacts have been uploaded here:
  
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fyunikorn%2F1.5.1-RC1%2F=05%7C02%7Cmdesai%40visa.com%7C2a3124b63a9d4c5c1e0e08dc70cced61%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638509272668929112%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=DjD5Z%2BWZJwP%2Brya2vzsYf%2BMawgZ%2B57Uc6ksy6daaOLk%3D=0

My public key is located in the KEYS file:
  
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2F%2Fyunikorn%2FKEYS=05%7C02%7Cmdesai%40visa.com%7C2a3124b63a9d4c5c1e0e08dc70cced61%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638509272668939209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=bSdAxF2fZu4mbBCmWSAFCtUr3lN8Ok1j6wFG%2FjCExt8%3D=0

JIRA issues that have been resolved in this release:
   
https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fissues%2F%3Ffilter%3D12353383=05%7C02%7Cmdesai%40visa.com%7C2a3124b63a9d4c5c1e0e08dc70cced61%7C38305e12e15d4ee888b9c4db1c477d76%7C0%7C0%7C638509272668945621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=YXpRtzAMX1WVourp29T3sm6hWciTzJDOFhPtjKwNMM4%3D=0

The release solves a deadlock issue. If possible, test Yunikorn with
workloads that put Yunikorn under stress (ie. thousands/tens of thousands
of pods).

Git tags for each component are as follows:
yunikorn-scheduler-interface: v1.5.1-1
yunikorn-core: v1.5.1-1
yunikorn-k8shim: v1.5.1-1
yunikorn-web: v1.5.1-1
yunikorn-release: v1.5.1-1

Once the release is voted on and approved, all repos will be tagged
1.5.1 for consistency.

Please review and vote. The vote will be open for at least 96 hours
and closes on Tuesday 14 May 2024, 20:00:00 CEST.

[ ] +1 Approve
[ ] +0 No opinion
[ ] -1 Disapprove (and the reason why)


Thank you,
Peter

Re: [VOTE] Release Apache YuniKorn 1.5.1 RC1

[Peter Bacsko]

Thanks everyone for the testing

I'll extend the voting deadline a bit (+24hrs) because one of our community
member (Jacob Salway) wants to test it on a larger cluster with thousands
of pods.

Peter

On Tue, May 14, 2024 at 4:20 PM TingYao  wrote:

> +1 (binding)
>
> - Verified signatures and checksums
> - Verified LICENSE and NOTICE files
> - Built release on Mac Sonoma (ARM64)
>   - make image with go 1.21.8
> - Ran make test, all tests passed
> - Installed locally on Kind cluster (1.29.4)
> - Ran simple sleep jobs
>
> Wilfred Spiegelenburg  於 2024年5月14日 週二 下午12:25寫道：
>
> > +1 (binding)
> >
> > - Verified signatures and checksums
> > - Verified LICENSE and NOTICE files
> > - Verified release tarball structure
> > - Built release on Mac Sonoma (ARM64):
> >   - make image with go 1.22 and 1.21
> > - Ran make test, all tests passed
> > - Installed locally on Kind cluster (1.29)
> >
> > - REST interface checks:
> >   - verified the SHA references in the cluster detail
> >   - verified the build date is set correctly
> > - checked REST endpoints and UI
> >
> > On Fri, 10 May 2024 at 18:40, Peter Bacsko  wrote:
> > >
> > > Hello everyone,
> > >
> > > I would like to call a vote for releasing Apache YuniKorn 1.5.1 RC1.
> > > This is a minor release which contains only bugfixes.
> > >
> > > The release artefacts have been uploaded here:
> > >   https://dist.apache.org/repos/dist/dev/yunikorn/1.5.1-RC1/
> > >
> > > My public key is located in the KEYS file:
> > >   https://downloads.apache.org//yunikorn/KEYS
> > >
> > > JIRA issues that have been resolved in this release:
> > >https://issues.apache.org/jira/issues/?filter=12353383
> > >
> > > The release solves a deadlock issue. If possible, test Yunikorn with
> > > workloads that put Yunikorn under stress (ie. thousands/tens of
> thousands
> > > of pods).
> > >
> > > Git tags for each component are as follows:
> > > yunikorn-scheduler-interface: v1.5.1-1
> > > yunikorn-core: v1.5.1-1
> > > yunikorn-k8shim: v1.5.1-1
> > > yunikorn-web: v1.5.1-1
> > > yunikorn-release: v1.5.1-1
> > >
> > > Once the release is voted on and approved, all repos will be tagged
> > > 1.5.1 for consistency.
> > >
> > > Please review and vote. The vote will be open for at least 96 hours
> > > and closes on Tuesday 14 May 2024, 20:00:00 CEST.
> > >
> > > [ ] +1 Approve
> > > [ ] +0 No opinion
> > > [ ] -1 Disapprove (and the reason why)
> > >
> > >
> > > Thank you,
> > > Peter
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> > For additional commands, e-mail: dev-h...@yunikorn.apache.org
> >
> >
>

Community over Code EU 2024: The countdown has started!

[Ryan Skraba]

[Note: You're receiving this email because you are subscribed to one
or more project dev@ mailing lists at the Apache Software Foundation.]

We are very close to Community Over Code EU -- check out the amazing
program and the special discounts that we have for you.

Special discounts

You still have the opportunity to secure your ticket for Community
Over Code EU. Explore the various options available, including the
regular pass, the committer and groups pass, and now introducing the
one-day pass tailored for locals in Bratislava.

We also have a special discount for you to attend both Community Over
Code and Berlin Buzzwords from June 9th to 11th. Visit our website to
find out more about this opportunity and contact te...@sg.com.mx to
get the discount code.

Take advantage of the discounts and register now!
https://eu.communityovercode.org/tickets/

Check out the full program!

This year Community Over Code Europe will bring to you three days of
keynotes and sessions that cover topics of interest for ASF projects
and the greater open source ecosystem including data engineering,
performance engineering, search, Internet of Things (IoT) as well as
sessions with tips and lessons learned on building a healthy open
source community.

Check out the program: https://eu.communityovercode.org/program/

Keynote speaker highlights for Community Over Code Europe include:

* Dirk-Willem Van Gulik, VP of Public Policy at the Apache Software
Foundation, will discuss the Cyber Resiliency Act and its impact on
open source (All your code belongs to Policy Makers, Politicians, and
the Law).

* Dr. Sherae Daniel will share the results of her study on the impact
of self-promotion for open source software developers (To Toot or not
to Toot, that is the question).

* Asim Hussain, Executive Director of the Green Software Foundation
will present a framework they have developed for quantifying the
environmental impact of software (Doing for Sustainability what Open
Source did for Software).

* Ruth Ikegah will  discuss the growth of the open source movement in
Africa (From Local Roots to Global Impact: Building an Inclusive Open
Source Community in Africa)

* A discussion panel on EU policies and regulations affecting
specialists working in Open Source Program Offices

Additional activities

* Poster sessions: We invite you to stop by our poster area and see if
the ideas presented ignite a conversation within your team.

* BOF time: Don't miss the opportunity to discuss in person with your
open source colleagues on your shared interests.

* Participants reception: At the end of the first day, we will have a
reception at the event venue. All participants are welcome to attend!

* Spontaneous talks: There is a dedicated room and social space for
having spontaneous talks and sessions. Get ready to share with your
peers.

* Lighting talks: At the end of the event we will have the awaited
Lighting talks, where every participant is welcome to share and
enlighten us.

Please remember:  If you haven't applied for the visa, we will provide
the necessary letter for the process. In the unfortunate case of a
visa rejection, your ticket will be reimbursed.

See you in Bratislava,

Community Over Code EU Team

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

Re: [VOTE] Release Apache YuniKorn 1.5.1 RC1

[TingYao]

+1 (binding)

- Verified signatures and checksums
- Verified LICENSE and NOTICE files
- Built release on Mac Sonoma (ARM64)
  - make image with go 1.21.8
- Ran make test, all tests passed
- Installed locally on Kind cluster (1.29.4)
- Ran simple sleep jobs

Wilfred Spiegelenburg  於 2024年5月14日 週二 下午12:25寫道：

> +1 (binding)
>
> - Verified signatures and checksums
> - Verified LICENSE and NOTICE files
> - Verified release tarball structure
> - Built release on Mac Sonoma (ARM64):
>   - make image with go 1.22 and 1.21
> - Ran make test, all tests passed
> - Installed locally on Kind cluster (1.29)
>
> - REST interface checks:
>   - verified the SHA references in the cluster detail
>   - verified the build date is set correctly
> - checked REST endpoints and UI
>
> On Fri, 10 May 2024 at 18:40, Peter Bacsko  wrote:
> >
> > Hello everyone,
> >
> > I would like to call a vote for releasing Apache YuniKorn 1.5.1 RC1.
> > This is a minor release which contains only bugfixes.
> >
> > The release artefacts have been uploaded here:
> >   https://dist.apache.org/repos/dist/dev/yunikorn/1.5.1-RC1/
> >
> > My public key is located in the KEYS file:
> >   https://downloads.apache.org//yunikorn/KEYS
> >
> > JIRA issues that have been resolved in this release:
> >https://issues.apache.org/jira/issues/?filter=12353383
> >
> > The release solves a deadlock issue. If possible, test Yunikorn with
> > workloads that put Yunikorn under stress (ie. thousands/tens of thousands
> > of pods).
> >
> > Git tags for each component are as follows:
> > yunikorn-scheduler-interface: v1.5.1-1
> > yunikorn-core: v1.5.1-1
> > yunikorn-k8shim: v1.5.1-1
> > yunikorn-web: v1.5.1-1
> > yunikorn-release: v1.5.1-1
> >
> > Once the release is voted on and approved, all repos will be tagged
> > 1.5.1 for consistency.
> >
> > Please review and vote. The vote will be open for at least 96 hours
> > and closes on Tuesday 14 May 2024, 20:00:00 CEST.
> >
> > [ ] +1 Approve
> > [ ] +0 No opinion
> > [ ] -1 Disapprove (and the reason why)
> >
> >
> > Thank you,
> > Peter
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
> For additional commands, e-mail: dev-h...@yunikorn.apache.org
>
>

[jira] [Resolved] (YUNIKORN-2623) Create unit tests for Clients

[Peter Bacsko (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Bacsko resolved YUNIKORN-2623.

Fix Version/s: 1.6.0
   Resolution: Fixed

> Create unit tests for Clients
> -
>
> Key: YUNIKORN-2623
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2623
> Project: Apache YuniKorn
>  Issue Type: Test
>  Components: shim - kubernetes
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>  Labels: pull-request-available
> Fix For: 1.6.0
>
>
> Follow-up on YUNIKORN-2621.
> Create proper coverage for {{{}clients.Clients{}}}. See PR comment 
> https://github.com/apache/yunikorn-k8shim/pull/838#issuecomment-2105557568.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-2626) Add flag to helm chart to disable web container

[Michael (Jira)]

Michael created YUNIKORN-2626:
-

 Summary: Add flag to helm chart to disable web container
 Key: YUNIKORN-2626
 URL: https://issues.apache.org/jira/browse/YUNIKORN-2626
 Project: Apache YuniKorn
  Issue Type: New Feature
  Components: deployment
Reporter: Michael


For our use case we only really need the admission controller and scheduler. 
The helm chart does currently not provide a way to disable deploying the web 
container and it would be great if that is possible.

Is there any reason not to disable the web container?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-2531) Create unit tests for AsyncRMCallback

[Wilfred Spiegelenburg (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-2531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wilfred Spiegelenburg resolved YUNIKORN-2531.
-
Fix Version/s: 1.6.0
   Resolution: Fixed

new tests added to the system to improve coverage

> Create unit tests for AsyncRMCallback
> -
>
> Key: YUNIKORN-2531
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2531
> Project: Apache YuniKorn
>  Issue Type: Test
>  Components: shim - kubernetes
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.6.0
>
>
> There are no unit tests for the {{AsyncRMCallback}} type in the shim 
> (scheduler_callback.go). It's tested indirectly but we have no idea about the 
> coverage or how it behaves in rare scenarios.
> At least longer methods such as {{UpdateApplication()}}, 
> {{UpdateAllocation()}} and {{UpdateNode()}} should be covered.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-2615) Remove named returns from predicate_manager.go

[Wilfred Spiegelenburg (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wilfred Spiegelenburg resolved YUNIKORN-2615.
-
Fix Version/s: 1.6.0
   Resolution: Fixed

refactor committed to master for 1.6.0

> Remove named returns from predicate_manager.go
> --
>
> Key: YUNIKORN-2615
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2615
> Project: Apache YuniKorn
>  Issue Type: Improvement
>  Components: shim - kubernetes
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Minor
>  Labels: pull-request-available
> Fix For: 1.6.0
>
>
> Predicate manager has defined named returns on some functions but does not 
> use them. They should be removed as the way they are used can cause issues 
> that are hard to debug.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-22) k8shim is hardcoded to the default partition

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved YUNIKORN-22.

Resolution: Won't Fix

see https://github.com/apache/yunikorn-k8shim/pull/293#issuecomment-2108922407

> k8shim is hardcoded to the default partition
> 
>
> Key: YUNIKORN-22
> URL: https://issues.apache.org/jira/browse/YUNIKORN-22
> Project: Apache YuniKorn
>  Issue Type: Bug
>  Components: shim - kubernetes
>Reporter: Wilfred Spiegelenburg
>Assignee: Rainie Li
>Priority: Major
>
> In the application and node code the partition is hardcoded to use the 
> DefaultPartition constant when creating new objects:
>  * application.NewApplication
>  * schedulerNode.addExistingAllocation
>  This means that in the configuration for the core we must have that same 
> partition and that we currently would not be able to create a second shim for 
> the same core as they would interfere with each other.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Resolved] (YUNIKORN-802) Supports to assign nodes to non-default partition

[Chia-Ping Tsai (Jira)]

 [ 
https://issues.apache.org/jira/browse/YUNIKORN-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved YUNIKORN-802.
-
Resolution: Won't Fix

> Supports to assign nodes to non-default partition
> -
>
> Key: YUNIKORN-802
> URL: https://issues.apache.org/jira/browse/YUNIKORN-802
> Project: Apache YuniKorn
>  Issue Type: Sub-task
>  Components: shim - kubernetes
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
>Priority: Major
>  Labels: pull-request-available
>
> see comment 
> (https://issues.apache.org/jira/browse/YUNIKORN-22?focusedCommentId=17398860=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17398860)
> Currently, all nodes are hardcode to be assigned to "default" partition. That 
> brings two disadvantages.
>  # we can't select specify nodes, which are used to execute spark job only, 
> from a cluster
>  # multi-partitions does not work since non-default partition can't get nodes
> Future works:
>  # support to change partition assignment of existent node (in this PR, the 
> update request will be skipped)
>  # support to remove existent node which had been reassigned (in this PR, 
> removing such node cause error message "Failed to update non existing node 
> ...")



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org

[jira] [Created] (YUNIKORN-2625) Refactor Clients to avoid hard-code checks

[Chia-Ping Tsai (Jira)]

Chia-Ping Tsai created YUNIKORN-2625:


 Summary: Refactor Clients to avoid hard-code checks
 Key: YUNIKORN-2625
 URL: https://issues.apache.org/jira/browse/YUNIKORN-2625
 Project: Apache YuniKorn
  Issue Type: Improvement
Reporter: Chia-Ping Tsai


 
{code:java}
if c.NodeInformer.Informer().HasSynced() &&
c.PodInformer.Informer().HasSynced() &&
c.PVCInformer.Informer().HasSynced() &&
c.PVInformer.Informer().HasSynced() &&
c.StorageInformer.Informer().HasSynced() &&
c.CSINodeInformer.Informer().HasSynced() &&
c.ConfigMapInformer.Informer().HasSynced() &&
c.NamespaceInformer.Informer().HasSynced() &&
c.PriorityClassInformer.Informer().HasSynced() {
return
} {code}
 
{code:java}
go c.NodeInformer.Informer().Run(stopCh)
go c.PodInformer.Informer().Run(stopCh)
go c.PVInformer.Informer().Run(stopCh)
go c.PVCInformer.Informer().Run(stopCh)
go c.StorageInformer.Informer().Run(stopCh)
go c.CSINodeInformer.Informer().Run(stopCh)
go c.ConfigMapInformer.Informer().Run(stopCh)
go c.NamespaceInformer.Informer().Run(stopCh)
go c.PriorityClassInformer.Informer().Run(stopCh) {code}
Instead of listing all informer manually, we should add a method to `Clients` 
to list all informers and then loop all of them. That can bring two benefits.

 
 # more readable
 # easy to write tests. We can override the `informers` method to return mock 
informer
 # avoid error-prone (we could miss one of informers in adding more helper 
methods in the future)

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org