[jira] [Resolved] (YUNIKORN-2262) propagate the error message when queue creation gets failed
[ https://issues.apache.org/jira/browse/YUNIKORN-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2262. -- Fix Version/s: 1.6.0 Resolution: Fixed > propagate the error message when queue creation gets failed > --- > > Key: YUNIKORN-2262 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2262 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chenchen Lai >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/partition.go#L334] > the error message of root cause is swallowed, so it is hard to be inspired by > the common message "failed to create rule based queue ..." > BTW, the error I met is the parent queue "is already a leaf". The error > message is helpful and it makes us catch up the root cause easily. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2719) Assert invalid group name in Get Group REST API
[ https://issues.apache.org/jira/browse/YUNIKORN-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2719. -- Fix Version/s: 1.6.0 Resolution: Fixed > Assert invalid group name in Get Group REST API > --- > > Key: YUNIKORN-2719 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2719 > Project: Apache YuniKorn > Issue Type: Test > Components: core - scheduler >Reporter: Manikandan R >Assignee: Yun Sun >Priority: Major > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > Assert invalid group name in Get Group REST API -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2726) Add "How to check E2E test logs?" to developer guide
[
https://issues.apache.org/jira/browse/YUNIKORN-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2726.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Add "How to check E2E test logs?" to developer guide
>
>
> Key: YUNIKORN-2726
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2726
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: documentation
>Reporter: Yu-Lin Chen
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Major
> Labels: newbie, pull-request-available
> Fix For: 1.6.0
>
> Attachments: image-2024-07-06-16-39-54-365.png
>
>
> After YUNIKORN-2305 , the logs of failed E2E test are dumped to local and
> upload to Github Action Artifact. We should let new developers know how to
> retrieve them.
> We should add some explaination to developer
> guide(https://yunikorn.apache.org/docs/next/developer_guide/e2e_test), below
> should be included:
> # Where to find the local e2e test logs after `make e2e_test` failed? (In
> yunikorn-k8shim/build/e2e/\{suite}/)
> # What's logs types we have
> a. \{specName}_k8sClusterInfo.txt
> b.\{specName}_ykContainerLog.txt
> c.\{specName}_ykFullStateDump.json
> # How to download logs in Github Action (Check below screenshot in [the
> failed CI
> Link|https://github.com/apache/yunikorn-k8shim/actions/runs/9807493804])
> !image-2024-07-06-16-39-54-365.png|width=573,height=307!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2207) Update user group documentation
[ https://issues.apache.org/jira/browse/YUNIKORN-2207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2207. -- Fix Version/s: 1.6.0 Resolution: Fixed > Update user group documentation > --- > > Key: YUNIKORN-2207 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2207 > Project: Apache YuniKorn > Issue Type: Improvement > Components: documentation >Reporter: Wilfred Spiegelenburg >Assignee: Hsien-Cheng(Ryan) Huang >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > The order in the [User & Group > Resolution|https://yunikorn.apache.org/docs/user_guide/usergroup_resolution/] > documentation should be reversed: > * current handling via the admission controller > * deprecated handling via the label > We should also add a removal notice for a specific YuniKorn version of the > old label. From that release we only support the annotation. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-182) fix lint issues
[ https://issues.apache.org/jira/browse/YUNIKORN-182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-182. - Resolution: Fixed all tasks are completed. [~sophiesun] [~rich7420] [~rosalai] thanks for all your contribution! > fix lint issues > --- > > Key: YUNIKORN-182 > URL: https://issues.apache.org/jira/browse/YUNIKORN-182 > Project: Apache YuniKorn > Issue Type: Task > Components: build >Reporter: Wilfred Spiegelenburg >Assignee: Yun Sun >Priority: Minor > Labels: pull-request-available > > When we added the lint test most major issues were fixed. There are still a > lot of issues specially in tests that need to be fixed. > This is a container Jira to track that work on both the k8shim as the core > repos. > Work should be split into multiple parts (per linter?) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2729) remove `--new-from-rev` from Makefile
[ https://issues.apache.org/jira/browse/YUNIKORN-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2729. -- Fix Version/s: 1.6.0 Resolution: Fixed > remove `--new-from-rev` from Makefile > - > > Key: YUNIKORN-2729 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2729 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Huang Guan Hao >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > It is time to show the power of lint :) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2763) add the documentation of REST API for specific queue
Chia-Ping Tsai created YUNIKORN-2763: Summary: add the documentation of REST API for specific queue Key: YUNIKORN-2763 URL: https://issues.apache.org/jira/browse/YUNIKORN-2763 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai The new call will be used by e2e (see YUNIKORN-2713), and hence it is worth having the documentation. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2718) Assert invalid user name in Get User REST API
[ https://issues.apache.org/jira/browse/YUNIKORN-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2718. -- Fix Version/s: 1.6.0 Resolution: Fixed > Assert invalid user name in Get User REST API > - > > Key: YUNIKORN-2718 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2718 > Project: Apache YuniKorn > Issue Type: Test > Components: core - common >Reporter: Manikandan R >Assignee: Yun Sun >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > Assert invalid user name in Get User REST API. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2760) `make tools` should check the version of tools
Chia-Ping Tsai created YUNIKORN-2760:
Summary: `make tools` should check the version of tools
Key: YUNIKORN-2760
URL: https://issues.apache.org/jira/browse/YUNIKORN-2760
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Makefile, by default, checks only the existence of file. Hence, developers need
to remove tools folder (or call `make distclean`) manually to trigger the
installation after we update the version of tools.
However, how developers can be aware of the tools updates? Personally, I smell
fishy from the error of warning, but that could be implicit and noisy :cry
In order to fix that, I'd like to introduce the new folder structure to tools
folder: /tools/{tool_name}/{version}. That offers a unique path to each version
of tool. Developers will not miss the updates anymore.
NOTED: we need to remove the existent tool binary if there is naming conflict
in creating the new path. For example, creating /tools/golangci-lint/1.57.2
will fail if /tools/golangci-lint is a existent file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2757) Consider adding new field `resolvedMaxResource` to queue dao to show the true limit
[ https://issues.apache.org/jira/browse/YUNIKORN-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2757. -- Resolution: Won't Fix > Consider adding new field `resolvedMaxResource` to queue dao to show the true > limit > --- > > Key: YUNIKORN-2757 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2757 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Priority: Major > > The true max resources of queue is based on all parents. It could be hard to > see/understand the true "max resources" of queue by human eyes if there is a > huge queue trees. > Hence, it would be nice to add the "resolved" max resources to restful APIs. > Also, our UI can leverages the field to help users to understand which max > resource will be used by this queue -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2759) Replace %w by Errors.join
Chia-Ping Tsai created YUNIKORN-2759: Summary: Replace %w by Errors.join Key: YUNIKORN-2759 URL: https://issues.apache.org/jira/browse/YUNIKORN-2759 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai original discussion: https://issues.apache.org/jira/browse/YUNIKORN-2262 Errors.join can make the code more performant and readable -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2758) Add unit test for `ProducerStateManager#readSnapshot` and `ProducerStateManager#writeSnapshot`
Chia-Ping Tsai created YUNIKORN-2758: Summary: Add unit test for `ProducerStateManager#readSnapshot` and `ProducerStateManager#writeSnapshot` Key: YUNIKORN-2758 URL: https://issues.apache.org/jira/browse/YUNIKORN-2758 Project: Apache YuniKorn Issue Type: Test Reporter: Chia-Ping Tsai We are going to introduce generated code to `ProducerStateManager`, so it would be nice to increase the test converge for now. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2757) Consider adding new field `resolvedMaxResource` to queue dao to show the true limit
Chia-Ping Tsai created YUNIKORN-2757: Summary: Consider adding new field `resolvedMaxResource` to queue dao to show the true limit Key: YUNIKORN-2757 URL: https://issues.apache.org/jira/browse/YUNIKORN-2757 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai The true max resources of queue is based on all parents. It could be hard to see/understand the true "max resources" of queue by human eyes if there is a huge queue trees. Hence, it would be nice to add the "resolved" max resources to restful APIs. Also, our UI can leverages the field to help users to understand which max resource will be used by this queue -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2756) Consider moving event_system#defaultEventChannelSize to configs#const
Chia-Ping Tsai created YUNIKORN-2756: Summary: Consider moving event_system#defaultEventChannelSize to configs#const Key: YUNIKORN-2756 URL: https://issues.apache.org/jira/browse/YUNIKORN-2756 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai All other event-related configs are in configs#const, so we should make them together. BTW, `defaultRingBufferSize` will be removed by https://github.com/apache/yunikorn-core/pull/915, since its replacement is in configs#const already. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2727) Fix Dead Links and Update readme for Docusaurus v3
[ https://issues.apache.org/jira/browse/YUNIKORN-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2727. -- Fix Version/s: 1.6.0 Resolution: Fixed > Fix Dead Links and Update readme for Docusaurus v3 > -- > > Key: YUNIKORN-2727 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2727 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Hsien-Cheng(Ryan) Huang >Assignee: Hsien-Cheng(Ryan) Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > Issue 1: Dead Link in "Deploy the Scheduler" > Problem: Dead link at example in "Deploy the Scheduler" section. > Current: > https://yunikorn.apache.org/docs/developer_guide/deployment/#deploy-the-admission-controller > Solution: Replace with correct links: > https://yunikorn.apache.org/docs/next/developer_guide/deployment/#deploy-the-scheduler > https://yunikorn.apache.org/docs/next/developer_guide/deployment/#Deploy-the-Scheduler > Cause: Migration to Docusaurus v3 with strict URL regulations. > Issue 2: Outdated Docusaurus Version in README > Problem: README mentions Docusaurus v2. > Current: "The website is built based using docusaurus-v2." > Solution: Update to v3. > New: "The website is built base using Docusaurus v3." -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2651) Update the unchecked error for make lint warnings
[ https://issues.apache.org/jira/browse/YUNIKORN-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2651. -- Fix Version/s: 1.6.0 Resolution: Fixed k8shim has no such warnings now, so resolve this issue > Update the unchecked error for make lint warnings > - > > Key: YUNIKORN-2651 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2651 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Yun Sun >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > fix the lint about "unhandled error" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2728) Config event.RESTResponseSize should be placed under Event System Settings
[ https://issues.apache.org/jira/browse/YUNIKORN-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2728. -- Fix Version/s: 1.6.0 Resolution: Fixed > Config event.RESTResponseSize should be placed under Event System Settings > -- > > Key: YUNIKORN-2728 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2728 > Project: Apache YuniKorn > Issue Type: Improvement > Components: documentation >Reporter: Kuan Po Tseng >Assignee: Chenchen Lai >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > [https://yunikorn.apache.org/docs/next/user_guide/service_config/#eventrestresponsesize] > event.RESTResponseSize is an event-related config and should be placed under > [#event-system-settings|https://yunikorn.apache.org/docs/next/user_guide/service_config/#event-system-settings] > instead of > [#health-settings|https://yunikorn.apache.org/docs/next/user_guide/service_config/#health-settings] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2655) Cleanup REST API documentation
[
https://issues.apache.org/jira/browse/YUNIKORN-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2655.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Cleanup REST API documentation
> --
>
> Key: YUNIKORN-2655
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2655
> Project: Apache YuniKorn
> Issue Type: Task
> Components: documentation
>Reporter: Wilfred Spiegelenburg
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> The REST API documentation is not up to date with the current behaviour as it
> does not show any 400 or 404 errors returned by a number of API calls.
> The error response only shows a 500 code with the same message for each call.
> We should move to a simple list for each call showing the applicable errors
> like this:
> {code:java}
> ### Error responses
> **Code** : `400 Bad Request` (URL query is invalid, missing partition name)
> **Code** : `404 Not Found` (Partition not found)
> **Code** : `500 Internal Server Error` {code}
> Remove the error examples as they do not add any detail required
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2182) Set ReadHeaderTimeout in http server
[ https://issues.apache.org/jira/browse/YUNIKORN-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2182. -- Resolution: Fixed both k8shim and core get fixed > Set ReadHeaderTimeout in http server > > > Key: YUNIKORN-2182 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2182 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: core - common, webapp >Reporter: Wilfred Spiegelenburg >Assignee: Chenchen Lai >Priority: Major > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > Potential Slowloris Attack because ReadHeaderTimeout is not configured in the > http.Server (gosec) > We do not set ReadTimeout or ReadHeaderTimeout so we do not have a timeout at > all at the moment. > BTW: this is not important for the webtest servers we build as they are just > for our tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2729) remove `--new-from-rev` from Makefile
Chia-Ping Tsai created YUNIKORN-2729: Summary: remove `--new-from-rev` from Makefile Key: YUNIKORN-2729 URL: https://issues.apache.org/jira/browse/YUNIKORN-2729 Project: Apache YuniKorn Issue Type: Sub-task Reporter: Chia-Ping Tsai It is time to show the power of lint :) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Reopened] (YUNIKORN-2182) Set ReadHeaderTimeout in http server
[ https://issues.apache.org/jira/browse/YUNIKORN-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened YUNIKORN-2182: -- [~rosalai] Could you please file PR to fix k8shim module also? https://github.com/apache/yunikorn-k8shim/blob/b337896b4924e00b07b5dceb564b0ac100e73e48/pkg/cmd/admissioncontroller/main.go#L155 > Set ReadHeaderTimeout in http server > > > Key: YUNIKORN-2182 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2182 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: core - common, webapp >Reporter: Wilfred Spiegelenburg >Assignee: Chenchen Lai >Priority: Major > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > Potential Slowloris Attack because ReadHeaderTimeout is not configured in the > http.Server (gosec) > We do not set ReadTimeout or ReadHeaderTimeout so we do not have a timeout at > all at the moment. > BTW: this is not important for the webtest servers we build as they are just > for our tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2658) add nolint:funlen to long functions to supress the lint warnings
[ https://issues.apache.org/jira/browse/YUNIKORN-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2658. -- Fix Version/s: 1.6.0 Resolution: Fixed > add nolint:funlen to long functions to supress the lint warnings > > > Key: YUNIKORN-2658 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2658 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Huang Guan Hao >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > as title -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2695) remove core dependency pkg/common
[ https://issues.apache.org/jira/browse/YUNIKORN-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2695. -- Fix Version/s: 1.6.0 Resolution: Fixed > remove core dependency pkg/common > - > > Key: YUNIKORN-2695 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2695 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: HUAN-IU LIOU >Assignee: Chenchen Lai >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2640) Conside removing config from Clients
[ https://issues.apache.org/jira/browse/YUNIKORN-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2640. -- Fix Version/s: 1.6.0 Resolution: Fixed > Conside removing config from Clients > > > Key: YUNIKORN-2640 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2640 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chenchen Lai >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > The config (`conf.SchedulerConf`) [0] references to a global singleton object > [1][2]. Also, in the code base `clients#GetConf()` is used 3 times [3] and > `conf.GetSchedulerConf()` is used 61 times [4] > It seems to me `clients#conf` should be removed to avoid confusion. > [0] > https://github.com/apache/yunikorn-k8shim/blob/master/pkg/client/clients.go#L42C8-L42C26 > [1] > https://github.com/apache/yunikorn-k8shim/blob/6f2800f689e9e341c736a6af8cbf178a711a9423/pkg/plugin/scheduler_plugin.go#L291 > [2] > https://github.com/apache/yunikorn-k8shim/blob/6f2800f689e9e341c736a6af8cbf178a711a9423/pkg/cmd/shim/main.go#L53 > [3] > https://github.com/search?q=repo%3Aapache%2Fyunikorn-k8shim+GetConf%28%29=code > [4] > https://github.com/search?q=repo%3Aapache%2Fyunikorn-k8shim+conf.GetSchedulerConf%28%29=code -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2269) remove the USER_LABEL_KEY from docs
[
https://issues.apache.org/jira/browse/YUNIKORN-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2269.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> remove the USER_LABEL_KEY from docs
> ---
>
> Key: YUNIKORN-2269
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2269
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Chenchen Lai
>Priority: Trivial
> Fix For: 1.6.0
>
>
> core does not support USER_LABEL_KEY after YUNIKORN-1405 got merged, so we
> should remove it from docs.
> https://yunikorn.apache.org/docs/user_guide/usergroup_resolution/#using-the-yunikornapacheorgusername-label
> {quote}
> The yunikorn.apache.org/username key can be customized by overriding the
> default value using the USER_LABEL_KEYenv variable in the K8s Deployment.
> This is particularly useful in scenarios where the user label is already
> being added or if the label has to be modified for some secuirty reasons.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2304) add instruction docs of looping flaky test
[
https://issues.apache.org/jira/browse/YUNIKORN-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2304.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> add instruction docs of looping flaky test
> --
>
> Key: YUNIKORN-2304
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2304
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Tseng Hsi-Huang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> The flaky is hard to be dig-in since it fails rarely. Hence, it would be
> better to have an example of looping specify flaky in our docs. That can be
> one-line command. For instance:
> {code:java}
> I=0; while go test -run TestNoFillWithoutEventPluginRegistered ./pkg/...
> -count=1; do (( I=$I+1 )); echo "Completed loop: $I"; sleep 1; done {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Reopened] (YUNIKORN-2651) Update the unchecked error for make lint warnings
[ https://issues.apache.org/jira/browse/YUNIKORN-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened YUNIKORN-2651: -- reopen for code module. [~rich7420] please fix it :) > Update the unchecked error for make lint warnings > - > > Key: YUNIKORN-2651 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2651 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Yun Sun >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > fix the lint about "unhandled error" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2651) Update the unchecked error for make lint warnings
[ https://issues.apache.org/jira/browse/YUNIKORN-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2651. -- Fix Version/s: 1.6.0 Resolution: Fixed > Update the unchecked error for make lint warnings > - > > Key: YUNIKORN-2651 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2651 > Project: Apache YuniKorn > Issue Type: Sub-task >Reporter: Chia-Ping Tsai >Assignee: Yun Sun >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > fix the lint about "unhandled error" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2622) Some /debug/pprof/ API response tested is different from example response in docs
[ https://issues.apache.org/jira/browse/YUNIKORN-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2622. -- Fix Version/s: 1.6.0 Resolution: Fixed > Some /debug/pprof/ API response tested is different from example response in > docs > - > > Key: YUNIKORN-2622 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2622 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Hsien-Cheng(Ryan) Huang >Assignee: Hsien-Cheng(Ryan) Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > /debug/pprof/symbol > tested response on 1.5.1: num_symbols: 1 > while doc: binary > https://yunikorn.apache.org/docs/next/api/system/#success-response-9 > /debug/pprof/cmdline also: > tested response on 1.5.1: /yunikorn-scheduler > while doc: binary > https://yunikorn.apache.org/docs/next/api/system/#cmdline -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2658) add nolint:funlen to long functions to supress the lint warnings
Chia-Ping Tsai created YUNIKORN-2658: Summary: add nolint:funlen to long functions to supress the lint warnings Key: YUNIKORN-2658 URL: https://issues.apache.org/jira/browse/YUNIKORN-2658 Project: Apache YuniKorn Issue Type: Sub-task Reporter: Chia-Ping Tsai as title -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2651) Update the unchecked error for make lint warnings
Chia-Ping Tsai created YUNIKORN-2651: Summary: Update the unchecked error for make lint warnings Key: YUNIKORN-2651 URL: https://issues.apache.org/jira/browse/YUNIKORN-2651 Project: Apache YuniKorn Issue Type: Sub-task Reporter: Chia-Ping Tsai fix the lint about "unhandled error" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2650) Complete or remove web_server_test#TestProxy
Chia-Ping Tsai created YUNIKORN-2650: Summary: Complete or remove web_server_test#TestProxy Key: YUNIKORN-2650 URL: https://issues.apache.org/jira/browse/YUNIKORN-2650 Project: Apache YuniKorn Issue Type: Test Reporter: Chia-Ping Tsai web_server_test has a empty test case: TestProxy [0]. It seems to me there is proxy-related test [1]. [0] https://github.com/apache/yunikorn-k8shim/blob/58adfe941d2d8dae5544af8b49e435f304678807/pkg/webtest/web_server_test.go#L82 [1] https://github.com/apache/yunikorn-k8shim/blob/58adfe941d2d8dae5544af8b49e435f304678807/pkg/webtest/web_server_test.go#L73 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2630) Release context lock in shim when processing config in the core
[ https://issues.apache.org/jira/browse/YUNIKORN-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2630. -- Fix Version/s: 1.6.0 1.5.2 Resolution: Fixed > Release context lock in shim when processing config in the core > --- > > Key: YUNIKORN-2630 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2630 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Wilfred Spiegelenburg >Assignee: Wilfred Spiegelenburg >Priority: Critical > Labels: pull-request-available > Fix For: 1.6.0, 1.5.2 > > > When an change comes in for a the configmaps we process the change under a > context lock as we need to merge the two configmaps. > We keep this lock even if all the work is done in the shim and processing has > been transferred to the core. This is unneeded as the core has its own > locking an serialisation of the changes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2640) Conside removing config from Clients
Chia-Ping Tsai created YUNIKORN-2640: Summary: Conside removing config from Clients Key: YUNIKORN-2640 URL: https://issues.apache.org/jira/browse/YUNIKORN-2640 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai The config (`conf.SchedulerConf`) [0] references to a global singleton object [1][2]. Also, in the code base `clients#GetConf()` is used 3 times [3] and `conf.GetSchedulerConf()` is used 61 times [4] It seems to me `clients#conf` should be removed to avoid confusion. [0] https://github.com/apache/yunikorn-k8shim/blob/master/pkg/client/clients.go#L42C8-L42C26 [1] https://github.com/apache/yunikorn-k8shim/blob/6f2800f689e9e341c736a6af8cbf178a711a9423/pkg/plugin/scheduler_plugin.go#L291 [2] https://github.com/apache/yunikorn-k8shim/blob/6f2800f689e9e341c736a6af8cbf178a711a9423/pkg/cmd/shim/main.go#L53 [3] https://github.com/search?q=repo%3Aapache%2Fyunikorn-k8shim+GetConf%28%29=code [4] https://github.com/search?q=repo%3Aapache%2Fyunikorn-k8shim+conf.GetSchedulerConf%28%29=code -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2616) Remove unused bool return from PreemptionPredicates()
[
https://issues.apache.org/jira/browse/YUNIKORN-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2616.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Remove unused bool return from PreemptionPredicates()
> -
>
> Key: YUNIKORN-2616
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2616
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: shim - kubernetes
>Reporter: Wilfred Spiegelenburg
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> The predicate manager method {{PreemptionPredicates()}} returns two values an
> int and boolean. The boolean is false if the integer is -1 and true for 0 or
> llarger. There is no need for the boolean as the -1 already indicates the same
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2602) Fix spelling/grammar in configvalidator
[ https://issues.apache.org/jira/browse/YUNIKORN-2602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2602. -- Fix Version/s: 1.6.0 Resolution: Fixed > Fix spelling/grammar in configvalidator > --- > > Key: YUNIKORN-2602 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2602 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - common >Reporter: Peter Bacsko >Assignee: Yun Sun >Priority: Trivial > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > Let's fix some minor grammar issues in configvalidator.go. > Eg.: "existed" -> "existing", but there could be other mistakes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-22) k8shim is hardcoded to the default partition
[ https://issues.apache.org/jira/browse/YUNIKORN-22?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-22. Resolution: Won't Fix see https://github.com/apache/yunikorn-k8shim/pull/293#issuecomment-2108922407 > k8shim is hardcoded to the default partition > > > Key: YUNIKORN-22 > URL: https://issues.apache.org/jira/browse/YUNIKORN-22 > Project: Apache YuniKorn > Issue Type: Bug > Components: shim - kubernetes >Reporter: Wilfred Spiegelenburg >Assignee: Rainie Li >Priority: Major > > In the application and node code the partition is hardcoded to use the > DefaultPartition constant when creating new objects: > * application.NewApplication > * schedulerNode.addExistingAllocation > This means that in the configuration for the core we must have that same > partition and that we currently would not be able to create a second shim for > the same core as they would interfere with each other. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-802) Supports to assign nodes to non-default partition
[ https://issues.apache.org/jira/browse/YUNIKORN-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-802. - Resolution: Won't Fix > Supports to assign nodes to non-default partition > - > > Key: YUNIKORN-802 > URL: https://issues.apache.org/jira/browse/YUNIKORN-802 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: shim - kubernetes >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Major > Labels: pull-request-available > > see comment > (https://issues.apache.org/jira/browse/YUNIKORN-22?focusedCommentId=17398860=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17398860) > Currently, all nodes are hardcode to be assigned to "default" partition. That > brings two disadvantages. > # we can't select specify nodes, which are used to execute spark job only, > from a cluster > # multi-partitions does not work since non-default partition can't get nodes > Future works: > # support to change partition assignment of existent node (in this PR, the > update request will be skipped) > # support to remove existent node which had been reassigned (in this PR, > removing such node cause error message "Failed to update non existing node > ...") -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2625) Refactor Clients to avoid hard-code checks
Chia-Ping Tsai created YUNIKORN-2625:
Summary: Refactor Clients to avoid hard-code checks
Key: YUNIKORN-2625
URL: https://issues.apache.org/jira/browse/YUNIKORN-2625
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
{code:java}
if c.NodeInformer.Informer().HasSynced() &&
c.PodInformer.Informer().HasSynced() &&
c.PVCInformer.Informer().HasSynced() &&
c.PVInformer.Informer().HasSynced() &&
c.StorageInformer.Informer().HasSynced() &&
c.CSINodeInformer.Informer().HasSynced() &&
c.ConfigMapInformer.Informer().HasSynced() &&
c.NamespaceInformer.Informer().HasSynced() &&
c.PriorityClassInformer.Informer().HasSynced() {
return
} {code}
{code:java}
go c.NodeInformer.Informer().Run(stopCh)
go c.PodInformer.Informer().Run(stopCh)
go c.PVInformer.Informer().Run(stopCh)
go c.PVCInformer.Informer().Run(stopCh)
go c.StorageInformer.Informer().Run(stopCh)
go c.CSINodeInformer.Informer().Run(stopCh)
go c.ConfigMapInformer.Informer().Run(stopCh)
go c.NamespaceInformer.Informer().Run(stopCh)
go c.PriorityClassInformer.Informer().Run(stopCh) {code}
Instead of listing all informer manually, we should add a method to `Clients`
to list all informers and then loop all of them. That can bring two benefits.
# more readable
# easy to write tests. We can override the `informers` method to return mock
informer
# avoid error-prone (we could miss one of informers in adding more helper
methods in the future)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2578) Refactor SchedulerCache.GetPod() remove bool return
[
https://issues.apache.org/jira/browse/YUNIKORN-2578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2578.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Refactor SchedulerCache.GetPod() remove bool return
> ---
>
> Key: YUNIKORN-2578
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2578
> Project: Apache YuniKorn
> Issue Type: Task
> Components: shim - kubernetes
>Reporter: Wilfred Spiegelenburg
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> SchedulerCache {{GetPod()}} and {{GetPodNoLock()}} retrun two values:
> # *v1.Pod
> # bool
> The boolean value is redundant as it is false if the pod is not found and a
> nil is returned for the pod. The boolean is true if the pod has a value.
> Testing for a nil pod has the same result.
> We do not cache a nil pod in the cache for a pod UID
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2617) Update kindest/node to v1.29.2 for Makefile
[ https://issues.apache.org/jira/browse/YUNIKORN-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2617. -- Fix Version/s: 1.6.0 Resolution: Fixed > Update kindest/node to v1.29.2 for Makefile > --- > > Key: YUNIKORN-2617 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2617 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Hsien-Cheng(Ryan) Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > see > https://github.com/apache/yunikorn-k8shim/blob/d884f194b2cf60e574717f60fe648305781b56ef/Makefile#L68 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2590) Handler tests should check for nil request on create
[
https://issues.apache.org/jira/browse/YUNIKORN-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2590.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Handler tests should check for nil request on create
>
>
> Key: YUNIKORN-2590
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2590
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: core - common, test - unit
>Reporter: Wilfred Spiegelenburg
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Major
> Labels: newbie, pull-request-available
> Fix For: 1.6.0
>
>
> In the handler_test.go file we have an anti pattern showing a large number
> (40+) warnings in an IDE:
> {quote}'req' might have 'nil' or other unexpected value as its corresponding
> error variable might be not 'nil'
> {quote}
> The warning are due to the fact that we have the following pattern:
> {code:java}
> req, err = http.NewRequest("GET", "path", strings.NewReader(""))
> req = req.WithContext(context.WithValue(req.Context(), httprouter.ParamsKey,
> httprouter.Params{})){code}
> There is no error assertion after the request creation. We should add a
> simple {{assert.NilError(t, err, "HTTP request create failed")}} inserted
> between creating and using the request.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2620) Remove redundant variable `errorExpected` from configvalidator_test.go
Chia-Ping Tsai created YUNIKORN-2620: Summary: Remove redundant variable `errorExpected` from configvalidator_test.go Key: YUNIKORN-2620 URL: https://issues.apache.org/jira/browse/YUNIKORN-2620 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Yun Sun This is similar to YUNIKORN-2598. We can check the existent of `validateFunc` instead of having a extra boolean flag. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2617) Update kindest/node to v1.29.2 for Makefile
Chia-Ping Tsai created YUNIKORN-2617: Summary: Update kindest/node to v1.29.2 for Makefile Key: YUNIKORN-2617 URL: https://issues.apache.org/jira/browse/YUNIKORN-2617 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Hsien-Cheng(Ryan) Huang see https://github.com/apache/yunikorn-k8shim/blob/d884f194b2cf60e574717f60fe648305781b56ef/Makefile#L68 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2577) Remove named returns from IsPodFitNodeViaPreemption
[
https://issues.apache.org/jira/browse/YUNIKORN-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2577.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Remove named returns from IsPodFitNodeViaPreemption
> ---
>
> Key: YUNIKORN-2577
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2577
> Project: Apache YuniKorn
> Issue Type: Improvement
> Components: shim - kubernetes
>Reporter: Wilfred Spiegelenburg
>Assignee: Hsien-Cheng(Ryan) Huang
>Priority: Minor
> Labels: newbie, pull-request-available
> Fix For: 1.6.0
>
>
> IsPodFitNodeViaPreemption has defined named returns but does not use them.
> They should be removed as the way they are used can cause issues that are
> hard to debug.
> As part of this change we need to further cleanup:
> * The variable {{ok}} also gets shadowed multiple times, not just from the
> named return declaration.
> * The if construct around {{GetPodNoLock()}} is not needed as it returns a
> nil for the pod if it returns false. Just adding the result for the pod
> always has the same effect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2599) Certain shim events are not handled by the state machine
[
https://issues.apache.org/jira/browse/YUNIKORN-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2599.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Certain shim events are not handled by the state machine
>
>
> Key: YUNIKORN-2599
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2599
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - yarn
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> After YUNIKORN-2597 got merged, it became clear that we keep sending an
> {{AppStateChange}} event which cannot be handled by the state machine. There
> isn't any state transition which are be triggered by it.
> {{AppTaskCompleted}} is very similar, it is only processed in {{Resuming}}
> state, but it's sent whenever a task is completed.
> If someone runs the test case TestApplicationScheduling, the following errors
> are displayed:
> {noformat}
> [...]
> 2024-05-02T18:08:14.856+0200 ERROR shim.contextcache/context.go:1316
> application event cannot be handled in the current state
> {"applicationID": "app0001", "event": "AppStateChange", "state": "Running"}
> github.com/apache/yunikorn-k8shim/pkg/shim.newShimSchedulerInternal.(*Context).ApplicationEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:1316
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.getEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:123
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.Start.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:225
> 2024-05-02T18:08:14.856+0200 INFOcore.scheduler.application
> [...]
> 2024-05-02T18:08:14.857+0200 INFOcore.scheduler.partition
> scheduler/partition.go:928 scheduler allocation processed {"appID":
> "app0001", "allocationKey": "task0002", "allocatedResource":
> "map[memory:1000 pods:1 vcore:1]", "placeholder": false, "targetNode":
> "test.host.02"}
> 2024-05-02T18:08:14.857+0200 ERROR shim.contextcache/context.go:1316
> application event cannot be handled in the current state
> {"applicationID": "app0001", "event": "AppStateChange", "state": "Running"}
> github.com/apache/yunikorn-k8shim/pkg/shim.newShimSchedulerInternal.(*Context).ApplicationEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:1316
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.getEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:123
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.Start.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:225
> [...]
> 2024-05-02T18:08:15.856+0200 INFOshim.fsmcache/task_state.go:380
> Task state transition {"app": "app0001", "task": "task0001", "taskAlias":
> "default/task0001", "source": "Bound", "destination": "Completed", "event":
> "CompleteTask"}
> 2024-05-02T18:08:15.856+0200 ERROR shim.contextcache/context.go:1316
> application event cannot be handled in the current state
> {"applicationID": "app0001", "event": "AppTaskCompleted", "state": "Running"}
> github.com/apache/yunikorn-k8shim/pkg/shim.newShimSchedulerInternal.(*Context).ApplicationEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:1316
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.getEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:123
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.Start.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:225
> [...]
> 2024-05-02T18:08:16.858+0200 INFOshim.fsmcache/task_state.go:380
> Task state transition {"app": "app0001", "task": "task0002", "taskAlias":
> "default/task0002", "source": "Bound", "destination": "Completed", "event":
> "CompleteTask"}
> 2024-05-02T18:08:16.858+0200 ERROR shim.contextcache/context.go:1316
> application event cannot be handled in the current state
> {"applicationID": "app0001", "event": "AppTaskCompleted", "state": "Running"}
> github.com/apache/yunikorn-k8shim/pkg/shim.newShimSchedulerInternal.(*Context).ApplicationEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:1316
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.getEventHandler.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:123
> github.com/apache/yunikorn-k8shim/pkg/dispatcher.Start.func1
> /home/bacskop/repos/yunikorn-k8shim/pkg/dispatcher/dispatcher.go:225
> [...]
> 2024-05-02T18:08:16.859+0200 ERROR
[jira] [Resolved] (YUNIKORN-2598) Update the unit test for checkQueues and checkNodeSortingPolicy
[ https://issues.apache.org/jira/browse/YUNIKORN-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2598. -- Fix Version/s: 1.6.0 Resolution: Fixed > Update the unit test for checkQueues and checkNodeSortingPolicy > --- > > Key: YUNIKORN-2598 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2598 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yun Sun >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > similar to YUNIKORN-2297 > they are -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2601) Update kindest/node: v1.29.1 to v1.29.2, v1.28.6 to v1.28.7, v1.27.10 to v1.27.11, v1.26.13 -> v1.26.14
Chia-Ping Tsai created YUNIKORN-2601: Summary: Update kindest/node: v1.29.1 to v1.29.2, v1.28.6 to v1.28.7, v1.27.10 to v1.27.11, v1.26.13 -> v1.26.14 Key: YUNIKORN-2601 URL: https://issues.apache.org/jira/browse/YUNIKORN-2601 Project: Apache YuniKorn Issue Type: Improvement Components: test - e2e Reporter: Chia-Ping Tsai as title -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2598) Update the unit test for checkQueues and checkNodeSortingPolicy
Chia-Ping Tsai created YUNIKORN-2598: Summary: Update the unit test for checkQueues and checkNodeSortingPolicy Key: YUNIKORN-2598 URL: https://issues.apache.org/jira/browse/YUNIKORN-2598 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Yun Sun similar to YUNIKORN-2297 they are -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2580) Remove executionTimeoutMilliSeconds or add facility for it
Chia-Ping Tsai created YUNIKORN-2580: Summary: Remove executionTimeoutMilliSeconds or add facility for it Key: YUNIKORN-2580 URL: https://issues.apache.org/jira/browse/YUNIKORN-2580 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai [https://github.com/apache/yunikorn-scheduler-interface/blob/b70081933c38018fd7f01c82635f5b186c4ef394/si.proto#L211] It is not used actually, and hence we should either remove it or add facility for it. Personally, I'd like to remove it to simplify the interface. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2563) [shim] Enable deadlock detection during unit tests
[ https://issues.apache.org/jira/browse/YUNIKORN-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2563. -- Fix Version/s: 1.6.0 Resolution: Fixed > [shim] Enable deadlock detection during unit tests > -- > > Key: YUNIKORN-2563 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2563 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: shim - kubernetes, test - unit >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2547) Queue: clean up logic when adding application
[
https://issues.apache.org/jira/browse/YUNIKORN-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2547.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Queue: clean up logic when adding application
> -
>
> Key: YUNIKORN-2547
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2547
> Project: Apache YuniKorn
> Issue Type: Sub-task
> Components: core - scheduler
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> We found two issues when adding an application to a queue:
> # Inside {{Queue.AddApplication()}}, we parse and process "quota" and
> "guaranteed" from the application tags, then we set them on the queue if they
> have a valid value. We shouldn't be doing this inside {{AddApplication()}},
> but rather when we're constructing the application object. That way, they're
> already available when the app is being added.
> # We an add application to the Queue, but this can be reverted immediately if
> the placeholder doesn't fit or the "sortType" is not FIFO.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2560) Install the ginkgo to tools folder
[ https://issues.apache.org/jira/browse/YUNIKORN-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2560. -- Fix Version/s: 1.6.0 Resolution: Fixed > Install the ginkgo to tools folder > -- > > Key: YUNIKORN-2560 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2560 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yun Sun >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > https://github.com/apache/yunikorn-k8shim/blob/master/scripts/run-e2e-tests.sh#L111 > `run-e2e-tests.sh` install the ginkgo in the `GOBIN` folder. However, our > `make e2e_test` assumes all tools are installed at yunikorn-k8shim/tools, and > hence we can see `Command 'ginkgo' not found` error if we don't export the > `GOBIN`. > It seems to me this jira should includes following changes. > 1. move the ginkgo installation from run-e2e-tests to makefile. Let `make > tools` install all required tools > 2. makefile should install ginkgo to tools folder. the tools folder is added > to PATH, so user can run e2e more easily. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2557) Shim cache update e2e-tests doc to install tools as pre-requisite
[ https://issues.apache.org/jira/browse/YUNIKORN-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2557. -- Resolution: Duplicate see YUNIKORN-2558 > Shim cache update e2e-tests doc to install tools as pre-requisite > - > > Key: YUNIKORN-2557 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2557 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Yun Sun >Assignee: Yun Sun >Priority: Major > Labels: pull-request-available > > When I want to run the e2e-tests for the first time, it is a time consuming > process to install the tools one by one, then learn from the failure to > figure out I need to link the path of each tool to the tools/ folder as > pre-requisite step. And there is an easy way to run the command `make tools` > to set up everything we need before we run the e2e-tests. > So it is great to update the doc for e2e-tests to set up the environment in > an easy way. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2558) Remove redundent conditional
[
https://issues.apache.org/jira/browse/YUNIKORN-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2558.
--
Fix Version/s: 1.6.0
(was: 1.5.0)
Resolution: Fixed
> Remove redundent conditional
>
>
> Key: YUNIKORN-2558
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2558
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
>Reporter: Hsien-Cheng(Ryan) Huang
>Assignee: Yun Sun
>Priority: Minor
> Labels: newbie, pull-request-available
> Fix For: 1.6.0
>
>
> The script skips the {{make tools}} if the tools folder is existent. That is
> a bit weird that we don't check all tools in the folder. The script should
> call {{make tools}} anyway, and let {{make tools}} do the check and install.
> remove the conditional(or the {{install_tools}} function, and use {{{}make
> tools{}}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2560) Install the ginkgo to tools folder
Chia-Ping Tsai created YUNIKORN-2560: Summary: Install the ginkgo to tools folder Key: YUNIKORN-2560 URL: https://issues.apache.org/jira/browse/YUNIKORN-2560 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai https://github.com/apache/yunikorn-k8shim/blob/master/scripts/run-e2e-tests.sh#L111 `run-e2e-tests.sh` install the ginkgo in the `GOBIN` folder. However, our `make e2e_test` assumes all tools are installed at yunikorn-k8shim/tools, and hence we can see `Command 'ginkgo' not found` error if we don't export the `GOBIN`. It seems to me this jira should includes following changes. 1. move the ginkgo installation from run-e2e-tests to makefile. Let `make tools` install all required tools 2. makefile should install ginkgo to tools folder. the tools folder is added to PATH, so user can run e2e more easily. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2538) Shim cache context pre-allocate slice
[ https://issues.apache.org/jira/browse/YUNIKORN-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2538. -- Fix Version/s: 1.6.0 Resolution: Fixed > Shim cache context pre-allocate slice > - > > Key: YUNIKORN-2538 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2538 > Project: Apache YuniKorn > Issue Type: Improvement > Components: shim - kubernetes >Reporter: Wilfred Spiegelenburg >Assignee: Yun Sun >Priority: Trivial > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > When building the reason string from all volume failure reasons we should > allocate a slice once based on the size of the reasons object we get returned. > See [review > comment|https://github.com/apache/yunikorn-k8shim/pull/810#discussion_r1550882867] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2554) Remove "rules" field from PartitionContext
[ https://issues.apache.org/jira/browse/YUNIKORN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2554. -- Fix Version/s: 1.6.0 Resolution: Fixed > Remove "rules" field from PartitionContext > -- > > Key: YUNIKORN-2554 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2554 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - scheduler >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Trivial > Labels: pull-request-available > Fix For: 1.6.0 > > > The "rules" field inside the PartitionContext is obsolete. > It is set but nothing reads it. It can also become out of sync with the > contents of the placement manager. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2551) Move `buildRules` out of `AppPlacementManager` to be a function
Chia-Ping Tsai created YUNIKORN-2551:
Summary: Move `buildRules` out of `AppPlacementManager` to be a
function
Key: YUNIKORN-2551
URL: https://issues.apache.org/jira/browse/YUNIKORN-2551
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai
https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/placement/placement.go#L70
{code:java}
func (m *AppPlacementManager) initialise(rules []configs.PlacementRule) error {
log.Log(log.Config).Info("Building new rule list for placement manager")
// build temp list from new config
// here!!!
tempRules, err := m.buildRules(rules)
if err != nil {
return err
}
m.Lock()
defer m.Unlock()
{code}
We call `m.buildRules` without holding AppPlacementManager lock since
`buildRules` does not touch any fields of `AppPlacementManager`. It seems to me
we should make `buildRules` be a function rather than method of
`AppPlacementManager`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2523) Bump go to 1.22
[ https://issues.apache.org/jira/browse/YUNIKORN-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2523. -- Fix Version/s: 1.6.0 Resolution: Fixed > Bump go to 1.22 > --- > > Key: YUNIKORN-2523 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2523 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Ryan Lo >Assignee: Ryan Lo >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > The latest go 1.22 released in this Feb. > https://go.dev/doc/go1.22 > We should change to use latest go version to build YK. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2524) add documentation for recovery queue (root.@recovery@)
Chia-Ping Tsai created YUNIKORN-2524: Summary: add documentation for recovery queue (root.@recovery@) Key: YUNIKORN-2524 URL: https://issues.apache.org/jira/browse/YUNIKORN-2524 Project: Apache YuniKorn Issue Type: Improvement Components: documentation Reporter: Chia-Ping Tsai the recovery queue is unqueryable directly but we can observe the recovery queue name via app Restful API (`ws/v1/partition/%s/application/%s`). Hence, we should write documents for recovery queue. Otherwise, it would be surprise to users when they see the incomprehensible queue and they get nothing from our docs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2192) fix the incorrect instruction docs of "creation/update the ConfigMap"
[ https://issues.apache.org/jira/browse/YUNIKORN-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2192. -- Fix Version/s: 1.6.0 Resolution: Fixed > fix the incorrect instruction docs of "creation/update the ConfigMap" > -- > > Key: YUNIKORN-2192 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2192 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: JiaChi Wang >Priority: Trivial > Labels: pull-request-available > Fix For: 1.6.0 > > > doc link: > https://yunikorn.apache.org/docs/next/developer_guide/deployment#createupdate-the-configmap > the example of Yunikorn-configs.yaml from > `https://raw.githubusercontent.com/apache/yunikorn-k8shim/master/deployments/scheduler/yunikorn-configs.yaml`should > be applied directly rather than creating the ConfigMap. > It seems to me the following changes are required. > 1) "modify the content of yunikorn-configs.yaml file as needed, and create > ConfigMap in kubernetes:" -> "modify the content of yunikorn-configs.yaml > file as needed, and apply yunikorn-configs.yaml file in kubernetes:" > 2) "kubectl create configmap yunikorn-configs > --from-file=yunikorn-configs.yaml" -> "kubectl apply -f yunikorn-configs.yaml" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2215) add examples of "--focus.file" and "--skip-file" for e2e docs
[ https://issues.apache.org/jira/browse/YUNIKORN-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2215. -- Target Version: (was: 1.6.0) Resolution: Duplicate close this as duplicate to https://issues.apache.org/jira/browse/YUNIKORN-2522 > add examples of "--focus.file" and "--skip-file" for e2e docs > - > > Key: YUNIKORN-2215 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2215 > Project: Apache YuniKorn > Issue Type: New Feature > Components: documentation, test - e2e >Reporter: Chia-Ping Tsai >Assignee: JiaChi Wang >Priority: Minor > Labels: pull-request-available > > [https://github.com/apache/yunikorn-k8shim/blob/master/test/e2e/README.md] > > Apart from "Launching specified test", testing specified test/folder is > useful instruction also. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2499) MockResponseWriter#write should merge new bytes with older one
[ https://issues.apache.org/jira/browse/YUNIKORN-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2499. -- Fix Version/s: 1.6.0 Resolution: Fixed > MockResponseWriter#write should merge new bytes with older one > -- > > Key: YUNIKORN-2499 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2499 > Project: Apache YuniKorn > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: JiaChi Wang >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > I notice this issue when reviewing > [https://github.com/apache/yunikorn-core/pull/757] > Normally, we call `write` only once to output the json data, and hence the > mock response work well. However, the gzip writer will call the writer > multi-times to complete the write with zip footer... and so the mock response > can't work with the new use case. > [https://github.com/apache/yunikorn-core/blob/master/pkg/webservice/handler_mock_test.go#L46] > I feel that can be one-line fix. > trw.outputBytes = append(trw.outputBytes, bytes...) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2455) Fix incorrect configs of historical event tracing in the documentation
[ https://issues.apache.org/jira/browse/YUNIKORN-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2455. -- Resolution: Duplicate it will be fixed by YUNIKORN-2444 > Fix incorrect configs of historical event tracing in the documentation > -- > > Key: YUNIKORN-2455 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2455 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Hsien-Cheng Huang >Priority: Minor > > The configs > (https://yunikorn.apache.org/docs/design/historical_usage_tracking/#configuration) > are very different from source code > (https://github.com/apache/yunikorn-core/blob/master/pkg/common/configs/configs.go#L36) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2511) Fix flaky TestApplicationHistoryTracking
[
https://issues.apache.org/jira/browse/YUNIKORN-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2511.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> Fix flaky TestApplicationHistoryTracking
>
>
> Key: YUNIKORN-2511
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2511
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: core - scheduler, test - unit
>Reporter: Chia-Ping Tsai
>Assignee: JiaChi Wang
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> {code}
> 2024-03-23T08:13:06.036Z INFOcore.events
> events/event_publisher.go:69Stopping shim event publisher
> --- FAIL: TestApplicationHistoryTracking (0.01s)
> application_tracking_test.go:62: assertion failed: 2 (int) != 0 (int):
> number of events generated
> {code}
> The test is not stable on my local.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2511) Fix flaky TestApplicationHistoryTracking
Chia-Ping Tsai created YUNIKORN-2511:
Summary: Fix flaky TestApplicationHistoryTracking
Key: YUNIKORN-2511
URL: https://issues.apache.org/jira/browse/YUNIKORN-2511
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler, test - unit
Reporter: Chia-Ping Tsai
{code}
2024-03-23T08:13:06.036ZINFOcore.events
events/event_publisher.go:69Stopping shim event publisher
--- FAIL: TestApplicationHistoryTracking (0.01s)
application_tracking_test.go:62: assertion failed: 2 (int) != 0 (int):
number of events generated
{code}
The test is not stable on my local.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2443) Update the features page with the event system
[ https://issues.apache.org/jira/browse/YUNIKORN-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2443. -- Fix Version/s: 1.6.0 Resolution: Fixed > Update the features page with the event system > -- > > Key: YUNIKORN-2443 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2443 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: documentation >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2499) MockResponseWriter#write should merge new bytes with older one
Chia-Ping Tsai created YUNIKORN-2499: Summary: MockResponseWriter#write should merge new bytes with older one Key: YUNIKORN-2499 URL: https://issues.apache.org/jira/browse/YUNIKORN-2499 Project: Apache YuniKorn Issue Type: Bug Reporter: Chia-Ping Tsai I notice this issue when reviewing [https://github.com/apache/yunikorn-core/pull/757] Normally, we call `write` only once to output the json data, and hence the mock response work well. However, the gzip writer will call the writer multi-times to complete the write with zip footer... and so the mock response can't work with the new use case. [https://github.com/apache/yunikorn-core/blob/master/pkg/webservice/handler_mock_test.go#L46] I feel that can be one-line fix. trw.outputBytes = append(trw.outputBytes, bytes...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2448) Expose 3rd party licenses in the web UI
[ https://issues.apache.org/jira/browse/YUNIKORN-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2448. -- Fix Version/s: 1.6.0 Resolution: Fixed > Expose 3rd party licenses in the web UI > --- > > Key: YUNIKORN-2448 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2448 > Project: Apache YuniKorn > Issue Type: Improvement > Components: webapp >Reporter: Wilfred Spiegelenburg >Assignee: Denis Coric >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 1.6.0 > > > We have a 3rd party license file that gets generated and included in the > deployment for the web UI. This 3rd party license file is accessible if you > know what its name is etc. > We should expose this detail to comply with the some requirements on > attribution etc as part of the web UI. Similar to how Jira exposes it as part > of its About Jira pop up. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2473) Out-dated/dead link @docs/developer_guide/env_setup.md
[ https://issues.apache.org/jira/browse/YUNIKORN-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2473. -- Fix Version/s: 1.6.0 Resolution: Fixed > Out-dated/dead link @docs/developer_guide/env_setup.md > -- > > Key: YUNIKORN-2473 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2473 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Hsien-Cheng Huang >Assignee: Hsien-Cheng Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > In > ([https://github.com/apache/yunikorn-site/blob/master/docs/developer_guide/env_setup.md#installation)|https://github.com/apache/yunikorn-site/blob/master/docs/developer_guide/env_setup.md#installation),] > "Follow the instructions > [here]([https://docs.docker.com/docker-for-mac/#kubernetes)] > to get Kubernetes up and running within Docker Desktop.", the link is broken > for the new Docker documentation which will lead to incorrect > page([https://docs.docker.com/desktop/get-started/#kubernetes).] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2470) remove the argument from task#DeleteTaskPod
[ https://issues.apache.org/jira/browse/YUNIKORN-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2470. -- Fix Version/s: 1.6.0 Resolution: Fixed > remove the argument from task#DeleteTaskPod > --- > > Key: YUNIKORN-2470 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2470 > Project: Apache YuniKorn > Issue Type: Bug >Reporter: Chia-Ping Tsai >Assignee: Nick Chao >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > [https://github.com/apache/yunikorn-k8shim/blob/master/pkg/cache/task.go#L179] > that is error-prone, so it would be better to fix it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2449) add PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 to github actions for web CI
[
https://issues.apache.org/jira/browse/YUNIKORN-2449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2449.
--
Fix Version/s: 1.6.0
Resolution: Fixed
> add PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 to github actions for web CI
> ---
>
> Key: YUNIKORN-2449
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2449
> Project: Apache YuniKorn
> Issue Type: Improvement
>Reporter: Chia-Ping Tsai
>Assignee: Ryan Lo
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.0
>
>
> from [~wilfreds] comment (YUNIKORN-2477)
> {quote}
> in the Makefile we do have an extra argument
> {{PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1}} which we do not have in the github
> action calls. There we just run the {{yarn}} commands directly. We might need
> to add the same in the github actions or call the make targets to get the
> equivalence.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2470) task#DeleteTaskPod does not honor the passed pod
Chia-Ping Tsai created YUNIKORN-2470: Summary: task#DeleteTaskPod does not honor the passed pod Key: YUNIKORN-2470 URL: https://issues.apache.org/jira/browse/YUNIKORN-2470 Project: Apache YuniKorn Issue Type: Bug Reporter: Chia-Ping Tsai Assignee: Nick Chao [https://github.com/apache/yunikorn-k8shim/blob/master/pkg/cache/task.go#L179] that is error-prone, so it would be better to fix it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2466) Eliminate all git error from Makefile
Chia-Ping Tsai created YUNIKORN-2466: Summary: Eliminate all git error from Makefile Key: YUNIKORN-2466 URL: https://issues.apache.org/jira/browse/YUNIKORN-2466 Project: Apache YuniKorn Issue Type: Bug Reporter: Chia-Ping Tsai I encounter the git error when testing 1.5.0 RC1 code. The root cause is that there is no git folder in the release code. It seems to me running the tests (and other make action) for release source code is necessary and the git error should disappear. We can use if-else (e.x `if [ -d "$(BASE_DIR)/.git" ]`) to eliminate the git error. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2464) Missing markdown list indentation @docs/design/historical_usage_tracking.md
[ https://issues.apache.org/jira/browse/YUNIKORN-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2464. -- Fix Version/s: 1.6.0 Resolution: Fixed thanks for this contribution! > Missing markdown list indentation @docs/design/historical_usage_tracking.md > --- > > Key: YUNIKORN-2464 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2464 > Project: Apache YuniKorn > Issue Type: Bug > Components: documentation >Reporter: Hsien-Cheng Huang >Assignee: Hsien-Cheng Huang >Priority: Minor > Labels: pull-request-available > Fix For: 1.6.0 > > > We should add indentation for the 2-level list (on Goals [h2] > [https://github.com/apache/yunikorn-site/blob/master/docs/design/historical_usage_tracking.md#goals]) > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2454) Event streaming: send instanceUUID before the events
[ https://issues.apache.org/jira/browse/YUNIKORN-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2454. -- Fix Version/s: 1.6.0 Resolution: Fixed > Event streaming: send instanceUUID before the events > > > Key: YUNIKORN-2454 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2454 > Project: Apache YuniKorn > Issue Type: Improvement > Components: core - scheduler >Reporter: Peter Bacsko >Assignee: Peter Bacsko >Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > It's necessary to send the InstanceUUID first before any events. This way, > clients can identify easily if Yunikorn has restarted or not. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2315) Update REST API docs with changes from YUNIKORN-2235
[ https://issues.apache.org/jira/browse/YUNIKORN-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2315. -- Fix Version/s: 1.5.0 Resolution: Fixed > Update REST API docs with changes from YUNIKORN-2235 > > > Key: YUNIKORN-2315 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2315 > Project: Apache YuniKorn > Issue Type: Improvement > Components: documentation >Reporter: Xie YiFan >Assignee: Xie YiFan >Priority: Minor > Labels: pull-request-available > Fix For: 1.5.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2461) Consider supporting /ws/v1/partition/{partitionName}/queue/{queueName}/applications/:state
Chia-Ping Tsai created YUNIKORN-2461:
Summary: Consider supporting
/ws/v1/partition/{partitionName}/queue/{queueName}/applications/:state
Key: YUNIKORN-2461
URL: https://issues.apache.org/jira/browse/YUNIKORN-2461
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
from https://github.com/apache/yunikorn-site/pull/390#discussion_r1468916528
We have both /ws/v1/partition/{partitionName}/application/{appId} and
/ws/v1/partition/{partitionName}/queue/{queueName}/application/{appId}, hence
having both /ws/v1/partition/{partitionName}/applications/:state and
/ws/v1/partition/{partitionName}/queue/{queueName}/applications/:state makes
our APIs more consistent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2445) Add comments around locking setup in tracker code
[
https://issues.apache.org/jira/browse/YUNIKORN-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved YUNIKORN-2445.
--
Fix Version/s: 1.5.0
Resolution: Fixed
> Add comments around locking setup in tracker code
> -
>
> Key: YUNIKORN-2445
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2445
> Project: Apache YuniKorn
> Issue Type: Task
> Components: core - scheduler
>Reporter: Wilfred Spiegelenburg
>Assignee: Yu-Lin Chen
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.5.0
>
>
> The QueueTracker code is lock free and should stay lock free. Each queue
> tracker object is always only linked to one UserTracker or GroupTracker.
> Locking is thus handled from those objects.
> This does mean that calls to the user or group trackers that can modify the
> underlying queue tracker structure must take a write lock.
> This specifically impacts the {{canRunApp()}} and {{headroom()}} calls as
> they add new entries in the queue hierarchy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2455) Fix incorrect configs of historical event tracing in the documentation
Chia-Ping Tsai created YUNIKORN-2455: Summary: Fix incorrect configs of historical event tracing in the documentation Key: YUNIKORN-2455 URL: https://issues.apache.org/jira/browse/YUNIKORN-2455 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai The configs (https://yunikorn.apache.org/docs/design/historical_usage_tracking/#configuration) are very different from source code (https://github.com/apache/yunikorn-core/blob/master/pkg/common/configs/configs.go#L36) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2453) Add `EventRecord_APP_NEW` to event of created/submitted application
Chia-Ping Tsai created YUNIKORN-2453: Summary: Add `EventRecord_APP_NEW` to event of created/submitted application Key: YUNIKORN-2453 URL: https://issues.apache.org/jira/browse/YUNIKORN-2453 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai the event detail of new application is "EventRecord_DETAILS_NONE" (see [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/objects/application_events.go#L94]), and it seems the "EventRecord_APP_NEW" is more acceptable (see [https://github.com/apache/yunikorn-scheduler-interface/blob/master/lib/go/si/si.pb.go#L350]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2452) node.go#GetAvailableResource ought to use read-lock instead of readwrite-lock
Chia-Ping Tsai created YUNIKORN-2452: Summary: node.go#GetAvailableResource ought to use read-lock instead of readwrite-lock Key: YUNIKORN-2452 URL: https://issues.apache.org/jira/browse/YUNIKORN-2452 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai see [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/objects/node.go#L264] The other similar methods (`GetAllocatedResource`, `GetOccupiedResource`, and `GetCapacity`) are using read-lock, so `GetAvailableResource` ought to apply read-lock too. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2451) add trackingType#String to decorate the logging output
Chia-Ping Tsai created YUNIKORN-2451: Summary: add trackingType#String to decorate the logging output Key: YUNIKORN-2451 URL: https://issues.apache.org/jira/browse/YUNIKORN-2451 Project: Apache YuniKorn Issue Type: Sub-task Reporter: Chia-Ping Tsai The tracking type in the log is an integer number. We can add String method for trackingType to offer more readable output. [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/ugm/queue_tracker.go#L90] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2450) Rename `updateLowestId` to `updateLowestID`, `TestLoggerIds` to `TestLoggerIDs`
Chia-Ping Tsai created YUNIKORN-2450: Summary: Rename `updateLowestId` to `updateLowestID`, `TestLoggerIds` to `TestLoggerIDs` Key: YUNIKORN-2450 URL: https://issues.apache.org/jira/browse/YUNIKORN-2450 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai This is follow-up of https://issues.apache.org/jira/browse/YUNIKORN-2413 The methods are shown below. [https://github.com/apache/yunikorn-core/blob/master/pkg/events/event_ringbuffer.go#L206] [https://github.com/apache/yunikorn-core/blob/master/pkg/log/logger_test.go#L38] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2449) add PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 to github actions for web CI
Chia-Ping Tsai created YUNIKORN-2449:
Summary: add PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 to github actions
for web CI
Key: YUNIKORN-2449
URL: https://issues.apache.org/jira/browse/YUNIKORN-2449
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
from [~wilfreds] comment (YUNIKORN-2477)
{quote}
in the Makefile we do have an extra argument
{{PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1}} which we do not have in the github
action calls. There we just run the {{yarn}} commands directly. We might need
to add the same in the github actions or call the make targets to get the
equivalence.
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Reopened] (YUNIKORN-2416) Cleanup replace directives
[ https://issues.apache.org/jira/browse/YUNIKORN-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reopened YUNIKORN-2416: -- reopen since k8shim is not updated > Cleanup replace directives > -- > > Key: YUNIKORN-2416 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2416 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yu-Lin Chen >Priority: Minor > Labels: pull-request-available > Fix For: 1.5.0 > > > The replace directives should be used only if > 1. the dependency is indirect, AND > 2. the indirect version is too old or has CVEs > For example: core repo has following deps in the replace > golang.org/x/crypto => golang.org/x/crypto v0.18.0 > this should be removed since the indirect version is v0.19.0 > golang.org/x/lint => golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 > this should be removed since we don't use it actually, and golangci-lint is > replacement in our CI. > golang.org/x/net => golang.org/x/net v0.20.0 > this should be removed since this dep is used directly > golang.org/x/sys => golang.org/x/sys v0.16.0 > this should be removed since the indirect version is v0.17.0 > golang.org/x/text => golang.org/x/text v0.14.0 > this should be removed since the indirect version is v0.14.0 > golang.org/x/tools => golang.org/x/tools v0.17.0 > this is the only one we should keep in the replace since the resolved version > is v0.6.0 and it is too stale (released on Feb 8, 2023) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2416) Cleanup replace directives
[ https://issues.apache.org/jira/browse/YUNIKORN-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2416. -- Fix Version/s: 1.5.0 Resolution: Fixed all reverts are completed > Cleanup replace directives > -- > > Key: YUNIKORN-2416 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2416 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yu-Lin Chen >Priority: Minor > Labels: pull-request-available > Fix For: 1.5.0 > > > The replace directives should be used only if > 1. the dependency is indirect, AND > 2. the indirect version is too old or has CVEs > For example: core repo has following deps in the replace > golang.org/x/crypto => golang.org/x/crypto v0.18.0 > this should be removed since the indirect version is v0.19.0 > golang.org/x/lint => golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 > this should be removed since we don't use it actually, and golangci-lint is > replacement in our CI. > golang.org/x/net => golang.org/x/net v0.20.0 > this should be removed since this dep is used directly > golang.org/x/sys => golang.org/x/sys v0.16.0 > this should be removed since the indirect version is v0.17.0 > golang.org/x/text => golang.org/x/text v0.14.0 > this should be removed since the indirect version is v0.14.0 > golang.org/x/tools => golang.org/x/tools v0.17.0 > this is the only one we should keep in the replace since the resolved version > is v0.6.0 and it is too stale (released on Feb 8, 2023) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2447) Failed to build web docker image
Chia-Ping Tsai created YUNIKORN-2447:
Summary: Failed to build web docker image
Key: YUNIKORN-2447
URL: https://issues.apache.org/jira/browse/YUNIKORN-2447
Project: Apache YuniKorn
Issue Type: Bug
Reporter: Chia-Ping Tsai
{code:java}
36.76 [cause]: Error: Download failed: server returned code 500. URL:
https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6167.85/linux64/chrome-linux64.zip
{code}
It seems the url is not available anymore (see
[https://github.com/puppeteer/puppeteer/issues/11967]), and there are two
solutions.
# update puppeteer to use new url
# add env
PUPPETEER_DOWNLOAD_BASE_URL=[https://storage.googleapis.com/chrome-for-testing-public]
to use new url to download chrome
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2446) Add OCI annotations to public docker images
Chia-Ping Tsai created YUNIKORN-2446:
Summary: Add OCI annotations to public docker images
Key: YUNIKORN-2446
URL: https://issues.apache.org/jira/browse/YUNIKORN-2446
Project: Apache YuniKorn
Issue Type: Improvement
Reporter: Chia-Ping Tsai
source: https://github.com/apache/yunikorn-web/pull/174#issuecomment-1959233281
OCI annotations:
https://github.com/opencontainers/image-spec/blob/main/annotations.md
I believe most Yunikorn users are using the docker images into which we push,
and so we should consider following a public protocol to set attributions for
our public images.
{code}
org.opencontainers.image.title="yunikorn-scheduler-k8s"
org.opencontainers.image.description="Apache Yunikorn"
org.opencontainers.image.created="${build_date}"
org.opencontainers.image.source="https://github.com/apache/yunikorn-k8shim;
org.opencontainers.image.url="https://hub.docker.com/r/apache/yunikorn;
org.opencontainers.image.revision="${build_revision}"
org.opencontainers.image.license="Apache-2.0"
org.opencontainers.image.documentation="https://yunikorn.apache.org;
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org
For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2403) Update golang.org/x/net and golang.org/x/time
[ https://issues.apache.org/jira/browse/YUNIKORN-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2403. -- Resolution: Fixed resolve it again since https://issues.apache.org/jira/browse/YUNIKORN-2416 will be the following-up :) > Update golang.org/x/net and golang.org/x/time > - > > Key: YUNIKORN-2403 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2403 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Kuan-Po Tseng >Priority: Major > Labels: core, pull-request-available > Fix For: 1.5.0 > > > golang.org/x/net v0.20.0 -> v0.21.0 > golang.org/x/time v0.3.0 -> v0.5.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2412) Avoid returning potential nil in handlers.go
[ https://issues.apache.org/jira/browse/YUNIKORN-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2412. -- Fix Version/s: 1.5.0 Resolution: Fixed > Avoid returning potential nil in handlers.go > > > Key: YUNIKORN-2412 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2412 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Ryan Lo >Assignee: Ryan Lo >Priority: Major > Labels: pull-request-available > Fix For: 1.5.0 > > > Issue split from YUNIKORN-2404 > Check if there are still API endpoints return null, and replace it with an > empty collection. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2387) Upgrade ginkgo
[ https://issues.apache.org/jira/browse/YUNIKORN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2387. -- Resolution: Fixed push addendum to master! [~lowc1012] thanks!!! > Upgrade ginkgo > -- > > Key: YUNIKORN-2387 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2387 > Project: Apache YuniKorn > Issue Type: Improvement > Components: test - e2e >Reporter: Wilfred Spiegelenburg >Assignee: Ryan Lo >Priority: Major > Labels: newbie, pull-request-available > Fix For: 1.5.0 > > > Upgrade ginkgo to 2.13.2 or 2.15. > * 2.13.2 upgrade fixes a FD leak and avoids allocations. > * 2.15 changes to go 1.20 and adds some new features. > Either would work, as long as all e2e tests compile pass with the version. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2426) Remove /ws/v1/periodicstatedump
[ https://issues.apache.org/jira/browse/YUNIKORN-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2426. -- Fix Version/s: 1.5.0 Resolution: Fixed [~douenergy] thanks for this contribution. BTW, please add release entry. > Remove /ws/v1/periodicstatedump > --- > > Key: YUNIKORN-2426 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2426 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Dong-Lin Hsieh >Priority: Major > Labels: pull-request-available > Fix For: 1.5.0 > > > `/ws/v1/periodicstatedump` was changed to be a no-op API by YUNIKORN-1500. It > seems the breaking changes are acceptable in minor release (see > YUNIKORN-1980). Also we gave a minor cycle (1.4.0) to remind users about the > deprecated `/ws/v1/periodicstatedump` . Hence, I think it should be safe to > remove those deprecated APIs from 1.5.0 release. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2427) Use r-lock instead of rw-lock in user_tracker.go#getGroupForApp
Chia-Ping Tsai created YUNIKORN-2427: Summary: Use r-lock instead of rw-lock in user_tracker.go#getGroupForApp Key: YUNIKORN-2427 URL: https://issues.apache.org/jira/browse/YUNIKORN-2427 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai see https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/ugm/user_tracker.go#L103 The function mutate nothing, so it is safe to use r-lock. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2401) Update github.com/uber/jaeger-client-go and github.com/uber/jaeger-lib
[ https://issues.apache.org/jira/browse/YUNIKORN-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2401. -- Resolution: Won't Fix YUNIKORN-2395 has removed both dependencies, so I'm going to close it as "Won't Fix" > Update github.com/uber/jaeger-client-go and github.com/uber/jaeger-lib > -- > > Key: YUNIKORN-2401 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2401 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yu-Lin Chen >Priority: Major > Labels: core > > as title, the latest version of jaeger-client-go is 2.30.0 > (https://github.com/jaegertracing/jaeger-client-go/releases) > the latest version of jaeger-lib is v2.4.1 > (https://github.com/jaegertracing/jaeger-lib/releases) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2395) Remove Jaeger dependencies and cleanup outdated traces implementation
[ https://issues.apache.org/jira/browse/YUNIKORN-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2395. -- Fix Version/s: 1.5.0 Resolution: Fixed > Remove Jaeger dependencies and cleanup outdated traces implementation > - > > Key: YUNIKORN-2395 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2395 > Project: Apache YuniKorn > Issue Type: Sub-task > Components: core - common >Reporter: Wilfred Spiegelenburg >Assignee: Yu-Lin Chen >Priority: Minor > Labels: pull-request-available > Fix For: 1.5.0 > > > The Jaeger client library has been deprecated for almost 2 years now and we > should no longer use this in our code. > We are also not using the latest version and should directly migrate to > OpenTelemetry as per the [notice in the > documentation.|https://www.jaegertracing.io/docs/latest/client-libraries/#deprecating-jaeger-clients] > Migrate to OpenTelemetry would require us to reimplement most of the code in > /pkg/traces. We could remove the outdated traces implementation first. Add > the OpenTelemetry version back if we decide to procceed with YUNIKORN-387. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2326) refactor `getAppsByState`, `getRejectedAppsByState`, and `getCompletedAppsByState`
[ https://issues.apache.org/jira/browse/YUNIKORN-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2326. -- Fix Version/s: 1.5.0 Resolution: Fixed [~priyansh] thanks for nice patch! > refactor `getAppsByState`, `getRejectedAppsByState`, and > `getCompletedAppsByState` > -- > > Key: YUNIKORN-2326 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2326 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Priyansh Choudhary >Priority: Minor > Labels: pull-request-available > Fix For: 1.5.0 > > > `getAppsByState`, `getRejectedAppsByState`, and `getCompletedAppsByState` do > the similar job that get apps according to specific state. Hence, it would be > better to have a helper method and then these three methods can reuse the > code. > > [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/partition.go#L1075] > [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/partition.go#L1088] > [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/partition.go#L1101] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Resolved] (YUNIKORN-2415) Remove Chinese meeting
[ https://issues.apache.org/jira/browse/YUNIKORN-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved YUNIKORN-2415. -- Fix Version/s: 1.5.0 Resolution: Fixed > Remove Chinese meeting > -- > > Key: YUNIKORN-2415 > URL: https://issues.apache.org/jira/browse/YUNIKORN-2415 > Project: Apache YuniKorn > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Yu-Lin Chen >Priority: Minor > Labels: newbie, pull-request-available > Fix For: 1.5.0 > > > The Chinese meeting is pending, and we should remove it in order to misdirect > YK users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
[jira] [Created] (YUNIKORN-2426) Remove /ws/v1/periodicstatedump
Chia-Ping Tsai created YUNIKORN-2426: Summary: Remove /ws/v1/periodicstatedump Key: YUNIKORN-2426 URL: https://issues.apache.org/jira/browse/YUNIKORN-2426 Project: Apache YuniKorn Issue Type: Improvement Reporter: Chia-Ping Tsai `/ws/v1/periodicstatedump` was changed to be a no-op API by YUNIKORN-1500. It seems the breaking changes are acceptable in minor release (see YUNIKORN-1980). Also we gave a minor cycle (1.4.0) to reminder the "deprecation" for users. Hence, I think it should be safe to remove those deprecated APIs from 1.5.0 release. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@yunikorn.apache.org For additional commands, e-mail: dev-h...@yunikorn.apache.org
