[
https://issues.apache.org/jira/browse/YUNIKORN-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17803186#comment-17803186
]
Chia-Ping Tsai commented on YUNIKORN-2287:
------------------------------------------
{quote}
The new feature was not documented or announced. This was discovered as part of
the documentation steps. We can use option 2 without an issue
{quote}
{quote}
I would also highly doubt that this is used by anyone yet. It was added for
extremely large setups 1000's of queues or a setup with a large number of user
limits objects. We have had nobody run into that yet outside of our, cloudera,
internal testing setups.
{quote}
I'd like to wait [~sidkrishna] to share the use cases. If there is no users, my
favor solution was to get rid of such feature directly.
{quote}
option 3 is always safe. For option 3 we could turn it around: if the base64
decode does not fail log a message and use the decoded data. If it fails ignore
it. Means we do not have to decompress twice, reusing the decompressor might be
a bit risky when it failed the first time. We can easily remove that step in
the next release.
{quote}
We all love trade-off story :)
Personally, we should pick up either compatibility or simplification. Means
that we should support both base64-encoding and double base64-encoding. Or we
remove those supports as the feature may be used by nobody currently.
> Decompress function doesn't need to decode base64
> -------------------------------------------------
>
> Key: YUNIKORN-2287
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2287
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Reporter: PoAn Yang
> Assignee: PoAn Yang
> Priority: Major
> Labels: e2e, pull-request-available
>
> I'm working on https://issues.apache.org/jira/browse/YUNIKORN-2267. I added
> an example as following, but compression configmap doesn't work.
>
> 1. Use gzip to compress and use base64 to encode the config
> ```
> echo "
> partitions:
> - name: default
> placementrules:
> - name: tag
> value: namespace
> create: true
> queues:
> - name: root
> submitacl: '*'
> queues:
> - name: parent
> submitacl: '*'" | gzip | base64
> ```
> 2. Set the result to `queues.yaml.gz` in binaryData field.
>
> Finally, we can see an error log in scheduler:
> ```
> 2023-12-22T15:32:15.913Z ERROR shim.config conf/schedulerconf.go:458
> failed to decode schedulerConfig entry \{"error": "illegal base64 data
> at input byte 0"}
> github.com/apache/yunikorn-k8shim/pkg/conf.Decompress
> github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:458
> github.com/apache/yunikorn-k8shim/pkg/conf.FlattenConfigMaps
> github.com/apache/yunikorn-k8shim/pkg/conf/schedulerconf.go:495
> github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).triggerReloadConfig
> github.com/apache/yunikorn-k8shim/pkg/cache/context.go:496
> github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateConfigMaps
> github.com/apache/yunikorn-k8shim/pkg/cache/context.go:394
> k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate
> k8s.io/[email protected]/tools/cache/controller.go:250
> k8s.io/client-go/tools/cache.FilteringResourceEventHandler.OnUpdate
> k8s.io/[email protected]/tools/cache/controller.go:315
> k8s.io/client-go/tools/cache.(*processorListener).run.func1
> k8s.io/[email protected]/tools/cache/shared_informer.go:971
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
> k8s.io/[email protected]/pkg/util/wait/backoff.go:226
> k8s.io/apimachinery/pkg/util/wait.BackoffUntil
> k8s.io/[email protected]/pkg/util/wait/backoff.go:227
> k8s.io/apimachinery/pkg/util/wait.JitterUntil
> k8s.io/[email protected]/pkg/util/wait/backoff.go:204
> k8s.io/apimachinery/pkg/util/wait.Until
> k8s.io/[email protected]/pkg/util/wait/backoff.go:161
> k8s.io/client-go/tools/cache.(*processorListener).run
> k8s.io/[email protected]/tools/cache/shared_informer.go:967
> k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1
> k8s.io/[email protected]/pkg/util/wait/wait.go:72
> ```
>
> I think the root cause is that the golang object is already decoded, so we
> don't need to decode it again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
