Nicholas Barton created SPARK-58565:
---------------------------------------
Summary: Some operator Helm chart RBAC tests don't run due to
misconfiguration
Key: SPARK-58565
URL: https://issues.apache.org/jira/browse/SPARK-58565
Project: Spark
Issue Type: Test
Components: Kubernetes
Affects Versions: kubernetes-operator-1.0.0, kubernetes-operator-1.1.0
Reporter: Nicholas Barton
h2. Summary:
Only the first assertion runs in the Helm RBAC test hooks. RBAC test hooks
don't provide full coverage.
h2. Description:
The `spark-kubernetes-operator-test-operator-rbac` and
`spark-kubernetes-operator-test-workload-rbac` hook pods declare their `kubectl
auth can-i` assertions as a multi-element `args` array under `command: ['bash',
'-c']`. Only the first element gets executed by bash; the remaining elements
become `$0`, `$1`, etc. and are never executed.
The upshot of this is that only one of the seven operator assertions runs, and
only one of the four workload assertions runs. The rest do not.
Concretely, you can understand the concept directly with:
{code:java}
$ docker run --rm --entrypoint bash bitnamisecure/kubectl:latest -c 'echo one;
echo two' 'echo three'
one
two
{code}
`helm test spark` passes either way, so we don't actually get any warning that
the rest of the test assertions are not being run. This also means that the
tests would succeed even if the operator was missing the permissions that the
dropped assertions were written to cover.
I think we can also expand the test coverage quite broadly for minimal extra
effort.
h2. Reproduction
We can observe that some tests are not running by revoking a permission that is
only tested by one of the assertions that gets silently dropped; the old test
suite passes anyway. For example, rules[0] of the operator ClusterRole is the
core grant — pods, services, configmaps, persistentvolumeclaims, events. As
written, the one assertion that does run checks sparkapplications, which lives
in rules[3]. We can revoke rules[0] and run the tests and see that things pass.
I used the following procedure to reproduce:
{code:java}
minikube start
helm install spark --create-namespace -f
build-tools/helm/spark-kubernetes-operator/values.yaml
build-tools/helm/spark-kubernetes-operator/
// check how many rules, view roles
kubectl describe clusterrole spark-operator-clusterrole // has verbs for
pods
kubectl get clusterrole spark-operator-clusterrole -o json | jq
'.rules|length'. // 7 rules
kubectl auth can-i create pods --all-namespaces
--as=system:serviceaccount:default:spark-operator. // yes
// now patch the clusterrole to remove rules[0]
kubectl patch clusterrole spark-operator-clusterrole --type=json
-p='[{"op":"remove","path":"/rules/0"}]'
// validate that the patch succeeded
kubectl describe clusterrole spark-operator-clusterrole // does NOT have
verbs for pods
kubectl get clusterrole spark-operator-clusterrole -o json | jq
'.rules|length'. // 6 rules
kubectl auth can-i create pods --all-namespaces
--as=system:serviceaccount:default:spark-operator. // no
// run the helm rbac tests
helm test spark --logs --filter
name=spark-kubernetes-operator-test-operator-rbac // they succeed, when they
should fail
{code}
h2. Proposed Fix
Make the test assertions one args element instead of an array.
h2. Disclosure
I used Claude Code (Opus 5) to investigate this issue and help put together the
report and repro.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
