dzueck opened a new pull request, #8971:
URL: https://github.com/apache/texera/pull/8971
<!--
Thanks for sending a pull request (PR)! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
[Contributing to
Texera](https://github.com/apache/texera/blob/main/CONTRIBUTING.md)
2. Ensure you have added or run the appropriate tests for your PR
3. If the PR is work in progress, mark it a draft on GitHub.
4. Please write your PR title to summarize what this PR proposes, we
are following Conventional Commits style for PR titles as well:
- `fix` is for behavior that worked before and no longer does; adding
or
removing a functionality, or reworking one so that user-facing
behavior
intentionally changes, is a `feat`; a change that leaves the
user-facing
behavior unchanged is a `refactor`.
- A test-only PR is `test(<module>): ...`; repairing a broken test is
`fix(test, <module>): ...`.
- A dependency bump is `fix(deps, <module>): ...` when it patches a CVE
and `chore(deps, <module>): ...` otherwise; GitHub Actions bumps take
`ci` as their module, e.g. `chore(deps, ci): ...`.
- A PR targeting a release branch appends the version as the last scope
component, e.g. `fix(deps, frontend, v1.2): ...`.
See CONTRIBUTING.md for the full convention.
5. Be sure to keep the PR description updated to reflect all changes.
-->
### What changes were proposed in this PR?
<!--
Please clarify what changes you are proposing. The purpose of this section
is to outline the changes. Here are some tips for you:
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
3. If it is a refactoring, clarify what has been changed.
3. It would be helpful to include a before-and-after comparison using
screenshots or GIFs.
4. Please consider writing useful notes for better and faster reviews.
-->
Adds support for running the k8s deployment under the
[Linkerd](https://linkerd.io/) service mesh, so that traffic between Texera
pods is encrypted with mTLS. This is important for security because traffic
between services (including database credentials and user data) is currently
plain text, and compute units can run arbitrary code submitted by users. This
PR adds the option to use Linkerd, but does not require it: the changes are
only annotations, and if Linkerd is not installed they have no effect. It is
only targeted at k8s deployments and was tested using RKE2.
To enable it, install Linkerd in the cluster following [these
instructions](https://linkerd.io/2/getting-started/), then install Texera as
usual. No changes to `values.yaml` or the install steps are needed.
To turn the mesh off for a deployment on a cluster that has Linkerd
installed, remove the Linkerd annotations from `values.yaml` or set
`linkerd.io/inject` to `"disabled"`.
Changes:
- **Texera services:** added a `podAnnotations` option to every Texera
template (services, jobs and DaemonSets) and set the Linkerd annotations for
each of them in `values.yaml`.
- **Third party charts:** set the Linkerd annotations through each chart's
own options: Postgres, RustFS, LakeFS (also covers the lakefs-setup job),
Lakekeeper (including its DB migration job) and Envoy Gateway (controller and
certgen job).
- **Gateway:** added an `EnvoyProxy` resource and pointed the Gateway at it
through `infrastructure.parametersRef`, since the Envoy proxy pods are created
by the Envoy Gateway controller and that is the only way to annotate them.
- **Compute units and JupyterLab:** these pods are created at runtime, so
their namespaces get the annotations (new `namespaceAnnotations` option)
instead of the pods. The pools' ResourceQuota rejects containers without
resource requests, so the namespace annotations also set requests for the
injected Linkerd containers.
- **Native sidecars:** all pods use
`config.linkerd.io/proxy-enable-native-sidecar`. Without this, jobs never
complete because the proxy keeps running, and init containers that need the
network (e.g. waiting for Lakekeeper) fail.
- **Postgres and RustFS:** their ports are marked as opaque so Linkerd
forwards them as plain TCP. Opaque traffic arrives on Linkerd's inbound port
4143, so the Postgres chart's NetworkPolicy now also allows 4143.
- **Prepuller memory:** raised the compute unit prepuller's memory limit
from 8Mi to 256Mi. VM runtimes such as Kata (#8572) size the pod's VM from its
memory limits, and 8Mi plus the Linkerd proxy is too small for the VM to boot.
Requests are unchanged, so scheduling is not affected.
Note for Kata: when Linkerd and Kata (#8572) are used together, Linkerd's
`linkerd-init` container fails inside the VM with `Permission denied (you must
be root)`. It runs as non-root and gets its network permissions from file
capabilities on the `iptables` binary, which Kata's virtiofs drops by default.
This is a host level Kata setting, not part of this chart. It can be fixed by
passing only file capabilities through, by adding this to
`virtio_fs_extra_args` in the Kata configuration:
```
--xattrmap=:ok:all:security.capability:::bad:all:::
```
### Any related issues, documentation, discussions?
<!--
Please use this section to link other resources if not mentioned already.
1. If this PR fixes an issue, please include `Fixes #1234`, `Resolves
#1234`
or `Closes #1234`. If it is only related, simply mention the issue
number.
2. If there is design documentation, please add the link.
3. If there is a discussion in the mailing list, please add the link.
-->
Closes #8970
Discussed in #6059
Has changes suporting #8572
### How was this PR tested?
<!--
If tests were added, say they were added here. Or simply mention that if the
PR
is tested with existing test cases. Make sure to include/update test cases
that
check the changes thoroughly including negative and positive cases if
possible.
If it was tested in a way different from regular unit tests, please clarify
how
you tested step by step, ideally copy and paste-able, so that other
reviewers can
test and check, and descendants can verify in the future. If tests were not
added,
please describe why they were not added and/or why it was difficult to add.
-->
Tested on a single node RKE2 v1.35.6 cluster with Linkerd edge-26.9.3, with
and without the Texera micro services running in Kata VMs
(`kata-qemu-runtime-rs`):
- Every pod starts with the Linkerd proxy (2/2 ready), including the pods
created at runtime.
- Checked that traffic between services is encrypted using the proxies'
metrics: all connections to other Texera services (including Postgres, LakeFS
and Lakekeeper) report `tls="true"` in addition to packet sniffing with
wireshark.
```
kubectl get --raw
"/api/v1/namespaces/<namespace>/pods/<pod>:4191/proxy/metrics" | grep
tcp_open_total
```
- The UI works, compute units can be created, and the setup jobs complete.
### Was this PR authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please include the phrase: 'Generated-by: ' followed by the name of the tool
and its version. If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
Generated-by: Claude Opus 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
