aicam opened a new pull request, #6617: URL: https://github.com/apache/texera/pull/6617
### What changes were proposed in this PR? Roadmap item 3 from #5891 — **core-services node placement + autoscaler safety** — ported onto the `base`/`aws`/`on-prem` layout. Two complementary mechanisms, **both no-op by default**, so a cluster autoscaler (e.g. Karpenter) can reclaim idle/transient nodes while always-on services stay available: 1. **`coreServices` node placement** (`values.yaml`): a `nodeSelector` + `tolerations` block, **empty by default** (no-op for local / on-prem), wired into the always-on singleton **computing-unit manager**. An AWS overlay pins it onto a dedicated, long-lived core-services node pool so it never lands on — and thereby blocks scale-down of — a transient/CU node. 2. **PodDisruptionBudgets** (`maxUnavailable: 1`) for the replicated, stateless app services — new `templates/base/pod-disruption-budgets/app-service-pdbs.yaml`. They stay continuously available while remaining relocatable, so the autoscaler can bin-pack and drain idle nodes. The three buckets, and why each service lands where it does: | Bucket | Services | PDB? | `coreServices` pin? | |--------|----------|------|---------------------| | Replicated & stateless | webserver, access-control, config, file, litellm, workflow-compiling | ✅ | ❌ | | Stateful / session-affine | agent-service, shared-editing-server, pylsp | ❌ | ❌ | | Always-on singleton | cu-manager (AWS also: lakekeeper, gateway proxy) | ❌ | ✅ | - **agent-service is intentionally excluded from the PDBs** — it holds per-workflow agent state in memory and is pinned per-workflow by a consistent-hash `BackendTrafficPolicy` on `X-Agent-Workflow-Id`; relocating a replica would sever in-flight sessions. - Deliberately **not** using `karpenter.sh/do-not-disrupt`: on replicated services it blocks consolidation, and on singletons/bare pods it can pin a node indefinitely (orphaned capacity — a real downstream incident that mass-killed live computing units). PDBs + node placement give the protection without that side effect. - Subcharts (postgres, minio, lakefs, lakekeeper) are out of scope — they manage their own StatefulSets/PVCs and emit their own PDBs. `values-aws.yaml` gains an example `coreServices` pin (node-pool label + toleration). ### Any related issues, documentation, discussions? Part of #5891 (item 3 of 6). Follows the template reorg (#5757) and the pluggable object-storage work (#5932, #6295). Supersedes the review-only #6609 (opened to eyeball the diff, then closed). Ports a design validated downstream. ### How was this PR tested? `helm template` render, default vs AWS overlay: ```bash cd bin/k8s && helm dependency build . # default: placement is a no-op — cu-manager pod spec unchanged (empty coreServices renders nothing) helm template test . | grep -c 'nodeSelector' # cu-manager: 0 # AWS overlay: cu-manager pinned onto the core-services node pool helm template test . -f values-aws.yaml | grep 'texera.io/node-pool' # PDBs: exactly 6, agent-service absent helm template test . --show-only templates/base/pod-disruption-budgets/app-service-pdbs.yaml ``` - **Default render:** cu-manager pod spec byte-identical to `main`; the 6 app-service PDBs render (inert on a static cluster — a PDB only gates *voluntary* evictions). - **AWS overlay render:** cu-manager carries the `nodeSelector` + toleration; all 6 PDBs present, agent-service excluded. - Also brought up end-to-end on minikube (`values-development.yaml`) — stack healthy, no placement/PDB regressions. **Open question for reviewers:** the 6 PDBs render unconditionally, so the *default* render is not byte-identical to `main` (it gains 6 inert objects). This matches downstream/best-practice, but happy to gate them behind a flag if the "default render strictly unchanged" guarantee should cover them too. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
