aicam commented on issue #5891: URL: https://github.com/apache/texera/issues/5891#issuecomment-4860584427
## Plan update: adopt Terraform for AWS provisioning; **not** using eksctl We've revisited the provisioning approach and decided **not to go with eksctl** (originally selected in the #5641 design discussion, which had ruled Terraform out as "overkill"). We're switching the AWS substrate to **Terraform**. ### Why the reversal The eksctl decision was made when the scope was "provision the cluster + IRSA, leave everything else manual." In practice the AWS substrate we need to standardize is broader than the cluster — S3 buckets (app tier, LakeFS blockstore, Lakekeeper Iceberg warehouse), IAM/IRSA roles, Route53/ACM, OIDC, EBS CSI + gp3 StorageClass, and the load-balancer/DNS wiring. eksctl only covers cluster + node groups + IRSA and leaves S3, DNS, and certs manual, which is exactly the reproducibility gap this issue is about. Terraform lets us codify the *entire* account setup as one reviewable, reproducible stack, and it unblocks the bigger win: replacing the static IAM access keys in `values-aws.yaml` with **IRSA** (no long-lived keys, per-workload least privilege). ### What changes vs. the original checklist - **Task 6 is rewritten**: "eksctl `ClusterConfig` + EBS StorageClass manifests + runbook" → **"Terraform root module + modules (`vpc`, `eks`, `storage-s3`, `irsa`, `dns-cert`, `addons`) + runbook"**, under a new `deployment/terraform/` tree. - **Tasks 1–5 are unaffected** — they're Helm chart concerns (template reorg ✅, pluggable object storage 🟡 #5932 + follow-up, node placement, EnvoyProxy NLB, warm CU pool). They stay cloud-provisioner-agnostic and simply consume Terraform outputs (IRSA role ARNs, ACM cert ARN, subnet IDs, node-group labels/taints) at install time. - **New follow-up chart PR**: service-account IRSA annotations + keyless S3 mode in `_helpers.tpl`, flipping `values-aws.yaml` from static access keys to IRSA (with a static-key/hybrid fallback if LakeFS/Lakekeeper keyless support proves incomplete). - Everything remains **non-breaking**: on-prem/local Helm install is untouched; AWS stays opt-in via `values-aws.yaml` + a Terraform-generated `values-aws.generated.yaml`. ### Updated Task 6 sub-PRs (small, independently reviewable) - **6a** — `vpc` + `eks` + OIDC + managed node groups (system pool + tainted CU pool). - **6b** — `storage-s3` (3 buckets) + `irsa` (per-workload roles, outputs). - **6c** — `addons` (EBS CSI/gp3 StorageClass, AWS LB Controller, external-dns, cert-manager/ACM, autoscaler/Karpenter). - **6d** — `dns-cert` + `values-aws.generated.tf` + end-to-end runbook + `terraform destroy` teardown notes. I'll also add an addendum to the #5641 design doc recording this Terraform-adoption decision so the history is clear. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
