SEZ9 commented on issue #11857: URL: https://github.com/apache/seatunnel/issues/11857#issuecomment-5674413226
Thanks for the follow-up. Keeping this on the design track and letting the STIP drive the discussion sounds right, and starting with YARN makes sense given the motivation described above. @det101, since you offered to help with design/implementation, it would be great if you could take the lead on the STIP for a YARN-first version. To keep the scope narrow and consistent with what was already discussed, I'd suggest it cover at minimum: 1. One job == one application / one Zeta cluster for v1. 2. A clear separation between application lifecycle (bootstrap control plane, join cluster, cleanup) and worker lifecycle (provision / release worker instances), including who owns each state transition. 3. Master/control-plane placement and the worker join path. 4. Failure and cleanup semantics for application bootstrap, worker provisioning, and job completion/failure, including partial-failure cleanup. 5. The minimum config/package changes needed for v1, with autoscaling, HA, Kerberos, and multi-job tenancy explicitly out of scope. If the high-level lifecycle states are defined in a platform-neutral way, a Kubernetes follow-up can reuse them later while keeping platform-specific details underneath, but I'd treat that as a separate implementation issue rather than part of this first STIP. @nzw921rx, once the STIP is posted, could you link it back here so we can continue the in-depth discussion in one place? If either of you has questions about the current placeholders while writing it up, feel free to raise them in this thread. <!-- streview-comment:1067 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
