nzw921rx opened a new issue, #13328:
URL: https://github.com/apache/seatunnel/issues/13328

   ### Background
   
   The umbrella #11857 and STIP-41 (#12457) describe running one Zeta job as an 
isolated application on an external resource manager. The Kubernetes 
implementation in #12460 introduced the shared Application Mode lifecycle. This 
issue tracks the native YARN provider.
   
   ### Application lifecycle
   
   - Submit one SeaTunnel job to a YARN queue and return its application ID. 
The ApplicationMaster runs the Zeta master and requests a fixed number of 
worker containers from the ResourceManager. Use the application ID to check 
status or cancel the job.
   - Localize the same SeaTunnel distribution from a local archive or HDFS for 
the AM and workers. Accept a job file from the submitter or HDFS, or define it 
inline. Distribute role-specific runtime configuration through YARN local 
resources.
   - Release workers and application-owned staging on completion, failure or 
cancellation. Use one AM attempt. To recover, submit a new application with 
`--restore-job-id` and the original durable checkpoint storage.
   - Allow the operator to enable the AM HTTP endpoint on an explicit port and 
publish its tracking URL to YARN.
   
   ### Kerberos
   
   As an extension to the initial STIP scope, support both simple Hadoop 
clusters and Kerberos clusters. Use the submitter's ticket cache or a local 
principal/keytab to acquire a delegation token for staging HDFS. Designate the 
ResourceManager principal as the renewer and forward the HDFS token to workers. 
The AM keeps its ResourceManager control token and the keytab stays on the 
submission host.
   
   ### Acceptance criteria
   
   - Exercise submission, worker allocation, batch completion, cancellation, 
worker loss and checkpoint restore on MiniYARNCluster with MiniDFSCluster.
   - Verify both login paths against MiniKdc and secure HDFS, including worker 
access with the forwarded token.
   - Provide a runnable deployment template and English and Chinese guides for 
submission, configuration, HTTP access and recovery.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to