nzw921rx opened a new pull request, #13329:
URL: https://github.com/apache/seatunnel/pull/13329

   ### Purpose of this pull request
   
   Implement the YARN provider for native Zeta Application Mode. Each 
application runs one SeaTunnel job. Its ApplicationMaster starts the Zeta 
master, allocates workers through Hadoop's asynchronous ResourceManager client, 
and releases application-owned resources when the job ends. Refs #13328 (parent 
#11857, STIP-41 #12457).
   
   ### Does this PR introduce any user-facing change?
   
   Yes. Submit with `bin/seatunnel-application.sh submit --target yarn 
--application-config yarn.conf --wait`. Use the returned YARN application ID 
for status and cancellation.
   
   - The AM and workers localize the same distribution from a local archive or 
HDFS. Deployment files can supply the job inline and distribute runtime 
configuration by role.
   - Simple and Kerberos clusters are supported. On Kerberos, use a ticket 
cache or submitter-side keytab. The ResourceManager principal can renew the 
staging HDFS token. Workers receive HDFS tokens, and the AM keeps its control 
token.
   - Set `yarn.master.http-port` to enable the AM endpoint on a fixed port and 
register its tracking URL. The bundled UI uses root-relative asset and API 
paths, which deployments must route when exposing it through a reverse proxy.
   - Each application gets one AM attempt. A new submission can use 
`--restore-job-id` to recover from a durable checkpoint.
   
   ### How was this patch tested?
   
   Spotless passed on the affected reactor. On macOS with JDK 11, 108 unit 
tests and 12 integration tests passed with zero failures or skips. 
MiniYARNCluster and MiniDFSCluster exercised local and HDFS archives, HTTP 
access, worker loss, cancellation and checkpoint restore. MiniKdc and secure 
HDFS exercised both submitter login methods, token renewal by the configured 
renewer, and worker reads with forwarded credentials.
   
   ### Check list
   
   - [x] Updated the distribution template and both the English and Chinese 
user guides.
   - [x] Added provider, CLI and MiniCluster coverage. The provider uses the 
existing shaded Hadoop runtime. Hadoop MiniCluster, MiniKdc and the JAXB 
activation API are test dependencies.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to