SEZ9 commented on issue #12566: URL: https://github.com/apache/seatunnel/issues/12566#issuecomment-5947238679
Thanks for the detailed report. To clarify where things stand on the two observations you raised: **Full-cluster restart recovery:** The current `dev` recovery path, `restoreAllRunningJobFromMasterNodeSwitch()`, is triggered when a node becomes the active master. It scans the shared running-job map and simply returns if that map is empty. That gives us master-switch recovery, but it does not establish an automatic, durable recovery contract after every master and worker has restarted. In particular, nothing in that path waits for a full-cluster MapStore reload to finish, so the empty `runningJobInfoIMap` you observed with EAGER async loading is consistent with the current design rather than a race the code currently guards against. **Non-savepoint jobs resyncing from scratch:** `CheckpointManager` restores checkpoint state only when the job is explicitly a restore job and has a non-null `restoreSourceJobId`. An ordinary submission does not get a synthesized restore source, so the from-scratch behaviour you saw is expected under today's semantics. So, to answer your question directly: Zeta is not currently designed to auto-recover running jobs after a full cluster restart. Until a full-cluster-restart contract is designed and verified, please use an explicit savepoint/restore workflow for controlled recovery rather than relying on automatic resumption. If you'd like to turn these observations into a design or bug proposal, it would help a lot if you could add: 1. The exact `dev` commit you tested against. 2. Your MapStore configuration and startup logs showing the map-load timing relative to the master's recovery scan. 3. A minimal before/after restart case that records the saved checkpoint and the source offsets the job resumed from. That will let us separate the persistence-readiness question (MapStore load timing) from the checkpoint-selection semantics (`restoreSourceJobId`) and decide how each should be addressed. <!-- streview-comment:1474 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
