bamaer commented on PR #9340:
URL: https://github.com/apache/hop/pull/9340#issuecomment-6797084096

   Thanks @hansva, all addressed in 5959374:
   
   - A batch stops at the first failure and the worker backs off (1 s doubling 
to 60 s), keeping the failed event and everything after it at the head, so 
order is kept. Shutdown no longer waits for a collector that is down.
   - Events leave the pending list only once settled, so `awaitDrain` and the 
shutdown count include in-flight ones. The worker closes the emitter itself.
   - The hub thread only queues; a spooler thread writes the spool. Replay is 
one scan of file names, and events are read when sent.
   - Each process spools into its own folder with a heartbeat and only takes 
over folders whose owner shut down or went stale (moving the files, so one 
process sends each).
   - `accept`/`shutdown` share a lock, and late batches are ignored.
   - `PendingRun` keeps each dataset once.
   - `HOP_LINEAGE_OPENLINEAGE_FILE_SPEC_NAMING` is back, default `N` (legacy 
naming), so an upgrade doesn't re-key file datasets.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to