bamaer commented on PR #9341:
URL: https://github.com/apache/hop/pull/9341#issuecomment-6798643975

   Thanks @hansva, all addressed in the latest push, which is also rebased on 
the updated #9339 and #9340. I reproduced the stale-retire, parent-edge and 
same-name cases with hop-run against DataHub 1.7 before fixing them, and 
checked them again after.
   
   1. **Retire (L227):** a task is now only retired if it hasn't run since the 
run being checked. For a late event, the newest run comes from DataHub; within 
one process, the sink also remembers what it delivered. A late event no longer 
writes flow/task aspects. With your v1/v2 case, "Sort rows" stays.
   2. **Dedupe (L166):** the first task's I/O UPSERT now keeps the parent edges 
DataHub already has (read before writing), so neither cache expiry nor a second 
parent action drops them. Checked with two workflows running the same pipeline 
from separate processes.
   3. **Batch outcome (L188):** each event now gets its own result.
   4. **Failed read (L306):** a read that may work later now makes the event 
RETRY, and the lineage is still written right away. A query DataHub can't run 
is skipped instead of retried forever. The "reported once" flag resets after a 
successful read.
   5. **Fallback (L419):** refused batches are bisected with a 40-request 
budget, and the backoff is capped at 30 s.
   6. **URNs (L61):** the flow id is now the file path in the project 
(`pipelines/load.hpl`), or the name when there is no file. The parent's file is 
passed along so action → pipeline links still resolve.
   7. **Marketplace (L20):** listed in `optional-plugins.yaml` with a note that 
it needs the OpenLineage plugin, since the marketplace has no dependency field. 
The hub now names the missing class instead of a bare NoClassDefFoundError.
   8. **Open runs (L173):** an action whose `execute()` throws now ends as 
FAILED (the #9339 fix). Any run still open at shutdown is ended with ABORT and 
shows as failed.
   9. **Hash (L273):** SHA-256.
   10. **Getters (L279):** Lombok, and `Locale` is imported.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to