bamaer commented on PR #9251: URL: https://github.com/apache/hop/pull/9251#issuecomment-6635994789
@hansva thanks, this does what #9248 asks. I built the PR and ran the Update tests: all pass. Against main, the new H2 test fails as described (error hop gets 100, 200 instead of 47, 94, 141, 188, 235). **Should fix** - **Batch mode changes the output row order** ([Update.java:219](https://github.com/apache/hop/blob/e05e8208ee3c46d464b758f91135f1b056205895/plugins/transforms/update/src/main/java/org/apache/hop/pipeline/transforms/update/Update.java#L219)). Rows that need no update (or whose key isn't found with "ignore errors") are passed on right away, while updated rows wait for the batch. With even ids unchanged and odd ids updated, the output is `2, 4, … 198, 1, 3, 5, …`; on main it is `1, 2, 3, …`. This happens with batch updates on, with or without an error hop, and breaks anything downstream that expects sorted input (Group By, Merge Join). Suggested fix: while a batch is pending, hold those rows too (with the number of batched rows before them) and release them in arrival order as `flushBatch()`/`routeFailedBatch()` emit the batched rows. I tried that locally: all PR tests pass, output keeps input order, and the errors stay 47, 94, 141, 188, 235. Tested: the Update unit tests including the new H2 test, plus a local ordering test. Not tested against SQL Server or Oracle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
