hansva opened a new issue, #9294: URL: https://github.com/apache/hop/issues/9294
### Apache Hop version? 2.20 ### Java version? 21 ### Operating system Other ### What happened? Filter rows has to compare a field in ascending order, even when the field is marked as sorted descending, or every comparison on that field is inverted. `FilterRowsMeta.getFields()` takes care of this by clearing the `sortedDescending` flag on each field the condition uses. At runtime, `FilterRows.processRow()` calls `getFields()` directly on `getInputRowMeta()`. That object is not a copy. It is the row metadata the previous transform passes along with its rows. When that transform sends its rows over more than one hop, other transforms read the same object too. As a result, after the first row Filter rows has changed metadata it does not own: - The previous transform's output fields lose their descending flag at runtime. - Any other transform that reads the same row metadata, for example a second hop out of a Sort rows transform, now compares those fields in ascending order. Depending on the transform, that gives wrong merge, grouping or comparison results. - What happens depends on which transform processes its first row first, so it is hard to reproduce. The comparison inside Filter rows itself is correct. Only the side effect on the shared metadata is wrong. ### Steps to reproduce 1. Sort rows: sort on `value`, descending. 2. Send its output to Filter rows with the condition `value < 10`, and also over a second hop to a transform that relies on the descending order of `value`. 3. After Filter rows has processed its first row, `value` is no longer marked as sorted descending in the row metadata the second transform receives. A unit test reproduces this directly: the input row metadata has `value` (Integer, sorted descending) and the row is `5`. The row is kept, because `5 < 10`, but afterwards the input row metadata no longer has `value` marked as sorted descending. ### Proposed fix On the first row, clone the input row metadata into `FilterRowsData.conditionRowMeta`, call `getFields()` on the clone, and evaluate the condition with the clone. The input row metadata stays untouched, and the comparison stays ascending as before. A test in `FilterRowsTest` covers both points: the row is kept, and the shared input row metadata keeps its descending flag. ### Issue Priority Priority: 3 ### Issue Component Component: Transforms -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
