JeonDaehong opened a new pull request, #1177: URL: https://github.com/apache/paimon-rust/pull/1177
### Purpose Linked issue: close #1174 ### Brief change log The residual filter compares FLOAT/DOUBLE with Arrow kernels, which order floats by their bits. A NaN with the sign bit set, which is what x86 produces for `0.0 / 0.0`, sorts below `-Inf` there and is not equal to `NaN`. Java's `Float.compare` / `Double.compare` treat every NaN as one value above `+Inf`. `evaluate_column_predicate` now rewrites such NaNs in the column to `NaN` before comparing. The FLOAT/DOUBLE literal is canonicalized the same way. A column without a non-canonical NaN is used as-is. A probe of 8 values × 8 literals × 6 operators for FLOAT and DOUBLE against paimon-bundle 2.1.0 `PredicateBuilder` went from 112 mismatches out of 768 to 0. Row evaluation (`eval_row`) and the BTree index are not changed here (see #1174). ### Tests `float_negative_nan_filter_test` writes one file with `0.0`, a sign-bit NaN, `-2.0` and `NaN`, and checks `<`, `>`, BETWEEN, `= NaN`, `!= NaN`, `= -NaN` and IN on FLOAT and DOUBLE. Without the change, `f < -1.0` returns `[1, 2]`. `test_bitmap_floating_candidates_preserve_residual_results` (#601) used the residual's old answers as its baseline, for example a sign-bit NaN matching `< 0` and `= NaN` matching only that exact bit pattern. Its expected rows now follow Java. The bitmap index still returns the same rows as the residual in every case. Ran the ci.yml steps locally (fmt, clippy, msrv, unit, rest-server, integration, datafusion, python, go). ### API and Format None. ### Documentation None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
