Doris-Breakwater commented on issue #68715: URL: https://github.com/apache/doris/issues/68715#issuecomment-5976951547
Breakwater-GitHub-Analysis-Slot: slot_5ab6a687cb2f **Triage: reproducible correctness issue in floating-point inverted-index comparisons (high priority).** The supplied four-row reproducer and the scan/index toggle are sufficient to isolate the symptom to index evaluation. I have reviewed the code at `master` commit `0c29961f45e` and the local `4.1.4-rc04` tag; I have not independently run a Doris cluster. **Code evidence / likely cause:** The FLOAT/DOUBLE key coder deliberately gives the two zeros different sortable keys: FLOAT `-0.0` is `7FFFFFFF` and `+0.0` is `80000000` (DOUBLE `7FFFFFFFFFFFFFFF` and `8000000000000000`). See [key encoding](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/key_coder.h#L359-L440) and [the existing special-value test](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/test/storage/key_coder_test.cpp#L751-L817). The [inverted-index writer](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_writer.cpp#L555-L584) stores that encoding in BKD. The [BKD reader](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_reader.cpp#L668-L691) encodes the predicate value into its search bounds, and its [visitor compares the encoded bytes](https://githu b.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/index/inverted/inverted_index_reader.cpp#L946-L1087). Thus an equality probe for `+0.0` misses the stored `-0.0`, `< +0.0` includes it, and `>= +0.0` excludes it. The [comparison predicate](https://github.com/apache/doris/blob/0c29961f45ee976ef61c6a5dfe7b38b40a6ec131/be/src/storage/predicate/comparison_predicate.h#L74-L123) implements `!=` by subtracting equality matches, so the same mismatch also explains its extra row. These are source-confirmed mechanisms consistent with the reported output; the exact execution path of each supplied SQL query still warrants an `EXPLAIN`/profile check. **Immediate mitigation:** For queries on affected FLOAT/DOUBLE indexed columns, use `SET enable_inverted_index_query = false` in the querying session, as the issue's control query demonstrates, until a correctness fix is available. This trades index performance for scan semantics. **Maintainer next steps:** Add a regression test with both signed zeros and both FLOAT and DOUBLE, covering `=`, `!=`, `<`, `<=`, `>`, `>=` with `+0.0` and `-0.0` predicate values. Make index comparisons treat the two zeros as one SQL equivalence class. Please account for *existing* index files, which already hold distinct encodings: changing only newly written keys would leave old data incorrect; a key-coder-wide format change also needs compatibility review. Verify the chosen fix on both newly built and pre-existing indexes in the affected release lines. **Useful confirmation from the reporter:** The posted actual results are identified as master output. Please attach the same query results from 4.1.4, plus `EXPLAIN` (or a query profile) for one indexed and one index-disabled query on each version, to confirm which index path each build used. The SQL and master output already provide enough detail for initial root-cause triage; no additional logs are needed to start work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
