Doris-Breakwater commented on issue #69218: URL: https://github.com/apache/doris/issues/69218#issuecomment-6236450560
Breakwater-GitHub-Analysis-Slot: slot_1a5636aa9ba7 **Assessment:** The reported V2 VARIANT double-rendering defect is supported by current upstream source and an independent formatter reproduction. This is a numeric correctness issue at the JSON text boundary. The issue currently has no labels; `bug` and the appropriate VARIANT component label would fit. A fix is already proposed in [PR #69219](https://github.com/apache/doris/pull/69219), which is open and unmerged at the time of this check. **Verified mechanism** — inspected upstream master at `f937d950d45313f20864d287df453e1b90f5c615`: - JSON ingestion passes `element.getDouble()` to `add_double()`, which constructs a double scalar. There is no decimal text formatting at this step. See [the ingestion path](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/parse/variant_string_parse.cpp#L152) and [the builder](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/core/value/variant/variant_batch_builder.cpp#L1679). - Object/array casts use the fallback JSON writer. Its finite-double branch calls `format_json_double()`, whose precision is `digits10 + 1`, i.e. 16 significant digits. Binary64 requires up to 17 digits for guaranteed round-trip conversion. See [the cast routing](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/cast/variant_v2/cast_variant_to_string.cpp#L70), [the JSON double branch](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/parse/variant_string_parse.h#L340), and [the formatter](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/parse/variant_string_parse.cpp#L272). - A scalar DOUBLE cast instead takes the concrete scalar conversion path; its finite-number formatter already uses `FMT_COMPILE("{}")`. This explains the document/element discrepancy. See [the scalar routing](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/cast/variant_v2/cast_variant_to_string.cpp#L36) and [the existing scalar formatter](https://github.com/apache/doris/blob/f937d950d45313f20864d287df453e1b90f5c615/be/src/exprs/function/cast/cast_to_string.h#L125). **Independent validation:** I compiled an isolated C++ check using the exact old/proposed formatting expressions and upstream's declared fmt 7.1.3. Parsing the result back to a double and comparing bits reproduced both reported collisions: | Input double | Current formatter | Default `{}` formatter | | --- | --- | --- | | `0.30000000000000004` | `0.3` | `0.30000000000000004` | | `1997.6000000000001` | `1997.6` | `1997.6000000000001` | | `123456789012345680.0` | `1.234567890123457e+17` | `1.2345678901234568e+17` | The proposed formatter passed bitwise round-trip checks for all seven listed doubles, selected boundary values (including signed zero, minimum normal/subnormal and maximum finite double), and 99,955 finite doubles sampled from randomly generated bit patterns. This validates the formatter mechanism; I did **not** build Doris, execute the SQL reproduction, or run the PR's Doris tests. The issue's “roughly one in five” frequency is not established by this investigation and depends on the input distribution. **Recommended next steps / remaining evidence:** 1. Continue with PR #69219's localized change to default `{}` formatting, which matches the existing scalar cast implementation. Its added test covers the seven values across typed/encoded SerDe output surfaces and the example document; test execution still needs CI evidence. 2. Add/run an end-to-end SQL regression for the supplied query and compare numeric values before and after the document-text round trip. Include neighboring representable values and finite extremes, preferably with bitwise round-trip assertions in a focused unit test. Keep non-finite JSON handling covered; the writer handles it separately. 3. Record the reporter's exact build commit and any nondefault settings used to select V2 ingestion, plus before/after SQL output on the PR build. These are needed to pin the affected build and verify SQL behavior, rather than to establish the formatter defect. Crash logs or a query profile are not needed for this deterministic rendering case. Released-version impact and backport targets remain unverified. For affected pipelines, extracting/casting the numeric element directly avoids this document-text formatter where practical. Previously rounded text alone cannot recover the original double; retained source VARIANT data should be used for regeneration after a validated fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
