anshulbaliga7 opened a new pull request, #13462: URL: https://github.com/apache/gluten/pull/13462
## What changes are proposed in this pull request? Velox's `str_to_map` now honors `throw_exception_on_duplicate_map_keys` (facebookincubator/velox#19519), which Gluten already derives from `spark.sql.mapKeyDedupPolicy` for every native task. The fix is in the pinned Velox since `dft-2026_10_22`. Under LAST_WIN it keeps the position of a repeated key's first occurrence and takes the last value, matching Spark's `ArrayBasedMapBuilder`, and entries without a key/value delimiter take part in deduplication the way they do in Spark. This removes the Velox backend's `genStringToMapTransformer` override, which rejected `str_to_map` whenever the policy was not EXCEPTION, together with the `StrToMapRestrictions` object it reported. The default hook returns `GenericExpressionTransformer`. The Bolt backend keeps its own copy of the restriction and is unaffected. The LAST_WIN guard on `transform_keys` in `ExpressionConverter` stays, since it tracks a different native gap (facebookincubator/velox#10219). ## How was this patch tested? Added a LAST_WIN test to `ScalarFunctionsValidateSuite` with repeated keys and key-only entries. It asserts the Project offloads and that the result, including `map_keys` order, matches vanilla Spark. The existing EXCEPTION test is unchanged. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude claude-opus-4-8 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
