peterxcli opened a new pull request, #20358: URL: https://github.com/apache/datafusion/pull/20358
## Which issue does this PR close? - Closes https://github.com/apache/datafusion/issues/20357. ## Rationale for this change neither datafusion-spark nor datafusion-comet expose the map expression implementation. so when you try to use map expression in spark sql, you will get: ``` Expected only Comet native operators, but found Project. plan: Project [COMET: map is not supported] datafusion hasnt had the map function, we should add one there before supporting it in comet. ``` https://github.com/apache/datafusion-comet/issues/3386 ## What changes are included in this PR? - the file is named `map_expr.rs` instead of `map.rs` is because the module name conflict - Implement ScalarUDFImpl trait for new Map struct and expose it to datafusion-spark map module - use comparison_coercion to automatically casting when returning type - use make_scalar_function - use the map_from_keys_values_offsets_nulls to unify spark specific logic and code reusing - transform the concat flatted keys and values with reorder_indices and arrow take ## Are these changes tested? y, with slt ## Are there any user-facing changes? not yet unless we port this to datafusion comet -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
