andygrove opened a new issue, #306: URL: https://github.com/apache/datafusion-spark/issues/306
### Describe the bug `unix_date` only works when the optimizer rewrites it. `simplify` ([datetime/unix.rs#L89](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/datetime/unix.rs#L89)) replaces it with casts to `Date32` and then `Int32`, and `invoke_with_args` ([datetime/unix.rs#L85](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/datetime/unix.rs#L85)) always returns an error. Any caller that evaluates the function without running `simplify` gets that error instead of a result. Every function needs a working `invoke_with_args`. `simplify` is an optional optimization on top of it. ### To Reproduce Build a session with the `simplify_expressions` optimizer rule removed, so the expression is not constant-folded and `ScalarUDFImpl::simplify` is never called. This is how engines that build physical expressions directly, such as Comet, run the function. ```rust let rules = Optimizer::new() .rules .into_iter() .filter(|rule| rule.name() != "simplify_expressions") .collect(); let state = SessionStateBuilder::new() .with_default_features() .with_spark_features() .with_optimizer_rules(rules) .build(); ``` ```sql SELECT unix_date('1970-01-02'::date); -- Internal error: invoke_with_args should not be called on SparkUnixDate. ``` ### Expected behavior `invoke_with_args` computes the same result as the rewritten expression, for scalar and array arguments. `simplify` can stay as an optimization. ### Additional context Found by running the `.slt` files with `simplify_expressions` disabled. The new `evaluate` mode of the sqllogictest runner does this, and it lists `datetime/unix.slt` as a known failure until this is fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
