tyrobro opened a new pull request, #301:
URL: https://github.com/apache/datafusion-spark/pull/301

   ## Which issue does this PR close?
   
   Closes #268.
   
   ## Rationale for this change
   
   Spark only evaluates `expr2` when `expr1` is NaN. DataFusion core's `nanvl`
   evaluates both arguments eagerly, so it can't be reused directly without
   surfacing errors Spark would never raise (e.g. `1 / 0` under ANSI mode).
   
   ## What changes are included in this PR?
   
   - Adds `SparkNanvl` (`crates/spark/src/function/conditional/nanvl.rs`):
     - `coerce_types`: FLOAT/FLOAT stays FLOAT; everything else (mixed 
FLOAT/DOUBLE,
       int, DECIMAL, STRING) coerces to DOUBLE.
     - `simplify()` rewrites the call to `CASE WHEN expr1 != expr1 THEN expr2 
ELSE
       expr1 END`, so DataFusion's lazy `CASE` evaluation reproduces Spark's
       short-circuiting of `expr2` and its NULL-propagation rule.
     - `invoke_with_args()` keeps an eager fallback for physical-execution paths
       that skip the logical optimizer (e.g. Comet).
   - Registers `nanvl` in `conditional/mod.rs`.
   - Adds `crates/sqllogictest/test_files/conditional/nanvl.slt` covering NULL
     propagation, NaN bit patterns, ±Infinity/-0.0, FLOAT/DOUBLE/int/DECIMAL
     coercion, scalar+array, and lazy evaluation under ANSI mode.
   
   ## Are these changes tested?
   
   Yes, via the `.slt` file above.
   
   ## Are there any user-facing changes?
   
   Yes — `nanvl` is now available via 
`SessionStateBuilderSpark::with_spark_features()`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to