pepijnve commented on issue #17801: URL: https://github.com/apache/datafusion/issues/17801#issuecomment-3497870319
@Omega359 ok so that would rule out option 2. Then we're back to the implementation challenges of option 1. The challenge there is how you asses the nullability of something like ``` CASE WHEN x IS NOT NULL THEN x ELSE y END ``` at the logical level. The current implementation is essentially check is any 'then' expression or the 'else' expression is nullable. In the example above `x` is nullable so the entire thing is deemed nullable. The idea I had to make this more accurate was to insert try to evaluate then when expression assuming the then expression would evaluate to `NULL`. If we inject a binding of `x` (the 'then' expression) to `NULL` in the example above the case expression becomes `CASE WHEN NULL IS NOT NULL THEN NULL ELSE y END`. It's obvious that the 'then' expression is not effectively reachable when it would evaluate to `NULL` so its nullability is irrelevant for the nullability of the case expression. So far so good, but how do you actually implement that in `expr`? The problem I ran into is that from `expr` I can't use the technique the constant evaluation optimiser uses of mapping to `PhysicalExpr` and evaluating with a dummy `RecordBatch`. So the best I could come up with is to make a simple const evaluator for predicates based on just the logical expressions. Not really an ideal solution since I'm basically reimplementing the physical expr implementations (partially). At that point I ran out of ideas. It might be feasible to reorganise the code so that the code in `expr_schema.rs` can use `physical-expr`, but it doesn't look like a trivial refactoring to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
