alamb commented on code in PR #11586:
URL: https://github.com/apache/datafusion/pull/11586#discussion_r1686982702
##########
datafusion/physical-expr/src/expressions/is_null.rs:
##########
@@ -117,6 +117,21 @@ pub(crate) fn compute_is_null(array: ArrayRef) ->
Result<BooleanArray> {
}
}
+/// workaround <https://github.com/apache/arrow-rs/issues/6017>,
+/// this can be replaced with a direct call to `arrow::compute::is_not_null`
once it's fixed.
+pub(crate) fn compute_is_not_null(array: ArrayRef) -> Result<BooleanArray> {
Review Comment:
this code looks good to me -- I think it also ends up supporting
`UnionArray` in `IS NOT NULL`
cc @samuelcolvin who added this code in
https://github.com/apache/datafusion/pull/11321
##########
datafusion/physical-expr/src/expressions/is_null.rs:
##########
@@ -117,6 +117,21 @@ pub(crate) fn compute_is_null(array: ArrayRef) ->
Result<BooleanArray> {
}
}
+/// workaround <https://github.com/apache/arrow-rs/issues/6017>,
+/// this can be replaced with a direct call to `arrow::compute::is_not_null`
once it's fixed.
+pub(crate) fn compute_is_not_null(array: ArrayRef) -> Result<BooleanArray> {
+ if let Some(union_array) = array.as_any().downcast_ref::<UnionArray>() {
+ let is_null = if let Some(offsets) = union_array.offsets() {
+ dense_union_is_null(union_array, offsets)?
+ } else {
+ sparse_union_is_null(union_array)?
+ };
+ compute::not(&is_null).map_err(Into::into)
+ } else {
+ compute::is_not_null(array.as_ref()).map_err(Into::into)
Review Comment:
This goes faster because it calls a single kernel (`compute::is_not_null`)
rather than 2 (`is_null` and `not`)?
Could we add some basic tests for union? Perhaps following the model in
https://github.com/apache/datafusion/pull/11321 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
