alamb commented on a change in pull request #1595:
URL: https://github.com/apache/arrow-datafusion/pull/1595#discussion_r787114561
##########
File path: datafusion/src/physical_plan/file_format/parquet.rs
##########
@@ -757,10 +784,8 @@ mod tests {
.enumerate()
.map(|(i, g)| row_group_predicate(g, i))
.collect::<Vec<_>>();
- // no row group is filtered out because the predicate expression can't
be evaluated
- // when a null array is generated for a statistics column,
- // because the null values propagate to the end result, making the
predicate result undefined
- assert_eq!(row_group_filter, vec![true, true]);
+ // First row group was filtered out because it contains no null value
on "c2".
+ assert_eq!(row_group_filter, vec![false, true]);
Review comment:
In sql `IsNull` is the correct way to test a column for null as well 👍
It would make a lot of sense to me to rewrite `x IS NULL` --> `0 >
x_null_count`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
