jayzhan211 commented on PR #25602:
URL: https://github.com/apache/datafusion/pull/25602#issuecomment-5814556888
Per file, the adapter often rewrites the probe child to `CAST(b@0 AS
Int64)`: a nullable table column over a REQUIRED parquet column, or an Int32
file under an Int64 table. The bare-`Column` check then drops the bitmap
silently. On a 20-row-group file, `b BIGINT` kept 20/20 row groups while `b
BIGINT NOT NULL` kept 4/20. Please accept same-type or widening integer casts;
`stat_column_expr` already rewrites the cast onto `b_min`/`b_max`. Also add an
slt case with a nullable table over REQUIRED data.
```diff
- let column = column_expr.downcast_ref::<phys_expr::Column>()?;
+ let column = match column_expr.downcast_ref::<phys_expr::CastExpr>() {
+ Some(cast) => {
+ let column = cast.expr().downcast_ref::<phys_expr::Column>()?;
+ let from = schema.fields().get(column.index())?.data_type();
+ if !is_integer_widening(from, cast.cast_type()) {
+ return None;
+ }
+ column
+ }
+ None => column_expr.downcast_ref::<phys_expr::Column>()?,
+ };
```
```rs
/// Casts that keep the bitmap's `u64` key ordering (sign/zero extension).
fn is_integer_widening(from: &DataType, to: &DataType) -> bool {
use DataType::*;
from == to
|| matches!(
(from, to),
(Int8, Int16 | Int32 | Int64)
| (Int16, Int32 | Int64)
| (Int32, Int64)
| (UInt8, Int16 | Int32 | Int64 | UInt16 | UInt32 | UInt64)
| (UInt16, Int32 | Int64 | UInt32 | UInt64)
| (UInt32, Int64 | UInt64)
)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
