asolimando commented on PR #24027: URL: https://github.com/apache/datafusion/pull/24027#issuecomment-5194142976
> That's the part I understand, but say we have a schema with a single string column and the table provider (or any other source) reports exact size and and exact row count, my understanding is that the current behavior (even before this PR) will always set it to `Inexact`. OK, I see what you mean. The only caller of this function today is [projection.rs#L724](https://github.com/apache/datafusion/blob/152d8c47eb6b7ddbaa40ec26f24d714c24572d40/datafusion/physical-expr/src/projection.rs#L724): if we used `byte_size` from `ColumnStatistics` instead of `DataType::primitive_width()`, I think we could keep exact (provided that the column-level byte sizes are exact too), even for non-primitive types (length varying types). This of course depends on the projection expressions, if they are pure input column references (dropping some columns, for instance), and that columns have exact byte size values, it would be doable. When you say it doesn't matter much in practice I guess it's because Parquet sets `Inexact` for non-primitive data types. If you agree on my reading and it feels useful, I can file a follow-up issue to track this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
