Rich-T-kid commented on issue #26696: URL: https://github.com/apache/datafusion/issues/26696#issuecomment-6424987483
>enable_rle_to_dictionary promotes every string column that has dictionary pages, and the planner can't guarantee low cardinality anyway: Parquet dictionaries are per row group, while a GROUP BY's output holds every distinct value in the partition. we are working on this in https://github.com/apache/datafusion/pull/26315 and https://github.com/apache/datafusion/issues/24116 + https://github.com/apache/datafusion/issues/24117. #26315 is in a ready state but the issue is the need to take more factors into consideration that NDV doesn't provide for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
