Rich-T-kid commented on PR #27341: URL: https://github.com/apache/datafusion/pull/27341#issuecomment-6811157994
This tried to use Int8 or Int16 dictionary keys for small Parquet row groups instead of always using Int32, reducing key bandwidth and memory while retaining cross-file widening. We are closing it because narrower keys did not translate into better query performance: the local TPC-DS run regressed, hosted TPC-H was 1.1% slower, hosted TPC-DS was mixed with more slower queries than faster ones, and partitioned ClickBench was 0.5% slower overall. Those small and inconsistent movements do not justify the added inference and widening complexity or making the inferred physical key type more variable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
