Rich-T-kid opened a new pull request, #27312: URL: https://github.com/apache/datafusion/pull/27312
Depends on apache/arrow-rs#12161. When RLE dictionary reads are enabled, Parquet physical Binary columns were not promoted to the requested Dictionary(Int32, Utf8) schema. The pushed RowFilter therefore evaluated a cast from Binary to Dictionary(Utf8) per batch. This change permits the existing binary-to-string coercion for dictionary value types and pins Arrow to the dependent fix that honors the requested dictionary value type. Local partitioned ClickBench, 12 partitions, pushdown enabled, RLE dictionary reads enabled on both parent and head, six iterations per query (median): - Q10: 122.5 ms -> 46.9 ms (2.61x, 61.7% faster) - Q12: 646.0 ms -> 393.3 ms (1.64x, 39.1% faster) - Q25: 353.2 ms -> 172.9 ms (2.04x, 51.1% faster) - Q27: 1231.1 ms -> 736.6 ms (1.67x, 40.2% faster) All runs returned the same row counts. This remains draft pending hosted partitioned ClickBench confirmation and the Arrow dependency PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
