lakshaydevoo-arch commented on issue #26848: URL: https://github.com/apache/datafusion/issues/26848#issuecomment-6528879207
I compared Int8, Int32, and UInt64 dictionary keys across 0%, 3%, 50%, and 97% null densities, using the same 8,192-row workloads and three benchmark runs. The results showed no consistent performance advantage for smaller key types. Most differences were within the observed benchmark variation, so I don't think additional key-width benchmarks are justified upstream. Separately, I've implemented and tested a focused optimization for nullable dictionary hash scattering. On my x86 setup, it reduces execution time by approximately 16–21% for the main 3% null-key cases, with larger improvements at higher null densities. Would a PR focused on this nullable-path improvement be useful, keeping non-null-path optimization separate? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
