Rich-T-kid commented on PR #27340: URL: https://github.com/apache/datafusion/pull/27340#issuecomment-6811164454
This tried to cache translations from independently encoded string or binary dictionaries to dense integer IDs, so a high-fanout single-column hash join could compare IDs instead of repeatedly resolving dictionary values. The focused HJ Q28 benchmark showed the intended effect: 201.20 ms to 155.42 ms on the hosted runner. We are still closing it because that win did not carry into an existing workload. TPC-H moved only 2.77 ms overall, with all 22 queries statistically unchanged; the TPC-DS movements did not reproduce and involved numeric join keys that cannot enter this path; and ClickBench has no joins. The optimization works for the constructed case, but without a real dictionary-join query showing a material gain, the added hash-join specialization is not justified. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
