adriangb opened a new pull request, #27324: URL: https://github.com/apache/datafusion/pull/27324
## Which issue does this PR close? - Follow-up to https://github.com/apache/datafusion/pull/27091, from review feedback there. > [!NOTE] > This PR is stacked on https://github.com/apache/datafusion/pull/27091. Only the last commit, `perf: skip probe hashing in the routed lookup when every map is an array map`, is new here. ## Rationale for this change A `Partitioned` hash join with dense integer keys builds an `ArrayMap` in every partition. Its dynamic filter then computes a hash for every probe row, but an `ArrayMap` looks keys up by value, so nothing reads those hashes. Skipping them makes the filter cheaper on every probe batch. ## What changes are included in this PR? `HashTableLookupExpr::evaluate_routed` hashes the probe keys only when at least one partition is a hash table. The counting sort by partition runs either way. ## What is the testing strategy for this PR? - A new unit test builds every partition as an `ArrayMap` and checks exact matches, including a null probe row. - The routed lookup fuzz test passes with and without `force_hash_collisions`. - Lookup cost on a GCP VM (16 partitions, batch 8192): 7.49 → 6.56 ns/row. ## Are there any user-facing changes? No. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
