jayzhan211 commented on code in PR #26793:
URL: https://github.com/apache/datafusion/pull/26793#discussion_r4504723420
##########
datafusion/physical-plan/src/joins/hash_join/exec.rs:
##########
@@ -3256,16 +3317,7 @@ fn build_in_memory(
)?;
Review Comment:
Optional cleanup: `update_hash` evaluates `on_left` again on every slice,
but `left_values` already holds those keys for the whole batch. Hashing slices
of them skips that work for expression keys such as `a + 1`. With this change
all 687 hash join tests pass and build time is the same. The import changes
from `update_hash` to `update_hash_from_values`.
```suggestion
for offset in (0..num_rows).step_by(chunk_rows).rev() {
let len = chunk_rows.min(num_rows - offset);
let chunk: Vec<ArrayRef> = left_values
.iter()
.map(|values| values.slice(offset, len))
.collect();
hashes_buffer.clear();
hashes_buffer.resize(len, 0);
update_hash_from_values(
&chunk,
&mut *hashmap,
offset,
random_state,
&mut hashes_buffer,
0,
true,
null_equality,
)?;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
