korowa commented on code in PR #2591:
URL: https://github.com/apache/arrow-datafusion/pull/2591#discussion_r881870265
##########
datafusion/core/src/physical_plan/hash_join.rs:
##########
@@ -791,6 +826,109 @@ fn build_join_indexes(
}
}
+fn apply_join_filter(
+ left: &RecordBatch,
+ right: &RecordBatch,
+ join_type: JoinType,
+ left_indices: UInt64Array,
+ right_indices: UInt32Array,
+ filter: &JoinFilter,
+) -> Result<(UInt64Array, UInt32Array)> {
+ if left_indices.is_empty() {
+ return Ok((left_indices, right_indices));
+ };
+
+ let (intermediate_batch, _) = build_batch_from_indices(
+ filter.schema(),
+ left,
+ right,
+ PrimitiveArray::from(left_indices.data().clone()),
+ PrimitiveArray::from(right_indices.data().clone()),
+ filter.column_indices(),
+ )?;
+
+ match join_type {
+ JoinType::Inner | JoinType::Left => {
+ // For both INNER and LEFT joins, input arrays contains only
indices for matched data.
Review Comment:
That was my initial plan, but it turned out there are [no nulls
appended](https://github.com/apache/arrow-datafusion/blob/8e71c887621254b3d18523ad182045a9e987ee12/datafusion/core/src/physical_plan/hash_join.rs#L728)
in case of left join while building matched indices. All null values are
handled
[later](https://github.com/apache/arrow-datafusion/blob/8e71c887621254b3d18523ad182045a9e987ee12/datafusion/core/src/physical_plan/hash_join.rs#L1024),
using visited left side indices
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
