waterWang opened a new pull request, #19191:
URL: https://github.com/apache/pinot/pull/19191
## Description
When a lookup join condition includes a literal value for a dimension table
primary key column (e.g., `dim_tbl.currency = 'gbp'`), the `LookupJoinOperator`
previously only used the equi-join key columns from `leftKeys` to build the
`PrimaryKey` for the dimension table lookup. The literal condition was treated
as a non-equi condition, causing the PrimaryKey to have fewer values than the
dimension table's primary key columns. This resulted in the lookup returning 0
rows.
## Root Cause
The `getKey()` method in `LookupJoinOperator` built the `PrimaryKey` using
only `_leftKeyIds` (the equi-join column indices from the left input). When a
dimension table primary key component was matched by a literal (e.g., `currency
= 'gbp'`), the `JoinInfo.analyzeCondition()` did not include this as an
equi-join key. The literal was instead placed in `nonEquiConditions`, which are
evaluated after the lookup. Since the PrimaryKey was incomplete (missing the
literal component), the dimension table lookup failed to find the row.
## Fix
- Pre-compute a mapping from the dimension table's primary key columns to
their value sources
- For each primary key column, determine whether the value comes from:
1. An equi-join key (left row column) — mapped via `leftKeys`/`rightKeys`
pairs
2. A literal constant — extracted from the non-equi conditions (e.g.,
`dim_tbl.currency = 'gbp'`)
- Build the `PrimaryKey` in the dimension table's primary key column order,
filling in literal values where applicable
- Apply the same fix to `fillKey()` used by SEMI and ANTI joins
## Testing
- Added `lookup_join_literal_key` test case to `LookupJoin.json`
- The test uses a dimension table with a 2-column primary key `[currency,
rate_start_date]`
- The join condition provides `currency = 'gbp'` as a literal and
`rate_start_date` as an equi-join key
- Expected output: `[gbp, 125]` (previously returned 0 rows)
Fixes #19188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
