退订
邢颖
[email protected]
原始邮件
发件人:Weiwei Su <[email protected]>
发件时间:2026年11月20日 16:03
收件人:dev <[email protected]>
主题:[BUG] Wrong result on inner self-join with empty right input:
HashAggregate(select=[]) + singleRowJoin preserves left groups
Hi Flink dev community,
I encountered a silent wrong-result bug in Flink SQL batch mode (reproduced
3/3 times on Flink 1.19.1 with an Iceberg table source) where an `INNER
JOIN ... ON TRUE` self-join with a filter that makes the right input empty
(`WHERE t3.region IS NULL`) incorrectly returns non-empty left-side groups
instead of 0 rows.
### Minimal reproducer
Given a 4-row table (`(1, 11, 'west')`, `(2, 12, 'west')`, `(5, 15,
'east')`, `(6, 16, 'east')`, with no `NULL` values in `region`):
```sql
SELECT t2.region
FROM ld_pd_target_20261006a /*+
OPTIONS('snapshot-id'='2218517376986569101') */ AS t2
JOIN ld_pd_target_20261006a /*+
OPTIONS('snapshot-id'='2218517376986569101') */ AS t3
ON TRUE
WHERE t3.region IS NULL
GROUP BY t2.region;
```
### Observed vs. expected result
- **Expected**: 0 rows (since `t3.region IS NULL` matches 0 rows, an inner
join with an empty right side must produce 0 rows). DuckDB, Trino, and
Spark all return 0 rows on the same snapshot.
- **Actual (Flink 1.19.1, 3/3 runs)**: returns 2 rows: `west` and `east`.
- **Related variants on Flink**:
- `SELECT DISTINCT t2.region ...` also returns `west`, `east`.
- `SELECT t2.region, COUNT(*) ... GROUP BY t2.region` returns `('west',
0)` and `('east', 0)`.
- **Control**: The equivalent pure `VALUES` query (`WITH src(order_id, v,
region) AS (VALUES ...)`) returns 0 rows in Flink, likely because the empty
filter is constant-folded at compile time rather than reaching the physical
join plan below.
### Root cause clue (`EXPLAIN PLAN FOR`)
In the optimized physical plan, Flink pushes a global
`HashAggregate(select=[])` (with no grouping keys) onto the filtered right
input and joins it with `NestedLoopJoin(joinType=[InnerJoin], ...,
singleRowJoin=[true])`. Because a global aggregate without grouping keys
produces a single row even on an empty input, the inner join preserves all
left-side groups instead of producing an empty result:
```text
== Optimized Physical Plan ==
NestedLoopJoin(joinType=[InnerJoin], where=[true], select=[region],
build=[right], singleRowJoin=[true])
:- HashAggregate(isMerge=[true], groupBy=[region], select=[region])
: +- Exchange(distribution=[hash[region]])
: +- LocalHashAggregate(groupBy=[region], select=[region])
: +- TableSourceScan(table=[[iceberg, default,
ld_pd_target_20261006a, project=[region]]], fields=[region],
hints=[[[OPTIONS options:{snapshot-id=2218517376986569101}]]])
+- Exchange(distribution=[broadcast])
+- HashAggregate(isMerge=[true], select=[])
+- Exchange(distribution=[single])
+- LocalHashAggregate(select=[])
+- Calc(select=[order_id], where=[IS NULL(region)])
+- TableSourceScan(table=[[iceberg, default,
ld_pd_target_20261006a, filter=[IS NULL(region)], project=[order_id,
region]]], fields=[order_id, region], hints=[[[OPTIONS
options:{snapshot-id=2218517376986569101}]]])
```
(My ASF JIRA account request has been pending for nearly a month, so I am
reporting this here first. If this is a confirmed bug, could a committer
help open a FLINK JIRA ticket? Happy to provide full cross-engine logs or
test on a newer Flink version if helpful.)
Best regards,
Weiwei Su