退订

邢颖
[email protected]



        



         原始邮件
         
       
发件人:Weiwei Su <[email protected]&gt;
发件时间:2026年11月20日 16:03
收件人:dev <[email protected]&gt;
主题:[BUG] Wrong result on inner self-join with empty right input: 
HashAggregate(select=[]) + singleRowJoin preserves left groups



       Hi&nbsp;Flink&nbsp;dev&nbsp;community,

I&nbsp;encountered&nbsp;a&nbsp;silent&nbsp;wrong-result&nbsp;bug&nbsp;in&nbsp;Flink&nbsp;SQL&nbsp;batch&nbsp;mode&nbsp;(reproduced
3/3&nbsp;times&nbsp;on&nbsp;Flink&nbsp;1.19.1&nbsp;with&nbsp;an&nbsp;Iceberg&nbsp;table&nbsp;source)&nbsp;where&nbsp;an&nbsp;`INNER
JOIN&nbsp;...&nbsp;ON&nbsp;TRUE`&nbsp;self-join&nbsp;with&nbsp;a&nbsp;filter&nbsp;that&nbsp;makes&nbsp;the&nbsp;right&nbsp;input&nbsp;empty
(`WHERE&nbsp;t3.region&nbsp;IS&nbsp;NULL`)&nbsp;incorrectly&nbsp;returns&nbsp;non-empty&nbsp;left-side&nbsp;groups
instead&nbsp;of&nbsp;0&nbsp;rows.

###&nbsp;Minimal&nbsp;reproducer

Given&nbsp;a&nbsp;4-row&nbsp;table&nbsp;(`(1,&nbsp;11,&nbsp;'west')`,&nbsp;`(2,&nbsp;12,&nbsp;'west')`,&nbsp;`(5,&nbsp;15,
'east')`,&nbsp;`(6,&nbsp;16,&nbsp;'east')`,&nbsp;with&nbsp;no&nbsp;`NULL`&nbsp;values&nbsp;in&nbsp;`region`):

```sql
SELECT&nbsp;t2.region
FROM&nbsp;ld_pd_target_20261006a&nbsp;/*+
OPTIONS('snapshot-id'='2218517376986569101')&nbsp;*/&nbsp;AS&nbsp;t2
JOIN&nbsp;ld_pd_target_20261006a&nbsp;/*+
OPTIONS('snapshot-id'='2218517376986569101')&nbsp;*/&nbsp;AS&nbsp;t3
&nbsp;&nbsp;ON&nbsp;TRUE
WHERE&nbsp;t3.region&nbsp;IS&nbsp;NULL
GROUP&nbsp;BY&nbsp;t2.region;
```

###&nbsp;Observed&nbsp;vs.&nbsp;expected&nbsp;result

-&nbsp;**Expected**:&nbsp;0&nbsp;rows&nbsp;(since&nbsp;`t3.region&nbsp;IS&nbsp;NULL`&nbsp;matches&nbsp;0&nbsp;rows,&nbsp;an&nbsp;inner
join&nbsp;with&nbsp;an&nbsp;empty&nbsp;right&nbsp;side&nbsp;must&nbsp;produce&nbsp;0&nbsp;rows).&nbsp;DuckDB,&nbsp;Trino,&nbsp;and
Spark&nbsp;all&nbsp;return&nbsp;0&nbsp;rows&nbsp;on&nbsp;the&nbsp;same&nbsp;snapshot.
-&nbsp;**Actual&nbsp;(Flink&nbsp;1.19.1,&nbsp;3/3&nbsp;runs)**:&nbsp;returns&nbsp;2&nbsp;rows:&nbsp;`west`&nbsp;and&nbsp;`east`.
-&nbsp;**Related&nbsp;variants&nbsp;on&nbsp;Flink**:
&nbsp;&nbsp;-&nbsp;`SELECT&nbsp;DISTINCT&nbsp;t2.region&nbsp;...`&nbsp;also&nbsp;returns&nbsp;`west`,&nbsp;`east`.
&nbsp;&nbsp;-&nbsp;`SELECT&nbsp;t2.region,&nbsp;COUNT(*)&nbsp;...&nbsp;GROUP&nbsp;BY&nbsp;t2.region`&nbsp;returns&nbsp;`('west',
0)`&nbsp;and&nbsp;`('east',&nbsp;0)`.
-&nbsp;**Control**:&nbsp;The&nbsp;equivalent&nbsp;pure&nbsp;`VALUES`&nbsp;query&nbsp;(`WITH&nbsp;src(order_id,&nbsp;v,
region)&nbsp;AS&nbsp;(VALUES&nbsp;...)`)&nbsp;returns&nbsp;0&nbsp;rows&nbsp;in&nbsp;Flink,&nbsp;likely&nbsp;because&nbsp;the&nbsp;empty
filter&nbsp;is&nbsp;constant-folded&nbsp;at&nbsp;compile&nbsp;time&nbsp;rather&nbsp;than&nbsp;reaching&nbsp;the&nbsp;physical
join&nbsp;plan&nbsp;below.

###&nbsp;Root&nbsp;cause&nbsp;clue&nbsp;(`EXPLAIN&nbsp;PLAN&nbsp;FOR`)

In&nbsp;the&nbsp;optimized&nbsp;physical&nbsp;plan,&nbsp;Flink&nbsp;pushes&nbsp;a&nbsp;global
`HashAggregate(select=[])`&nbsp;(with&nbsp;no&nbsp;grouping&nbsp;keys)&nbsp;onto&nbsp;the&nbsp;filtered&nbsp;right
input&nbsp;and&nbsp;joins&nbsp;it&nbsp;with&nbsp;`NestedLoopJoin(joinType=[InnerJoin],&nbsp;...,
singleRowJoin=[true])`.&nbsp;Because&nbsp;a&nbsp;global&nbsp;aggregate&nbsp;without&nbsp;grouping&nbsp;keys
produces&nbsp;a&nbsp;single&nbsp;row&nbsp;even&nbsp;on&nbsp;an&nbsp;empty&nbsp;input,&nbsp;the&nbsp;inner&nbsp;join&nbsp;preserves&nbsp;all
left-side&nbsp;groups&nbsp;instead&nbsp;of&nbsp;producing&nbsp;an&nbsp;empty&nbsp;result:

```text
==&nbsp;Optimized&nbsp;Physical&nbsp;Plan&nbsp;==
NestedLoopJoin(joinType=[InnerJoin],&nbsp;where=[true],&nbsp;select=[region],
build=[right],&nbsp;singleRowJoin=[true])
:-&nbsp;HashAggregate(isMerge=[true],&nbsp;groupBy=[region],&nbsp;select=[region])
:&nbsp;&nbsp;+-&nbsp;Exchange(distribution=[hash[region]])
:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;LocalHashAggregate(groupBy=[region],&nbsp;select=[region])
:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;TableSourceScan(table=[[iceberg,&nbsp;default,
ld_pd_target_20261006a,&nbsp;project=[region]]],&nbsp;fields=[region],
hints=[[[OPTIONS&nbsp;options:{snapshot-id=2218517376986569101}]]])
+-&nbsp;Exchange(distribution=[broadcast])
&nbsp;&nbsp;&nbsp;+-&nbsp;HashAggregate(isMerge=[true],&nbsp;select=[])
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;Exchange(distribution=[single])
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;LocalHashAggregate(select=[])
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;Calc(select=[order_id],&nbsp;where=[IS&nbsp;NULL(region)])
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;+-&nbsp;TableSourceScan(table=[[iceberg,&nbsp;default,
ld_pd_target_20261006a,&nbsp;filter=[IS&nbsp;NULL(region)],&nbsp;project=[order_id,
region]]],&nbsp;fields=[order_id,&nbsp;region],&nbsp;hints=[[[OPTIONS
options:{snapshot-id=2218517376986569101}]]])
```

(My&nbsp;ASF&nbsp;JIRA&nbsp;account&nbsp;request&nbsp;has&nbsp;been&nbsp;pending&nbsp;for&nbsp;nearly&nbsp;a&nbsp;month,&nbsp;so&nbsp;I&nbsp;am
reporting&nbsp;this&nbsp;here&nbsp;first.&nbsp;If&nbsp;this&nbsp;is&nbsp;a&nbsp;confirmed&nbsp;bug,&nbsp;could&nbsp;a&nbsp;committer
help&nbsp;open&nbsp;a&nbsp;FLINK&nbsp;JIRA&nbsp;ticket?&nbsp;Happy&nbsp;to&nbsp;provide&nbsp;full&nbsp;cross-engine&nbsp;logs&nbsp;or
test&nbsp;on&nbsp;a&nbsp;newer&nbsp;Flink&nbsp;version&nbsp;if&nbsp;helpful.)

Best&nbsp;regards,
Weiwei&nbsp;Su

Reply via email to