[
https://issues.apache.org/jira/browse/FLINK-37971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maximilian Michels updated FLINK-37971:
---------------------------------------
Affects Version/s: 1.20.3
1.20.2
1.20.0
(was: 2.0.0)
(was: 1.19.2)
> SQL hints are dropped when combining multiple SELECT into a single source
> -------------------------------------------------------------------------
>
> Key: FLINK-37971
> URL: https://issues.apache.org/jira/browse/FLINK-37971
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner, Table SQL / Runtime
> Affects Versions: 1.20.0, 1.20.1, 1.20.2, 1.20.3
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Attachments: image-2025-07-07-13-53-24-292.png
>
>
> The IcebergSource supports using SQL hints to set some runtime properties
> like whether to run in batch or streaming mode. For example, this creates an
> Iceberg source and an Iceberg sink:
> {noformat}
> INSERT into output
> SELECT * FROM sample /*+ OPTIONS('streaming'='true',
> 'monitor-interval'='1s')*/ ;
> {noformat}
> In the below query, the resulting DAG would contain only one IcebergSource
> and two Iceberg sinks:
> {noformat}
> INSERT into output1
> SELECT * FROM sample /*+ OPTIONS('streaming'='true',
> 'monitor-interval'='1s')*/ WHERE ... ;
> INSERT INTO output2
> SELECT * FROM sample /*+ OPTIONS('streaming'='true',
> 'monitor-interval'='1s')*/ WHERE ... ;
> {noformat}
> However, the SQL hints will not be available when creating the source, i.e.
> the {{FlinkDynamicTableFactory#createDynamicTableSource(Context context)}}
> will not receive the options derived from the SQL hint via
> {{context.getCatalogTable().getOptions()}}, as is the case before the
> optimizer combines the two SELECT statements.
> It starts to work again when {{table.optimizer.reuse-source-enabled: false}}
> is set.
> The issue manifests here, where the new {{ScanTableSource}} does not have the
> properties supplied in the SQL hints:
> https://github.com/apache/flink/blob/ef5b03297092c5eda7c8862f1d7eb08aa10e480b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/ScanReuser.java#L212
> For the issue to surface, these two if blocks need to be circumvented:
> https://github.com/apache/flink/blob/ef5b03297092c5eda7c8862f1d7eb08aa10e480b/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/reuse/ScanReuser.java#L143
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
