[ https://issues.apache.org/jira/browse/FLINK-26098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518956#comment-17518956 ]
Marios Trivyzas edited comment on FLINK-26098 at 4/7/22 3:35 PM: ----------------------------------------------------------------- For {{DataStream}} API, with [FLIP-27|[https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface]] , there is {{WatermarkStrategy}} interface and {{WatermarkStrategyWithIdleness}} to configure the idleness, which is only in the context of {{DataStream}} API. For TableApi/SQL we have {{PushWatermarkIntoTableSourceScanRuleBase}} which uses the {{table.exec.source.idle-timeout and in turn}} create a {{WatermarkStrategyWithIdleness}} to be used. So I think that this issue can now be closed, since it has been addressed by the work in [[FLINK-16835]|https://issues.apache.org/jira/browse/FLINK-16835] and is already available for {{{}Flink 1.15{}}}. was (Author: matriv): For {{DataStream}} API, with [FLIP-27|[https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface]] , there is {{WatermarkStrategy}} interface and {{WatermarkStrategyWithIdleness}} to configure the idleness, which is only in the context of {{DataStream}} API. For TableApi/SQL we have {{PushWatermarkIntoTableSourceScanRuleBase}} which uses the {{table.exec.source.idle-timeout and in turn}} create a {{WatermarkStrategyWithIdleness}} to be used. So I think that this issue can now be closed, since it has been addressed by the work in https://issues.apache.org/jira/browse/FLINK-16835. > TableAPI does not forward idleness configuration from DataStream > ---------------------------------------------------------------- > > Key: FLINK-26098 > URL: https://issues.apache.org/jira/browse/FLINK-26098 > Project: Flink > Issue Type: Bug > Components: Table SQL / API > Affects Versions: 1.15.0, 1.14.3 > Reporter: Till Rohrmann > Assignee: Marios Trivyzas > Priority: Major > Attachments: Screenshot_20220407_150020.png, > Screenshot_20220407_151012.png > > > The TableAPI does not forward the idleness configuration from a DataStream > source. That can lead to the halt of processing if all sources are idle > because {{WatermarkAssignerOperator}} [1] will never set a channel to active > again. The only way to mitigate the problem is to explicitly configure the > idleness for table sources via {{table.exec.source.idle-timeout}}. > Configuring this value is actually not easy because creating a > {{StreamExecutionEnvironment}} via {{create(StreamExecutionEnvironment, > TableConfig)}} is deprecated. > [1] > https://github.com/apache/flink/blob/master/flink-table/flink-table-runtime/src/main/java/org/apache/flink/table/runtime/operators/wmassigners/WatermarkAssignerOperator.java#L103 -- This message was sent by Atlassian Jira (v8.20.1#820001)