[jira] [Updated] (FLINK-19934) [FLIP-27 source] add new API: SplitEnumeratorContext.runInCoordinatorThread(Runnable)
[ https://issues.apache.org/jira/browse/FLINK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xintong Song updated FLINK-19934: - Fix Version/s: 1.11.3 > [FLIP-27 source] add new API: > SplitEnumeratorContext.runInCoordinatorThread(Runnable) > - > > Key: FLINK-19934 > URL: https://issues.apache.org/jira/browse/FLINK-19934 > Project: Flink > Issue Type: New Feature > Components: API / DataStream >Affects Versions: 1.11.2 >Reporter: Steven Zhen Wu >Assignee: Steven Zhen Wu >Priority: Major > Labels: pull-request-available > Fix For: 1.12.0, 1.11.3 > > > Here is the motivation use case. We are implementing event-time alignment > across sources in Iceberg source. Basically, each Iceberg source/enumerator > tracks its watermark using min/max timestamps captures in the column stats of > the data files. > When the watermark from another source advances, notified source/enumerator > can try `assignSplits` as constraints may be satisfied now. This callback is > initiated from the coordinator thread from the other source. If we have > `SplitEnumeratorContext.execute(Runnable r)` API, we can ensure that all the > actions by enumerator and assigner are serialized by the coordinator thread. > That can avoid the need of locks. > [~becket_qin] [~sewen] what do you think? cc [~sundaram] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (FLINK-19934) [FLIP-27 source] add new API: SplitEnumeratorContext.runInCoordinatorThread(Runnable)
[ https://issues.apache.org/jira/browse/FLINK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Zhen Wu updated FLINK-19934: --- Summary: [FLIP-27 source] add new API: SplitEnumeratorContext.runInCoordinatorThread(Runnable) (was: [FLIP-27 source] add new API: SplitEnumeratorContext.execute(Runnable)) > [FLIP-27 source] add new API: > SplitEnumeratorContext.runInCoordinatorThread(Runnable) > - > > Key: FLINK-19934 > URL: https://issues.apache.org/jira/browse/FLINK-19934 > Project: Flink > Issue Type: New Feature > Components: API / DataStream >Affects Versions: 1.11.2 >Reporter: Steven Zhen Wu >Priority: Major > Labels: pull-request-available > > Here is the motivation use case. We are implementing event-time alignment > across sources in Iceberg source. Basically, each Iceberg source/enumerator > tracks its watermark using min/max timestamps captures in the column stats of > the data files. > When the watermark from another source advances, notified source/enumerator > can try `assignSplits` as constraints may be satisfied now. This callback is > initiated from the coordinator thread from the other source. If we have > `SplitEnumeratorContext.execute(Runnable r)` API, we can ensure that all the > actions by enumerator and assigner are serialized by the coordinator thread. > That can avoid the need of locks. > [~becket_qin] [~sewen] what do you think? cc [~sundaram] -- This message was sent by Atlassian Jira (v8.3.4#803005)
