Steven Zhen Wu created FLINK-19934:
--------------------------------------
Summary: [FLIP-27 source] add new API:
SplitEnumeratorContext.execute(Runnable)
Key: FLINK-19934
URL: https://issues.apache.org/jira/browse/FLINK-19934
Project: Flink
Issue Type: New Feature
Components: API / DataStream
Affects Versions: 1.11.2
Reporter: Steven Zhen Wu
Here is the motivation use case. We are implementing event-time alignment
across sources in Iceberg source. Basically, each Iceberg source/enumerator
tracks its watermark using min/max timestamps captures in the column stats of
the data files.
When the watermark from another source advances, notified source/enumerator can
try `assignSplits` as constraints may be satisfied now. This callback is
initiated from the coordinator thread from the other source. If we have
`SplitEnumeratorContext.execute(Runnable r)` API, we can ensure that all the
actions by enumerator and assigner are serialized by the coordinator thread.
That can avoid the need of locks.
[~becket_qin] [~sewen] what do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
