Prabhu Joseph created FLINK-33753:
-------------------------------------
Summary: ContinuousFileReaderOperator consume records as mini batch
Key: FLINK-33753
URL: https://issues.apache.org/jira/browse/FLINK-33753
Project: Flink
Issue Type: Improvement
Affects Versions: 1.18.0
Reporter: Prabhu Joseph
The ContinuousFileReaderOperator reads and collects the records from a split in
a loop. If the split size is large, then the loop will take more time, and then
the mailbox executor won't have a chance to process the checkpoint barrier.
This leads to checkpoint timing out. ContinuousFileReaderOperator could be
improved to consume the records in a mini batch, similar to Hudi's
StreamReadOperator (https://issues.apache.org/jira/browse/HUDI-2485).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
