Hello Fabian,
Thanks for the answer. However, my question is a little bit different.
Let me rephrase my example and my question:
* I have 10,000 unsplittable small files to read, which, in total, has
about 10M output lines.
* From Flink's reporting web GUI, I can see that CFMF and
ContinuousFileReaderOperator (CFRO) are reported separately.
- CFMF needs about 10 seconds to generate all 10,000 records (as
you said, in this case, 1 record = 1 file split).
- CFRO generates about 2M records per minute (which means CFRO
is processing at the rate of 2,000 files per minute)
* I set the checkpointing interval = 1 minute.
In this example, /will the 1st barrier be injected into the stream of
file-splits 50 seconds after the 10,000th split, or after the 2,000th one?/
Sorry for being confusing.
Thanks and best regards,
Averell
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
