Hi, CMCF is not a source, only the file monitoring function is. Barriers are injected by the FMF when the JM sends a checkpoint message. The barriers then travel to the CMCF and trigger the Checkpoint ING.
Fabian Averell <lvhu...@gmail.com> schrieb am Di., 28. Aug. 2018, 12:02: > Hello Fabian, > > Thanks for the answer. However, my question is a little bit different. > Let me rephrase my example and my question: > * I have 10,000 unsplittable small files to read, which, in total, has > about 10M output lines. > * From Flink's reporting web GUI, I can see that CFMF and > ContinuousFileReaderOperator (CFRO) are reported separately. > - CFMF needs about 10 seconds to generate all 10,000 records > (as > you said, in this case, 1 record = 1 file split). > - CFRO generates about 2M records per minute (which means CFRO > is processing at the rate of 2,000 files per minute) > * I set the checkpointing interval = 1 minute. > In this example, /will the 1st barrier be injected into the stream of > file-splits 50 seconds after the 10,000th split, or after the 2,000th one?/ > > Sorry for being confusing. > > Thanks and best regards, > Averell > > > > > -- > Sent from: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/ >
