Re: Checkpointing barriers

2018-04-24 Thread Fabian Hueske
Hi Alex, That's correct. The n refers to the n-th checkpoint. The checkpoint ID is important, because operators need to align the barriers to ensure that they consumed all inputs up to the point, where the barriers were injected into the stream. Each operator checkpoints its own state. For

Re: Checkpointing barriers

2018-04-24 Thread Alexander Smirnov
ok, I got it. Barrier-n is an indicator or n-th checkpoint. My first impression was that barriers are carrying offset information, but it was wrong. Thanks for unblocking ;-) Alex

Re: Checkpointing barriers

2018-04-23 Thread Ted Yu
barrier n appearing in all the streams serves as synchronization point. As explained in the subsequent paragraph: bq. Otherwise, it would mix records that belong to snapshot *n*and with records that belong to snapshot *n+1*. Cheers On Mon, Apr 23, 2018 at 7:21 AM, Alexander Smirnov <

Checkpointing barriers

2018-04-23 Thread Alexander Smirnov
Hi, I'm reading documentation about checkpointing: https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html It describes a case, when an operator receives data from all its incoming streams alongs with barriers. There's also an illustration on that page for the