By default, Flink uses aligned checkpoint where we wait for all in-flight data before the barriers to be fully processed and then make the checkpoints. There's no in need to store in-flight buffers in this case at the cost of additional barrier alignment, which may take a long time at the presence of backpressure. Unaligned checkpoint is introduced to solve this problem, where in-flight buffers are stored in cp without the need of alignment.
Best, Zhanghao Chen ________________________________ From: Enric Ott <[email protected]> Sent: Wednesday, July 17, 2024 16:35 To: [email protected] <[email protected]> Subject: flight buffer local storage Hello,Community: Why doesn't flink store in flight buffers to local disks when it checkpoints? Thanks.
