No worries, thanks for the reply Gyula. Ah yes, I see how those points you raised make the feature tricky to implement. Could this be considered for a FLIP (or two) in the future?
On Wed, Mar 20, 2024 at 2:21 PM Gyula Fóra <gyula.f...@gmail.com> wrote: > Sorry for the late reply Kevin. > > I think what you are suggesting makes sense, it would be basically a > `last-state` startup mode. This would also help in cases where the current > last-state mechanism fails to locate HA metadata (and the state). > > This is somewhat of a tricky feature to implement: > 1. The operator will need FS plugins and access to the different user envs > (this will not work in many prod environments unfortunately) > 2. Flink doesn't expose a good way to detect the latest checkpoint just by > looking at the FS so we need to figure out something here. Probably some > changes are necessary on Flink core side as well > > Gyula >
