+1 for the proposal, thanks for driving this! Best, Ferenc
On Sunday, 5 May 2024 at 01:58, Steven Wu <stevenz...@gmail.com> wrote: > +1 for the proposal of adding more table maintenance to Flink. > > It is great that the maintenance actions can be run in two modes: (1) > embedded in the Flink writer job as post commit stage (2) standalone Flink > batch job/action. I probably wouldn't label the two goals as primary and > secondary. Different users may have different preferences. Or different > maintenance tasks may favor one mode over another. > > On Fri, May 3, 2024 at 11:30 AM Péter Váry <peter.vary.apa...@gmail.com> > wrote: > >> Hi everyone, >> >> I would like to make a proposal [1] to support Flink Table Maintenance in >> Iceberg. The main goal is to have a solution where Flink can execute the >> Maintenance Tasks as part of the streaming job. Especially Rewrite Data >> Files, Rewrite Manifest Files and Expire Snapshots. >> The secondary goal is to provide building blocks for Flink batch jobs to >> execute the Maintenance Tasks independently, where the scheduling is done >> outside of Flink. >> >> This proposal is the outcome of extensive community discussions on the >> mailing list [2, 3]. >> >> Please respond with your recommendation: >> +1 if you support moving forward with the two separate objects model. >> 0 if you are neutral. >> -1 if you disagree with the two separate objects model. >> >> Thanks, >> Peter >> >> [1] https://github.com/apache/iceberg/issues/10264 >> [2] https://lists.apache.org/thread/yjcwbf1037jdq4prty6rtrrqmjzc71o0 >> [3] https://lists.apache.org/thread/10mdf9zo6pn0dfq791nf4w1m7jh9k3sl
