Hi Alexey, Overall, the FLIP looks good and makes sense to me.
1. For the testing, I'd call the tests "execution" tests rather than "restore" tests. For streaming execution, restore tests have the compiled plan and intermediate state; the tests verify that those can work together and continue processing. For batch execution, I think we just want that all existing compiled plans can be executed in future versions. 2. The FLIP implicitly suggests "completeness tests" (to use FLIP-190's words). Do we need "change detection tests"? I'm a little unsure if that is presently happening in an automatic way for streaming operators. In RestoreTestBase, generateTestSetupFiles is disabled and has to be run manually when tests are being written. Cheers, Jim On Tue, May 7, 2024 at 5:11 AM Paul Lam <paullin3...@gmail.com> wrote: > Hi Alexey, > > Thanks a lot for bringing up the discussion. I’m big +1 for the FLIP. > > I suppose the goal doesn’t involve the interchangeability of json plans > between batch mode and streaming mode, right? > In other words, a json plan compiled in a batch program can’t be run in > streaming mode without a migration (which is not yet supported). > > Best, > Paul Lam > > > 2024年5月7日 14:38，Alexey Leonov-Vendrovskiy <vendrov...@gmail.com> 写道： > > > > Hi everyone, > > > > PTAL at the proposed FLIP-456: CompiledPlan support for Batch Execution > > Mode. It is pretty self-describing. > > > > Any thoughts are welcome! > > > > Thanks, > > Alexey > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-456%3A+CompiledPlan+support+for+Batch+Execution+Mode > > . > >
