Hi Timo,
Thanks for driving this discussion, the sql job's upgrading compatibility always
is a big pain point. In the last version we completed some work, this FLIP will
make the whole upgrade story possible.
I have a few comments:
1) "EXPLAIN PLAN EXECUTE STATEMENT SET BEGIN ... END" is missing.
It's better we can add this syntax and make the API more complete.
2) about the annotation of the ExecNode, it's hard to maintain the supported
versions for "supportedPlanChanges" and "supportedSavepointChanges".
Imagine that, when we are upgrading Flink from 1.15 to 1.16, most ExecNodes are
not changed (high probability scenarios), but we need add supported
version (1.16)
to most (even all) ExecNodes manually. Considering that the supported
versions are
continuous, we only need annotate the start version (when the ExecNode
is introduced)
and the end version (when the change is compatible and a new ExecNode
with new version
needs to be introduced) for supportedPlanChanges and supportedSavepointChanges.
e.g. supportedSavepointChanges ={start=1_15, end=1_16}
Best,
Godfrey
wenlong.lwl <wenlong88....@gmail.com> 于2021年11月22日周一 下午9:07写道：
>
> Hi, Timo, thanks for driving the discussion and the preparation on the
> FLIP. This is a pain point of Flink SQL complaining by our users badly. I
> have seen many cases where our users suffer while trying to upgrade the
> flink version in order to take advantage of the bug fixes and performance
> improvements on the new version. It often takes a long time verifying the
> new plan, reoptimizing the config, recomputing the state, waiting for a
> safe point to make the new job active in production, etc. There are many
> times that new problems show up in upgrading.
>
> I have a question on COMPILE AND EXECUTE. It doesn't look so good that we
> just execute the plan and ignore the statement when the plan already
> exists, but the plan and SQL are not matched. The result would be quite
> confusing if we still execute the plan directly, we may need to add a
> validation. Personally I would prefer not to provide such a shortcut, let
> users use COMPILE PLAN IF NOT EXISTS and EXECUTE explicitly, which can be
> understood by new users even without inferring the docs.
>
> Best,
> Wenlong
