AW: State Schema Evolution within SQL API

2021-03-04 Thread Jan Oelschlegel
: State Schema Evolution within SQL API Hi Qinghui Maybe this strategy with evolving an application without the need for state restore could help you [1] [1] https://docs.ververica.com/v2.3/user_guide/sql_development/sql_scripts.html#sql-script-changes Best, Jan Von: XU Qinghui mailto:qinghui

AW: State Schema Evolution within SQL API

2021-03-04 Thread Jan Oelschlegel
Wysakowicz Cc: Jan Oelschlegel ; user Betreff: Re: State Schema Evolution within SQL API Hello Dawid I'm interested in this discussion because I'm currently trying to upgrade flink from 1.9 to 1.12 for a bunch of sql jobs running in production. From what you said, this seems to

Re: State Schema Evolution within SQL API

2021-03-04 Thread XU Qinghui
Hello Dawid I'm interested in this discussion because I'm currently trying to upgrade flink from 1.9 to 1.12 for a bunch of sql jobs running in production. From what you said, this seems to be a risky operation which might corrupt the states. Is there any recommendation to mitigate the risk? It wo

Re: State Schema Evolution within SQL API

2021-03-04 Thread Dawid Wysakowicz
Hi Jan, As of now Flink does not give any guarantees for Table/SQL API savepoint compatibility if you change the query or Flink version. Flink Table/SQL API uses an optimizer that can apply different optimizations or operations reordering based on the queried fields or computations that can result

State Schema Evolution within SQL API

2021-03-01 Thread Jan Oelschlegel
Hi at all, i would like to know how far a state schema evolution is possible by using SQL API of Flink. Which query changes can I do without disrupting the schema of my savepoint? In the documentation is, only for the DataStream API , written what are the do's and don'ts regarding a safe sch