Hi all,

Currently, we work a lot in the maintenance of compatibility.  
There exist much code in runtime to support the migration of savepoints (most 
of which are deprecated), making it hard to focus on the current implementation.
When more versions are released, much more efforts will be needed if we try to 
make these released versions compatible.

I agree with Tzu-Li that we should provide a method to let users upgrade Flink 
in a reasonable pace.
But i am against the proposal that we only offer backwards compatibility for 
one previous version.
According our time-based release model, a major version is released every four 
month. 
That means, users have to upgrade their versions every 8 months. Otherwise they 
will have difficulties in the migration of existing savepoints.

My suggestions include

(1) We can release Long-Term Support (LTS) versions which are widely adopted in 
other open-source projects. 
LTS versions should be stable and are free of found bugs. Savepoints in LTS 
versions are guaranteed to be back-compatible so that users can easily upgrade 
to newer LTS versions.

The releasing of LTS versions is slower than that of major versions (maybe once 
a year, determined by users’ upgrade frequency). 
Each LTS version will be supported a period of time and typically there are no 
more than three active LTS versions.
By encouraging users to use LTS versions, we can ease the maintenance of 
released versions (bug fixes, back compatibility, and critical performance 
improvement).

(2) We can provide a third-party tool to do the migration of old-versioned 
savepoints. 
When users upgrade their versions, they can use the provided tool to migrate 
existing savepoints.
This can help move the code for savepoint migration out of the actual codebase, 
 making code focuses on current implementation.

What do you think?

Regards,
Xiaogang


> 在 2017年5月22日,下午1:39,Tzu-Li (Gordon) Tai <tzuli...@apache.org> 写道:
> 
> Hi Kostas,
> 
> Thanks for bringing this up!
> I think it is reasonable to keep this coherent with our timely-based release 
> model guarantees.
> 
> With the timely-based release model, there is a guarantee that the current 
> latest major version and the previous one is supported.
> For example, upon releasing 1.3, only 1.3 and 1.2 will still be supported by 
> the community for any required bug fixes.
> I think this was initially decided not only to ease old version maintenance 
> efforts for the community, but also as a means to let users upgrade their 
> Flink versions in a reasonable pace (at least every other major release.)
> 
> Therefore, I think its also reasonable to also clearly state that savepoints 
> compatibility will only be guaranteed for the previous release.
> Although I think at the moment almost, if not all, of the current code still 
> maintains compatibility for 1.1, in the long run these migration codes would 
> definitely start to pile up and pollute the actual codebase if we try to 
> always be compatible with all previous versions.
> 
> Cheers,
> Gordon
> 
> 
> On 21 May 2017 at 2:24:53 AM, Kostas Kloudas (k.klou...@data-artisans.com) 
> wrote:
> 
> Hi Chesnay, 
> 
> I believe that for APIs we already have a pretty clear policy with the 
> annotations. 
> I was referring to savepoints and state related backwards compatibility. 
> 
> 
>> On May 20, 2017, at 7:20 PM, Chesnay Schepler <ches...@apache.org> wrote: 
>> 
>> I think it would be a good to clarify what kind of backwards-compatibilitiy 
>> we're talking about here. As in are we talking about APIs or savepoints? 
>> 
>> On 20.05.2017 19:09, Kostas Kloudas wrote: 
>>> Hi all, 
>>> 
>>> As we are getting closer to releasing Flink-1.3, I would like to open a 
>>> discussion 
>>> on how far back we provide backwards compatibility for. 
>>> 
>>> The reason for opening the discussion is that i) for the users and for the 
>>> adoption of the project, it is good to have an explicitely stated policy 
>>> that implies 
>>> certain guarantees, and ii) keeping code and tests for backwards 
>>> compatibility with 
>>> Flink-1.1 does not offer much. On the contrary, I think that it leads to: 
>>> 
>>> 1) dead or ugly code in the codebase, e.g. deprecated class fields that 
>>> could go away and 
>>> ugly if() loops (see aligned window operators that were deprecated in 1.2 
>>> and are now 
>>> normal windows), etc 
>>> 2) expensive tests (as, normally, they read from a savepoint) 
>>> 3) binary files in the codebase for holding the aforementioned savepoints 
>>> 
>>> My proposal for such a policy would be to offer backwards compatibility for 
>>> one previous version. 
>>> 
>>> This means that 1.3 will be compatible with 1.2 (not 1.1). This still 
>>> allows a clear 
>>> "backwards compatibility" path when jumping versions (a user that goes 
>>> from 1.1 to 1.3 can go initially 1.1 -> 1.2, take a savepoint, and then 1.2 
>>> -> 1.3), 
>>> while also allowing us to clean up the codebase a bit. 
>>> 
>>> What do you think? 
>>> 
>>> Kostas 
>> 
>> 
> 

Reply via email to