Hi Weijie,
I appreciate your thoughts on removing the outdated Flink versions. I
completely agree that it's essential to focus on maintaining the still
relevant versions.
To streamline our process, I propose we build a mechanism to expire
old clients after a certain period, ensuring that only actively
supported Flink/Spark/MR versions are in use. This would simplify our
support efforts and encourage users to update to newer versions.
Regarding your question about how many Flink versions we should retain
support for, keeping support for the last {4} or {5} major versions
could be a good approach, depending on user feedback.
Let me know your thoughts!
Thanks,
Ethan Feng
Nicholas Jiang <[email protected]> 于2024年12月19日周四 11:36写道：
>
> Hi weijie,
>
> Thanks for driving the out-of-dated flink versions. +1 for removing the
> out-of-dated flink versions because the support of different Flink versions
> are similar.
>
> IMO, We could firstly remove the support for Flink 1.14 and 1.15 version,
> which versions are very old and few functional features. Meanwhile, the
> support of other Flink versions could be gradually removed in the Celeborn
> major version.
>
> BTW, how many Flink versions should we retain support for?
>
> Regards,
> Nicholas Jiang
>
> On 2024/12/18 06:18:17 weijie guo wrote:
> > Hi all,
> >
> > I would like to suggest that we consider ending the support for some of the
> > older Flink versions. The main reasons are as follows:
> >
> > 1. Currently, the minimum supported version is Flink-1.14. But Flink
> > releases like 1.14 and 1.15 have been out of date for a long time, and
> > some important features (speculative execution, AQE, etc.) of Flink batch
> > are missing.
> >
> > 2. The Flink community currently does not support versions lower than 1.18,
> > considering that Flink 1.20 will be the first LTS version, the existing 1.x
> > users will gradually upgrade to 1.20 also.
> >
> > 3. There is a big difference between the shuffle API of the old version and
> > the current Flink code base, and it is tedious to maintain compatibility.
> > Moreover, new features such as JM Failover and hybrid shuffle cannot be
> > supported in out-of-dated Flink release. If users do have a need to adapt
> > older Flink versions, it's not difficult for them to do it themselves, but
> > this can reduce our maintenance burden.
> >
> > For now, communities such as Apache Iceberg have taken a more radical
> > strategy that in line with Flink's official supported version. We don't
> > have to go that far yet, but it feels like at least we can remove the
> > versions 1.14 and 1.15 for now.
> >
> >
> > Best regards,
> >
> > Weijie
> >
