Hi Dian, +1 to drop Python 2 directly.
Just as @jincheng said, things would be more complicated if we are going to support python UDFs. The python UDFs will introduce a lot of python dependencies which will also drop the support of Python 2, such as beam, pandas, pyarrow, etc. Given this and Python 2 will reach EOL on Jan 1 2020. I think we can drop Python 2 in Flink as well. As for the two options, I think we can drop it directly in 1.10. The flink-python is introduced just from 1.9, I think it's safe to drop it for now. And we can also benefit from it when we add support for python UDFs. Best, Hequn On Wed, Oct 9, 2019 at 8:40 AM jincheng sun <sunjincheng...@gmail.com> wrote: > Hi Dian， > > Thanks for bringing this discussion! > > In Flink 1.9 we only add Python Table API mapping to Java Table API(without > Python UDFs), there no special requirements for Python version, so we add > python 2,7 support. But for Flink 1.10, we add the Python UDFs support, > i.e., user will add more python code in Flink job and more requirements for > the features of the Python language.So I think It's better to follow the > rhythm of Python official. > > Option 2 is the most conservative and correct approach, but for the current > situation, we cooperate with the Beam community and use Beam's portability > framework for UDFs support, so we prefer to adopt the Option 1. > > Best, > Jincheng > > > > Dian Fu <dian0511...@gmail.com> 于2019年10月8日周二 下午10:34写道： > > > Hi everyone, > > > > I would like to propose to drop Python 2 support(Currently Python 2.7, > > 3.5, 3.6, 3.7 are all supported in Flink) as it's coming to an end at Jan > > 1, 2020 [1]. A lot of projects [2][3][4] has already stated or are > planning > > to drop Python 2 support. > > > > The benefits of dropping Python 2 support are: > > 1. Maintaining Python 2/3 compatibility is a burden and it makes the code > > complicate as Python 2 and Python 3 is not compatible. > > 2. There are many features which are only available in Python 3.x such as > > Type Hints[5]. We can only make use of this kind of features after > dropping > > the Python 2 support. > > 3. Flink-python depends on third-part projects, such as Apache Beam (may > > add more dependencies such as pandas, etc in the near future), it's not > > possible to upgrade them to the latest version once they drop the Python > 2 > > support. > > > > Here are the options we have: > > 1. Drop Python 2 support in 1.10: > > As flink-python module is a new module added since 1.9.0 and so dropping > > Python 2 support at the early stage seems a good choice for us. > > 2. Deprecate Python 2 in 1.10 and drop its support in 1.11: > > As 1.10 is planned to be released around the beginning of 2020. This is > > also aligned with the official Python 2 support. > > > > Personally I prefer option 1 as flink-python is new module and there is > no > > much history reasons to consider. > > > > Looking forward to your feedback! > > > > Regards, > > Dian > > > > [1] https://pythonclock.org/ <https://pythonclock.org/> > > [2] https://python3statement.org/ <https://python3statement.org/> > > [3] > https://spark.apache.org/news/plan-for-dropping-python-2-support.html > > <https://spark.apache.org/news/plan-for-dropping-python-2-support.html> > > [4] > > > https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E > > < > > > https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E > > > > > [5] > > > https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 > > < > > > https://stackoverflow.com/questions/32557920/what-are-type-hints-in-python-3-5 > > > >
