5 versions is too many IMO. We've had issues with Python precommit resource usage in the past, and adding another version would surely exacerbate those issues. And we have also already had to leave out certain features on 3.5 [1]. Therefore, I am in favor of dropping 3.5 before adding 3.8. After dropping Python 2 and adding 3.8, that will leave us with the latest three minor versions (3.6, 3.7, 3.8), which I think is closer to the "sweet spot." Though I would be interested in hearing if there are any users who would prefer we continue supporting 3.5.
[1] https://github.com/apache/beam/blob/8658b95545352e51f35959f38334f3c7df8b48eb/sdks/python/apache_beam/runners/portability/flink_runner.py#L55 On Wed, Feb 26, 2020 at 3:00 PM Valentyn Tymofieiev <valen...@google.com> wrote: > I would like to start a discussion about identifying a guideline for > answering questions like: > > 1. When will Beam support a new Python version (say, Python 3.8)? > 2. When will Beam drop support for an old Python version (say, Python > 3.5)? > 3. How many Python versions should we aim to support concurrently > (investigate issues, have continuous integration tests)? > 4. What comes first: adding support for a new version (3.8) or deprecating > older one (3.5)? This may affect the max load our test infrastructure needs > to sustain. > > We are already getting requests for supporting Python 3.8 and there were > some good reasons[1] to drop support for Python 3.5 (at least, early > versions of 3.5). Answering these questions would help set expectations in > Beam user community, Beam dev community, and may help us establish > resource requirements for test infrastructure and plan efforts. > > PEP-0602 [2] establishes a yearly release cycle for Python versions > starting from 3.9. Each release is a long-term support release and is > supported for 5 years: first 1.5 years allow for general bug fix support, > remaining 3.5 years have security fix support. > > At every point, there may be up to 5 Python minor versions that did not > yet reach EOL, see "Release overlap with 12 month diagram" [3]. We can try > to support all of them, but that may come at a cost of velocity: we will > have more tests to maintain, and we will have to develop Beam against a > lower version for a longer period. Supporting less versions will have > implications for user experience. It also may be difficult to ensure > support of the most recent version early, since our dependencies (e.g. > picklers) may not be supporting them yet. > > Currently we support 4 Python versions (2.7, 3.5, 3.6, 3.7). > > Is 4 versions a sweet spot? Too much? Too little? What do you think? > > [1] https://github.com/apache/beam/pull/10821#issuecomment-590167711 > [2] https://www.python.org/dev/peps/pep-0602/ > [3] https://www.python.org/dev/peps/pep-0602/#id17 >
