I mean python 2 _will be_ deprecated in Spark 3. On Thu, 15 Aug 2019, 18:37 Hyukjin Kwon, <gurwls...@gmail.com> wrote:
> Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is > already deprecated. > > On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko, <fo...@driesprong.frl> > wrote: > >> Sorry for the late reply, was a bit busy lately, but I still would like >> to share my thoughts on this. >> >> For Apache Airflow we're dropping support for Python 2 in the next major >> release. We're now supporting Python 3.5+. Mostly because: >> >> - Easier to maintain and test, and less if/else constructions for the >> different Python versions. Also, not having to test against Python 2.x >> reduces the build matrix. >> - Python 3 has support for typing. From Python 3.5 you can include >> provisional type hints. An excellent presentation by Guido himself: >> https://www.youtube.com/watch?v=2wDvzy6Hgxg. From Python 3.5 it is >> still provisional, but it is a really good idea. From Airflow we've >> noticed >> that using mypy is catching bugs early: >> - This will put less stress on the (boring part of the) reviewing >> process since a lot of this stuff is checked automatically. >> - For new developers, it is easier to read the code because of the >> annotations. >> - Can be used as an input for generated documentation (or check if >> it still in sync with the docstrings) >> - Easier to extend the code since you know what kind of types you >> can expect, and your IDE will also pick up the hinting. >> - Python 2.x will be EOL end this year >> >> I have a strong preference to migrate everything to Python 3. >> >> Cheers, Fokko >> >> >> Op wo 7 aug. 2019 om 12:14 schreef Weichen Xu <weichen...@databricks.com >> >: >> >>> All right we could support both Python 2 and Python 3 for spark 3.0. >>> >>> On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: >>> >>>> We didn't drop Python 2 yet although it's deprecated. So I think It >>>> should support both Python 2 and Python 3 at the current status. >>>> >>>> 2019년 8월 7일 (수) 오후 6:54, Weichen Xu <weichen...@databricks.com>님이 작성: >>>> >>>>> Hi all, >>>>> >>>>> I would like to discuss the compatibility for dev scripts. Because we >>>>> already decided to deprecate python2 in spark 3.0, for development scripts >>>>> under dev/ , we have two choice: >>>>> 1) Migration from Python 2 to Python 3 >>>>> 2) Support both Python 2 and Python 3 >>>>> >>>>> I tend to option (2) which is more friendly to maintenance. >>>>> >>>>> Regards, >>>>> Weichen >>>>> >>>>
