+1 and the draft sounds good On Thu, May 30, 2019, 11:32 AM Xiangrui Meng <men...@gmail.com> wrote:
> Here is the draft announcement: > > === > Plan for dropping Python 2 support > > As many of you already knew, Python core development team and many > utilized Python packages like Pandas and NumPy will drop Python 2 support > in or before 2020/01/01. Apache Spark has supported both Python 2 and 3 > since Spark 1.4 release in 2015. However, maintaining Python 2/3 > compatibility is an increasing burden and it essentially limits the use of > Python 3 features in Spark. Given the end of life (EOL) of Python 2 is > coming, we plan to eventually drop Python 2 support as well. The current > plan is as follows: > > * In the next major release in 2019, we will deprecate Python 2 support. > PySpark users will see a deprecation warning if Python 2 is used. We will > publish a migration guide for PySpark users to migrate to Python 3. > * We will drop Python 2 support in a future release in 2020, after Python > 2 EOL on 2020/01/01. PySpark users will see an error if Python 2 is used. > * For releases that support Python 2, e.g., Spark 2.4, their patch > releases will continue supporting Python 2. However, after Python 2 EOL, we > might not take patches that are specific to Python 2. > === > > Sean helped make a pass. If it looks good, I'm going to upload it to Spark > website and announce it here. Let me know if you think we should do a VOTE > instead. > > On Thu, May 30, 2019 at 9:21 AM Xiangrui Meng <men...@gmail.com> wrote: > >> I created https://issues.apache.org/jira/browse/SPARK-27884 to track the >> work. >> >> On Thu, May 30, 2019 at 2:18 AM Felix Cheung <felixcheun...@hotmail.com> >> wrote: >> >>> We don’t usually reference a future release on website >>> >>> > Spark website and state that Python 2 is deprecated in Spark 3.0 >>> >>> I suspect people will then ask when is Spark 3.0 coming out then. Might >>> need to provide some clarity on that. >>> >> >> We can say the "next major release in 2019" instead of Spark 3.0. Spark >> 3.0 timeline certainly requires a new thread to discuss. >> >> >>> >>> >>> ------------------------------ >>> *From:* Reynold Xin <r...@databricks.com> >>> *Sent:* Thursday, May 30, 2019 12:59:14 AM >>> *To:* shane knapp >>> *Cc:* Erik Erlandson; Mark Hamstra; Matei Zaharia; Sean Owen; Wenchen >>> Fen; Xiangrui Meng; dev; user >>> *Subject:* Re: Should python-2 be supported in Spark 3.0? >>> >>> +1 on Xiangrui’s plan. >>> >>> On Thu, May 30, 2019 at 7:55 AM shane knapp <skn...@berkeley.edu> wrote: >>> >>>> I don't have a good sense of the overhead of continuing to support >>>>> Python 2; is it large enough to consider dropping it in Spark 3.0? >>>>> >>>>> from the build/test side, it will actually be pretty easy to continue >>>> support for python2.7 for spark 2.x as the feature sets won't be expanding. >>>> >>> >>>> that being said, i will be cracking a bottle of champagne when i can >>>> delete all of the ansible and anaconda configs for python2.x. :) >>>> >>> >> On the development side, in a future release that drops Python 2 support >> we can remove code that maintains python 2/3 compatibility and start using >> python 3 only features, which is also quite exciting. >> >> >>> >>>> shane >>>> -- >>>> Shane Knapp >>>> UC Berkeley EECS Research / RISELab Staff Technical Lead >>>> https://rise.cs.berkeley.edu >>>> >>>
