This ticket also suggest function that doesn't support Python 3 https://issues.apache.org/jira/browse/AIRFLOW-2243
Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Sunday, February 24, 2019 9:25 PM, Bas Harenslak <basharens...@godatadriven.com> wrote: > Following the comments to warn users about dropping Python 2, I went ahead > and added the deprecation library to Airflow in this PR: > https://github.com/apache/airflow/pull/4763, and added the first @deprecated > annotation. > > I checked a few major packages and all use some self-made annotation for > showing deprecation warnings. I didn’t want to re-invent the wheel and used a > tiny package for it. More details are in the PR itself. If you know a better > way to do this, let me know :-) > > Bas > > On 17 Feb 2019, at 09:00, Bas Harenslak > <basharenslak@godatadriven.commailto:basharens...@godatadriven.com> wrote: > > The PR comments say drop hdfs3 and use pyarrow, however this is not > implemented yet. > > On 17 Feb 2019, at 08:42, zhong jiajie > <zhongjiajie955@hotmail.commailto:zhongjiajie...@hotmail.com> wrote: > > I overlook the PRhttps://github.com/apache/airflow/pull/3560 still use hdfs3 > rather than PyArrow. So which package we choice finilly? > [https://avatars3.githubusercontent.com/u/307739?s=400&v=4]https://github.com/apache/airflow/pull/3560 > > [AIRFLOW-2697] Drop snakebite in favour of hdfs3 by jrderuiter ・ Pull Request > #3560 ・ apache/airflow ・ GitHubhttps://github.com/apache/airflow/pull/3560 > Commit #52a2ff3 changes the HdfsSensor classes for the new HdfsHook. The > biggest change is that it rewrites the HdfsSensor + HdfsSensorFolder (from > contrib) sensors into a HdfsFileSensor and a HdfsFolderSensor class, which > aim to retain the functionality of the previous classes. > github.comhttp://github.com > > One blocker is HDFS support. Right now we rely on Snakebite which is > Python2 only, so we need to update this as well. There is an open ticket: > https://jira.apache.org/jira/browse/AIRFLOW-2697 > > Op vr 15 feb. 2019 om 07:55 schreef zhong jiajie <zhongjiajie...@hotmail.com > : > > Agree with XD's second idea, we should warn Airflow users. > > My two cents: > > 1. Can we discuss on the timeline for 2.0? Another big change is FAB-UI > ONLY, which is already in master branch. Personally I think it’s already > worth considering 2.0, unless there is other existing expectation for 2.0. > > 2. Before we completely drop Python2 support, we can start adding > warning/reminder mentioning that it’s stronly recommended to use py3 & py2 > drooping can happen “at any time” for Airflow (we can add these warnings > in 1.10.3?) > > XD > > On Thu, Feb 14, 2019 at 18:42 Ash Berlin-Taylor a...@apache.org wrote: > > > We talked about this in the past < > > http://mail-archives.apache.org/mod_mbox/airflow-dev/201808.mbox/<CAK%2Be0oujpY1A0mD0G775CmmRneVDjQMTi99aqK%2BA_C07ZW8ySA%40mail.gmail.com> > > but I'd like to bring it up again with a more concrete proposal. > > At the time Airflow 2.0 wasn't very concrete, but now with a lot of > otherwise breaking changes already on master how about we use that time > to > also drop support for Py2? > > (Reminder: Python 2 is reaching End of Life in January 1, 2020 and will > receive zero updates, even security ones past this date. At least not by > the Python team. I imagine RedHat will still be running Python 2.6 in > 2120 > ;) ) > > -ash
