I wasn't paying enough attention. I can reproduce it. I'm not sure it was intentional but the bug was introduced in https://github.com/apache/incubator-airflow/pull/3171 <https://github.com/apache/incubator-airflow/pull/3171>
I'd rather not release with a regression since 1.9, so updating my vote to -0.5 (binding), but not blocking/vetoing release if others are okay with it. -ash > On 12 Jul 2018, at 13:46, Ash Berlin-Taylor <ash_airflowl...@firemirror.com> > wrote: > > That said I can't reproduce it myself (I will take discussion to the Jira > ticket) > > -ash > >> On 12 Jul 2018, at 13:36, Ash Berlin-Taylor <ash_airflowl...@firemirror.com> >> wrote: >> >> Possible blocking regression here - .airflowignore doesn't seem to be >> respected anymore >> >> https://issues.apache.org/jira/browse/AIRFLOW-2729 >> <https://issues.apache.org/jira/browse/AIRFLOW-2729> >> >> -ash >>> On 12 Jul 2018, at 08:45, Bolke de Bruin <bdbr...@gmail.com> wrote: >>> >>> Hi Jakob, >>> >>> Thanks. We do include an INSTALL document that explains how to well install >>> airflow and is quite a standard location for install instructions. Or did >>> the install instruction needed to be included in the Vote? >>> >>> B. >>> >>> >>> >>> Verstuurd vanaf mijn iPad >>> >>>> Op 12 jul. 2018 om 07:54 heeft Jakob Homan <jgho...@gmail.com> het >>>> volgende geschreven: >>>> >>>> +1 (binding) >>>> >>>> * Sigs look good >>>> * Artifact has incubating in name >>>> * LICENSE/NOTICE/DISCLAIMER look good >>>> - nit: DISCLAIMER is not word wrapped to 80 chars as I've seen in >>>> other projects. >>>> * Nit: Last release Sebb called out "Copyright 2016 and onwards" in >>>> NOTICE as being imprecise. This language remains. >>>> * Spot check on license headers looks ok. >>>> * Nit: Last release there was a request for instructions on how to >>>> build the code. This isn't included. >>>> >>>> -Jakob >>>> >>>>> On 11 July 2018 at 19:50, Sid Anand <san...@apache.org> wrote: >>>>> FYI! >>>>> I just installed the release candidate. The first thing I noticed is a >>>>> missing tool tip for the Null State in the Recent Tasks column on the >>>>> landing page. Since the null globe is new to this UI, users will likely >>>>> hover over it to inquire what it means... and will be left wanting. Of >>>>> course, they could click on the globe, which will take them to >>>>> http://localhost:8080/admin/taskinstance/?flt1_dag_id_equals=example_bash_operator&flt2_state_equals=null, >>>>> which will always show an empty list, leaving them a bit more confused. >>>>> >>>>> -s >>>>> >>>>> On Wed, Jul 11, 2018 at 3:13 PM Carl Johan Gustavsson >>>>> <carl.jo...@tictail.com.invalid> wrote: >>>>> >>>>>> Hi Bolke, >>>>>> >>>>>> (Switching email to avoid moderation on my emails.) >>>>>> >>>>>> The normal Airflow test suite does not fail as it uses a LC_ALL set to >>>>>> utf-8. >>>>>> >>>>>> I think it is a proper test though, it is a minimal reproducible version >>>>>> of >>>>>> the code that fails. And the only difference in behaviour is at 3.7 which >>>>>> we don’t support anyway so I’m fairly sure it is broken for all supported >>>>>> Python 3 versions. >>>>>> >>>>>> I now tried running the tests in docker using 3.5 with the LC_ALL/LANG >>>>>> unset and I see the same failure. >>>>>> >>>>>> I don’t think this is a big thing though and we could release it without >>>>>> the fix I made. I think most people run it with a sane LC_ALL, but >>>>>> apparently we didn’t. >>>>>> Here’s the log for the test: >>>>>> >>>>>>> docker run -t -i -v `pwd`:/airflow/ python:3.5 bash >>>>>> root@b99b297df111:/# locale >>>>>> LANG=C.UTF-8 >>>>>> LANGUAGE= >>>>>> LC_CTYPE="C.UTF-8" >>>>>> LC_NUMERIC="C.UTF-8" >>>>>> LC_TIME="C.UTF-8" >>>>>> LC_COLLATE="C.UTF-8" >>>>>> LC_MONETARY="C.UTF-8" >>>>>> LC_MESSAGES="C.UTF-8" >>>>>> LC_PAPER="C.UTF-8" >>>>>> LC_NAME="C.UTF-8" >>>>>> LC_ADDRESS="C.UTF-8" >>>>>> LC_TELEPHONE="C.UTF-8" >>>>>> LC_MEASUREMENT="C.UTF-8" >>>>>> LC_IDENTIFICATION="C.UTF-8" >>>>>> LC_ALL= >>>>>>> unset LANG >>>>>> root@b99b297df111:/# locale >>>>>> LANG= >>>>>> LANGUAGE= >>>>>> LC_CTYPE="POSIX" >>>>>> LC_NUMERIC="POSIX" >>>>>> LC_TIME="POSIX" >>>>>> LC_COLLATE="POSIX" >>>>>> LC_MONETARY="POSIX" >>>>>> LC_MESSAGES="POSIX" >>>>>> LC_PAPER="POSIX" >>>>>> LC_NAME="POSIX" >>>>>> LC_ADDRESS="POSIX" >>>>>> LC_TELEPHONE="POSIX" >>>>>> LC_MEASUREMENT="POSIX" >>>>>> LC_IDENTIFICATION="POSIX" >>>>>> LC_ALL= >>>>>> root@b99b297df111:/# pip install -e .[devel] >>>>>> root@b99b297df111:/airflow# ./run_unit_tests.sh >>>>>> + export AIRFLOW_HOME=/root/airflow >>>>>> + AIRFLOW_HOME=/root/airflow >>>>>> + export AIRFLOW__CORE__UNIT_TEST_MODE=True >>>>>> + AIRFLOW__CORE__UNIT_TEST_MODE=True >>>>>> + export AIRFLOW__TESTSECTION__TESTKEY=testvalue >>>>>> + AIRFLOW__TESTSECTION__TESTKEY=testvalue >>>>>> + export AIRFLOW_USE_NEW_IMPORTS=1 >>>>>> + AIRFLOW_USE_NEW_IMPORTS=1 >>>>>> +++ dirname ./run_unit_tests.sh >>>>>> ++ cd . >>>>>> ++ pwd >>>>>> + DIR=/airflow >>>>>> + export PYTHONPATH=:/airflow/tests/test_utils >>>>>> + PYTHONPATH=:/airflow/tests/test_utils >>>>>> + nose_args= >>>>>> + which airflow >>>>>> + echo 'Initializing the DB' >>>>>> Initializing the DB >>>>>> + airflow resetdb >>>>>> + yes >>>>>> Traceback (most recent call last): >>>>>> File "/usr/local/bin/airflow", line 6, in <module> >>>>>> exec(compile(open(__file__).read(), __file__, 'exec')) >>>>>> File "/airflow/airflow/bin/airflow", line 21, in <module> >>>>>> from airflow import configuration >>>>>> File "/airflow/airflow/__init__.py", line 35, in <module> >>>>>> from airflow import configuration as conf >>>>>> File "/airflow/airflow/configuration.py", line 106, in <module> >>>>>> DEFAULT_CONFIG = f.read() >>>>>> File "/usr/local/lib/python3.5/encodings/ascii.py", line 26, in decode >>>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>>> 21082: >>>>>> ordinal not in range(128) >>>>>> + '[' '' ']' >>>>>> + '[' -z '' ']' >>>>>> + nose_args='--with-coverage --cover-erase --cover-html >>>>>> --cover-package=airflow --cover-html-dir=airflow/www/static/coverage >>>>>> --with-ignore-docstrings --rednose --with-timer -s -v >>>>>> --logging-level=DEBUG ' >>>>>> + echo 'Starting the unit tests with the following nose arguments: >>>>>> --with-coverage' --cover-erase --cover-html --cover-package=airflow >>>>>> --cover-html-dir=airflow/www/static/coverage --with-ignore-docstrings >>>>>> --rednose --with-timer -s -v --logging-level=DEBUG >>>>>> Starting the unit tests with the following nose arguments: >>>>>> --with-coverage >>>>>> --cover-erase --cover-html --cover-package=airflow >>>>>> --cover-html-dir=airflow/www/static/coverage --with-ignore-docstrings >>>>>> --rednose --with-timer -s -v --logging-level=DEBUG >>>>>> + nosetests --with-coverage --cover-erase --cover-html >>>>>> --cover-package=airflow --cover-html-dir=airflow/www/static/coverage >>>>>> --with-ignore-docstrings --rednose --with-timer -s -v >>>>>> --logging-level=DEBUG >>>>>> nose.plugins.cover: ERROR: Coverage not available: unable to import >>>>>> coverage module >>>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in >>>>>> position 21082: ordinal not in range(128)) ... ERROR >>>>>> Failure: UnicodeDecodeError ('ascii' codec can't decode byte 0xe2 in >>>>>> position 21082: ordinal not in range(128)) ... ERROR >>>>>> ====================================================================== >>>>>> 1) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode byte >>>>>> 0xe2 >>>>>> in position 21082: ordinal not in range(128)) >>>>>> ---------------------------------------------------------------------- >>>>>> Traceback (most recent call last): >>>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in >>>>>> runTest >>>>>> raise self.exc_val.with_traceback(self.tb) >>>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in >>>>>> loadTestsFromName >>>>>> addr.filename, addr.module) >>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in >>>>>> importFromPath >>>>>> return self.importFromDir(dir_path, fqname) >>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in >>>>>> importFromDir >>>>>> mod = load_module(part_fqname, fh, filename, desc) >>>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module >>>>>> return load_package(name, filename) >>>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package >>>>>> return _load(spec) >>>>>> <frozen importlib._bootstrap> line 693 in _load >>>>>> >>>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked >>>>>> >>>>>> <frozen importlib._bootstrap_external> line 697 in exec_module >>>>>> >>>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed >>>>>> >>>>>> airflow/__init__.py line 35 in <module> >>>>>> from airflow import configuration as conf >>>>>> airflow/configuration.py line 106 in <module> >>>>>> DEFAULT_CONFIG = f.read() >>>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode >>>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>>> 21082: ordinal not in range(128) >>>>>> ====================================================================== >>>>>> 2) ERROR: Failure: UnicodeDecodeError ('ascii' codec can't decode byte >>>>>> 0xe2 >>>>>> in position 21082: ordinal not in range(128)) >>>>>> ---------------------------------------------------------------------- >>>>>> Traceback (most recent call last): >>>>>> /usr/local/lib/python3.5/site-packages/nose/failure.py line 39 in >>>>>> runTest >>>>>> raise self.exc_val.with_traceback(self.tb) >>>>>> /usr/local/lib/python3.5/site-packages/nose/loader.py line 418 in >>>>>> loadTestsFromName >>>>>> addr.filename, addr.module) >>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 47 in >>>>>> importFromPath >>>>>> return self.importFromDir(dir_path, fqname) >>>>>> /usr/local/lib/python3.5/site-packages/nose/importer.py line 94 in >>>>>> importFromDir >>>>>> mod = load_module(part_fqname, fh, filename, desc) >>>>>> /usr/local/lib/python3.5/imp.py line 245 in load_module >>>>>> return load_package(name, filename) >>>>>> /usr/local/lib/python3.5/imp.py line 217 in load_package >>>>>> return _load(spec) >>>>>> <frozen importlib._bootstrap> line 693 in _load >>>>>> >>>>>> <frozen importlib._bootstrap> line 673 in _load_unlocked >>>>>> >>>>>> <frozen importlib._bootstrap_external> line 697 in exec_module >>>>>> >>>>>> <frozen importlib._bootstrap> line 222 in _call_with_frames_removed >>>>>> >>>>>> tests/__init__.py line 25 in <module> >>>>>> from .configuration import * >>>>>> tests/configuration.py line 28 in <module> >>>>>> from airflow import configuration >>>>>> airflow/__init__.py line 35 in <module> >>>>>> from airflow import configuration as conf >>>>>> airflow/configuration.py line 106 in <module> >>>>>> DEFAULT_CONFIG = f.read() >>>>>> /usr/local/lib/python3.5/encodings/ascii.py line 26 in decode >>>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>>> 21082: ordinal not in range(128) >>>>>> >>>>>> [error] 100.00% nose.failure.Failure.runTest: 0.0004s >>>>>> >>>>>> ----------------------------------------------------------------------------- >>>>>> 2 tests run in 0.060 seconds. >>>>>> 2 errors (0 tests passed) >>>>>> >>>>>> >>>>>> -- >>>>>> Carl Johan Gustavsson >>>>>> >>>>>> On 11 July 2018 at 23:24:01, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>>> >>>>>> Hi Carl, >>>>>> >>>>>> That is not a real check, ie. Does Airflow have the same issue clean >>>>>> install on 3.5? Travis’ tests run on 3.5. >>>>>> >>>>>> B. >>>>>> >>>>>> Verstuurd vanaf mijn iPad >>>>>> >>>>>>> Op 10 jul. 2018 om 15:10 heeft Carl Johan Gustavsson < >>>>>> carl.j.gustavs...@gmail.com> het volgende geschreven: >>>>>>> >>>>>>> Hi Bolke, >>>>>>> >>>>>>> >>>>>>> I did a quick test on 3.5.5, 3.4.0 and 3.7.0 on OS X now, all but 3.7 >>>>>> breaks >>>>>>> >>>>>>> Quick repro: >>>>>>> >>>>>>> ➜ ~ pyenv local 3.5.5 >>>>>>> ➜ ~ locale >>>>>>> LANG= >>>>>>> LC_COLLATE="C" >>>>>>> LC_CTYPE="C" >>>>>>> LC_MESSAGES="C" >>>>>>> LC_MONETARY="C" >>>>>>> LC_NUMERIC="C" >>>>>>> LC_TIME="C" >>>>>>> LC_ALL= >>>>>>> ➜ ~ cat testweird.txt >>>>>>> ’ >>>>>>> ➜ ~ python >>>>>>> Python 3.5.5 (default, Jul 7 2018, 17:00:56) >>>>>>> [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin >>>>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>>>>> open('testweird.txt').read() >>>>>>> Traceback (most recent call last): >>>>>>> File "<stdin>", line 1, in <module> >>>>>>> File >>>>>>> "/Users/cjg/.pyenv/versions/3.5.5/lib/python3.5/encodings/ascii.py", >>>>>> line 26, in decode >>>>>>> return codecs.ascii_decode(input, self.errors)[0] >>>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: >>>>>> ordinal not in range(128) >>>>>>>>>> >>>>>>> >>>>>>> >>>>>>> Maybe not a blocking change but it is a breaking change from 1.9 I >>>>>>> guess. >>>>>>> >>>>>>> >>>>>>> / Carl Johan >>>>>>> >>>>>>> >>>>>>>> On 10 July 2018 at 14:48:47, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>>>>> >>>>>>>> Hi Carl, >>>>>>>> >>>>>>>> Did you this on python 3.5 as well? 3.6 is not an officially supported >>>>>> (yet). As a workaround is available I won’t consider this blocking btw. >>>>>>>> >>>>>>>> Bolke >>>>>>>> >>>>>>>> Verstuurd vanaf mijn iPad >>>>>>>> >>>>>>>> Op 10 jul. 2018 om 11:53 heeft Carl Johan Gustavsson < >>>>>> carl.j.gustavs...@gmail.com> het volgende geschreven: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> First of all, thank you for all the work with the release management. >>>>>>>>> >>>>>>>>> I ran in to a weird issue testing the RC1, running under Python 3.6.0 >>>>>>>>> / >>>>>> Ubuntu 14.04.5, upgrading from a master build from February. >>>>>>>>> >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 Traceback >>>>>> (most recent call last): >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>>> "/opt/virtualenv/tictail/pipeline/bin/airflow", line 21, in <module> >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from >>>>>>>>> airflow >>>>>> import configuration >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>>> >>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/__init__.py", >>>>>> line 35, in <module> >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 from >>>>>>>>> airflow >>>>>> import configuration as conf >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>>> >>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/site-packages/airflow/configuration.py", >>>>>> line 106, in <module> >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 >>>>>> DEFAULT_CONFIG = f.read() >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 File >>>>>> "/opt/virtualenv/tictail/pipeline/lib/python3.6/encodings/ascii.py", line >>>>>> 26, in decode >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 return >>>>>> codecs.ascii_decode(input, self.errors)[0] >>>>>>>>> Jul 10 08:50:33 hostname supervisord: airflow-webserver-01 >>>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position >>>>>> 20770: >>>>>> ordinal not in range(128) >>>>>>>>> >>>>>>>>> Removing the ’ on >>>>>> >>>>>> https://github.com/apache/incubator-airflow/blob/master/airflow/config_templates/default_airflow.cfg#L613 >>>>>> solved the issue for me, and digging a bit deeper it seems Airflow now >>>>>> requires setting LC_ALL=en_US.UTF-8 in the environment or similar to >>>>>> force >>>>>> Python to read the file as utf-8 and not ascii. (I think this was changed >>>>>> in to default to utf-8 in Python 3.7). >>>>>>>>> >>>>>>>>> I see 3 solutions for this >>>>>>>>> 1. Document that Airflow need to run with LC_ALL=en_US.UTF-8 or >>>>>> similar. >>>>>>>>> 2. Change the default config file to not contain non-ascii characters. >>>>>>>>> 3. Always read the file as unicode regardless of the LC_ALL >>>>>> environment, by the encoding='utf-8’ parameter to open(). >>>>>>>>> >>>>>>>>> I think 3 is the best solution, and I can prepare a PR for that if >>>>>> necessary . >>>>>>>>> >>>>>>>>> I guess this counts as -1 (non-binding) >>>>>>>>> >>>>>>>>> All the best >>>>>>>>> >>>>>>>>> Carl Johan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 8 July 2018 at 22:02:33, Bolke de Bruin (bdbr...@gmail.com) wrote: >>>>>>>>>> >>>>>>>>>> Hey all, >>>>>>>>>> >>>>>>>>>> I have cut Airflow 1.10.0 RC1. This email is calling a vote on the >>>>>> release, >>>>>>>>>> which will last for 72 hours. Consider this my (binding) +1. >>>>>>>>>> >>>>>>>>>> Airflow 1.10.0 RC 1 is available at: >>>>>>>>>> >>>>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/ < >>>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/1.10.0rc1/> >>>>>>>>>> >>>>>>>>>> apache-airflow-1.10.0rc1+incubating-source.tar.gz is a source release >>>>>> that >>>>>>>>>> comes with INSTALL instructions. >>>>>>>>>> apache-airflow-1.10.0rc1+incubating-bin.tar.gz is the binary Python >>>>>> "sdist" >>>>>>>>>> release. >>>>>>>>>> >>>>>>>>>> Public keys are available at: >>>>>>>>>> >>>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/ < >>>>>> https://dist.apache.org/repos/dist/release/incubator/airflow/> >>>>>>>>>> >>>>>>>>>> The amount of JIRAs fixed is over 700. Please have a look at the >>>>>> changelog. >>>>>>>>>> >>>>>>>>>> Please note that the version number excludes the `rcX` string as well >>>>>>>>>> as the "+incubating" string, so it's now simply 1.10.0. This will >>>>>> allow us >>>>>>>>>> to rename the artifact without modifying the artifact checksums when >>>>>> we >>>>>>>>>> actually release. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Bolke >>>>>> >> >
