We have a few non-deterministic unit test failures that are affecting many -- but not all -- PRs. I believe they are being ignored as "unrelated" but they have the potential to mask real issues and should be addressed. Unfortunately they're out of my expertise so I'm going to list the ones I've identified and hope someone smarter than me can see if they can help!
In particular, we have a number of simple PR's that should obviously have no problems (typos, readme edits, etc.) that are nonetheless failing tests, causing frustration for all. Here is one from just this morning: https://github.com/apache/incubator-airflow/pull/1705/files Thanks in advance! 1. Python 3 Mysql (this one is pretty common), due to not being able to find "beeline" which I believe is related to Hive. This is the error: ====================================================================== ERROR: test_mysql_to_hive_partition (tests.TransferTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/apache/incubator-airflow/tests/operators/operators.py", line 208, in test_mysql_to_hive_partition t.run(start_date=DEFAULT_DATE, end_date=DEFAULT_DATE, force=True) File "/home/travis/build/apache/incubator-airflow/airflow/models.py", line 2350, in run force=force,) File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py", line 54, in wrapper result = func(*args, **kwargs) File "/home/travis/build/apache/incubator-airflow/airflow/models.py", line 1388, in run result = task_copy.execute(context=context) File "/home/travis/build/apache/incubator-airflow/airflow/operators/mysql_to_hive.py", line 131, in execute recreate=self.recreate) File "/home/travis/build/apache/incubator-airflow/airflow/hooks/hive_hooks.py", line 322, in load_file self.run_cli(hql) File "/home/travis/build/apache/incubator-airflow/airflow/hooks/hive_hooks.py", line 212, in run_cli cwd=tmp_dir) File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 858, in __init__ restore_signals, start_new_session) File "/opt/python/3.4.2/lib/python3.4/subprocess.py", line 1456, in _execute_child raise child_exception_type(errno_num, err_msg) nose.proxy.FileNotFoundError: [Errno 2] No such file or directory: 'beeline' 2. Python 3 Postgres (this one is really infrequent): ====================================================================== FAIL: Test that ignore_first_depends_on_past doesn't affect results ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/travis/build/apache/incubator-airflow/tests/jobs.py", line 349, in test_dagrun_deadlock_ignore_depends_on_past run_kwargs=dict(ignore_first_depends_on_past=True)) File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py", line 54, in wrapper result = func(*args, **kwargs) File "/home/travis/build/apache/incubator-airflow/tests/jobs.py", line 221, in evaluate_dagrun self.assertEqual(ti.state, expected_state) nose.proxy.AssertionError: None != 'success' 3. Mysql (py2 and py3, infrequent). This appears to happen when the SLA code is called wiht mysql. Bizarrely, this doesn't appear to actually raise an error in the test -- it just prints a logging error. It must be trapped somewhere. ERROR [airflow.jobs.SchedulerJob] Boolean value of this clause is not defined Traceback (most recent call last): File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py", line 667, in _do_dags self.manage_slas(dag) File "/home/travis/build/apache/incubator-airflow/airflow/utils/db.py", line 53, in wrapper result = func(*args, **kwargs) File "/home/travis/build/apache/incubator-airflow/airflow/jobs.py", line 301, in manage_slas .all() File "/home/travis/build/apache/incubator-airflow/.tox/py34-cdh-airflow_backend_mysql/lib/python3.4/site-packages/sqlalchemy/sql/elements.py", line 2760, in __bool__ raise TypeError("Boolean value of this clause is not defined") TypeError: Boolean value of this clause is not defined
