josh-fell commented on pull request #18562: URL: https://github.com/apache/airflow/pull/18562#issuecomment-932587632
Looks like there is differing sorting behavior between Postgres and other backend databases for this test. The `TaskInstance` rows returned aren't always in the same order especially between database types. This is an issue because the assertion result is relative to a certain order of `TaskInstance` rows. In the failing test, these are the states of the `TaskInstances`: ```python tis = [ TaskInstance(dag.get_task("runme_0"), run_id=dr.run_id, state="success"), TaskInstance(dag.get_task("runme_1"), run_id=dr.run_id, state="failed"), ] ``` Running this test and printing the assertion list comprehension with multiple backends, you can see that Postgres retrieves the data in a different order than MySQL and SQLite. The `TaskInstance` "example_bash_operator.runme_1" is sorted before "example_bash_operator.runme_0": ``` shell **PostgreSQL 9.6** tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:00:12,320] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 21:00:12,630] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 21:00:12,641] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>] tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:00:13,373] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 21:00:13,699] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 21:00:13,709] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>, <TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>] **SQLite** tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 20:56:52,545] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 20:56:52,930] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 20:56:52,942] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>] tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 20:56:53,677] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 20:56:54,016] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 20:56:54,026] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>] *MySQL 5.7** tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[failed] [2021-10-01 21:06:09,569] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 21:06:09,891] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 21:06:09,901] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>] tests/www/views/test_views_dagrun.py::test_set_dag_runs_action[running] [2021-10-01 21:06:10,632] {dagbag.py:501} INFO - Filling up the DagBag from /opt/airflow/tests/dags [2021-10-01 21:06:10,965] {test_task_view_type_check.py:50} INFO - class_instance type: <class 'unusual_prefix_5d280a9b385120fec3c40cfe5be04e2f41b6b5e8_test_task_view_type_check.CallableClass'> [2021-10-01 21:06:10,977] {dagbag.py:366} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping. [<TaskInstance: example_bash_operator.runme_0 test_dag_runs_action [success]>, <TaskInstance: example_bash_operator.runme_1 test_dag_runs_action [failed]>] ``` I can create an issue to update the `test_set_dag_runs_action` test such that the rows retrieved as sorted consistently or explicitly checking the state of individual `TaskInstance` states. WDYT? Either solution should fix the flaky test IMO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org