This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-7-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v2-7-test by this push:
new 7536737e23 Attempt to fix the flaky TestTriggererJob test (#34075)
7536737e23 is described below
commit 7536737e23c944ffacb9b737ddf047a56341ae99
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Mon Sep 4 16:40:24 2023 +0200
Attempt to fix the flaky TestTriggererJob test (#34075)
As documented in #33323, we have frequent failures of the flaky
triggerer job tests.
The flaky failures are about some errors when we close all the
sessions in teardown of the test. It turns out that the tests
had side-effect - they have not waited for the TriggererJob thread
to complete, they merely marked them to be stopped, but they have
not waited for those to complete - this is quite plausible
explanation of the flaky test failures - since those threads have
1 second sleep, it's more than likely that the session has been
created and used by the thread while the teardown has been attempting
to close all the sessions.
This side effect could also have an effect for other tests that
were run after - because in a busy test run machine, the side
effects could propagate further than just to the teardown, so it
could also explain why sometimes (very rarely) other job tests
failed with similar errors.
The fix is to join the runner after marking it to be stopped.
Fixes: #33323 (Hopefully)
(cherry picked from commit 47f79b9198f3350951dc21808c36f889bee0cd06)
---
tests/jobs/test_triggerer_job.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/tests/jobs/test_triggerer_job.py b/tests/jobs/test_triggerer_job.py
index 35ebe99b31..e79202d6f6 100644
--- a/tests/jobs/test_triggerer_job.py
+++ b/tests/jobs/test_triggerer_job.py
@@ -143,6 +143,8 @@ def test_trigger_logging_sensitive_info(session, capsys):
finally:
# We always have to stop the runner
triggerer_job_runner.trigger_runner.stop = True
+ triggerer_job_runner.trigger_runner.join(30)
+
# Since we have now an in-memory process of forwarding the logs to stdout,
# give it more time for the trigger event to write the log.
time.sleep(0.5)
@@ -257,6 +259,7 @@ def test_trigger_lifecycle(session):
finally:
# We always have to stop the runner
job_runner.trigger_runner.stop = True
+ job_runner.trigger_runner.join(30)
class TestTriggerRunner:
@@ -408,7 +411,7 @@ def test_trigger_create_race_condition_18392(session,
tmp_path):
pytest.fail("did not observe 2 loops in the runner thread")
finally:
job_runner.trigger_runner.stop = True
- job_runner.trigger_runner.join()
+ job_runner.trigger_runner.join(30)
thread.join()
instances = path.read_text().splitlines()
assert len(instances) == 1
@@ -500,7 +503,7 @@ def test_trigger_runner_exception_stops_triggerer(session):
finally:
job_runner.trigger_runner.stop = True
# with suppress(MockTriggerException):
- job_runner.trigger_runner.join()
+ job_runner.trigger_runner.join(30)
thread.join()
@@ -531,6 +534,7 @@ def test_trigger_firing(session):
finally:
# We always have to stop the runner
job_runner.trigger_runner.stop = True
+ job_runner.trigger_runner.join(30)
def test_trigger_failing(session):
@@ -564,6 +568,7 @@ def test_trigger_failing(session):
finally:
# We always have to stop the runner
job_runner.trigger_runner.stop = True
+ job_runner.trigger_runner.join(30)
def test_trigger_cleanup(session):
