github-actions[bot] opened a new pull request, #66606:
URL: https://github.com/apache/airflow/pull/66606
* fix(db_cleanup): add --error-on-cleanup-failure flag to airflow db clean
airflow db clean suppresses all per-table cleanup errors via
_suppress_with_logging() and exits 0 even when tables could not be
cleaned. This makes it impossible to detect silent failures in automated
DAG-based maintenance workflows, which can lead to unchecked table
growth and eventual migration failures on upgrade.
This commit adds an opt-in --error-on-cleanup-failure flag that causes
run_cleanup() to raise AirflowException (and the CLI to exit 1) if any
table cleanup encountered an error. Default behaviour is unchanged.
Additionally, a warning listing all tables that were not cleaned is now
always emitted when failures occur, even without the flag, improving
observability without requiring any opt-in.
Changes:
- airflow/utils/db_cleanup.py: update _suppress_with_logging to track
whether an exception was suppressed via a SimpleNamespace context
object; collect failed table names in run_cleanup(); emit a warning
summary and optionally raise AirflowException.
- airflow/cli/cli_config.py: add ARG_DB_ERROR_ON_CLEANUP_FAILURE and
wire it into the db clean ActionCommand args list.
- airflow/cli/commands/db_command.py: forward error_on_cleanup_failure
from CLI args to run_cleanup().
- tests/utils/test_db_cleanup.py: add unit tests covering the new flag
and the warning summary behaviour.
Made-with: Cursor
* fix(db_cleanup): improve type annotations and add docs for
--error-on-cleanup-failure
- Use collections.abc.Generator for the _suppress_with_logging return
type annotation (ruff UP035 compliant) instead of typing.Generator.
- Expand the _suppress_with_logging docstring to describe the yielded
SimpleNamespace context object and the failure-tracking behaviour.
- Add a new "Detecting cleanup failures" section to
docs/howto/usage-cli.rst documenting the --error-on-cleanup-failure
flag and the --skip-archive recommendation for large tables.
Made-with: Cursor
* fix(db_cleanup): address CI failures on PR #65239
- Fix mypy arg-type errors: OperationalError third argument must be a
BaseException, not None. Replace OperationalError("", {}, None) with
OperationalError("", {}, Exception("mock db error")) in three new
tests in test_db_cleanup.py.
- Fix ruff ISC violation: collapse implicit string concatenation in the
run_cleanup() warning call into a single string literal.
- Update existing CLI tests in test_db_command.py to include the new
error_on_cleanup_failure=False kwarg in all ten
assert_called_once_with assertions.
- Add test_error_on_cleanup_failure to test_db_command.py to verify the
--error-on-cleanup-failure flag is correctly forwarded to run_cleanup.
Made-with: Cursor
* fix(db_cleanup): add --error-on-cleanup-failure flag documentation and
tests
- Introduce a new news fragment detailing the addition of the
``--error-on-cleanup-failure`` flag to the ``airflow db clean`` command,
allowing for better error handling during table cleanup.
- Update unit tests in `test_db_cleanup.py` to ensure proper functionality
of the new flag, including checks for raised exceptions and warning messages
for failed tables.
- Adjust the known exceptions list to reflect changes in `db_cleanup.py`.
* fix(db_cleanup): update error handling for table cleanup failures
- Change the behavior of the `--error-on-cleanup-failure` flag to raise a
RuntimeError instead of an AirflowException when table cleanup encounters
errors.
- Update the documentation and help text for the flag to clarify its
functionality.
- Ensure that warning messages for failed tables are always emitted,
regardless of the flag's state.
- Modify unit tests in `test_db_cleanup.py` to reflect the new error
handling and verify the correct logging behavior.
This update improves error visibility during automated workflows by ensuring
that cleanup failures are properly reported.
* fix(db_cleanup): remove redundant cleanup commit
(cherry picked from commit 4364908da2c0e798800509f2427fce90777af3ec)
Co-authored-by: Hemkumar Chheda <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
