roshanprabu opened a new pull request, #71308: URL: https://github.com/apache/airflow/pull/71308
## Summary `_run_task_and_map_outcome` in the Task SDK's `task_runner.py` logged every task failure via `log.exception` (ERROR level, with traceback) unconditionally -- even when the failure was about to be retried automatically. For a DAG with many task instances hitting transient errors, this means error-tracking integrations (Sentry, etc.) get paged on every retry attempt, not just the final, terminal failure. The retry-vs-fail decision is already computed by `_handle_current_task_failed` (via `ti._ti_context_from_server.should_retry` and any custom retry policy) immediately after the log call used to happen. This moves the log call after that decision and adds `_log_task_failure`, which: - logs at `WARNING` (with the exception attached via `exc_info=`) when the outcome is `UP_FOR_RETRY` - logs at `ERROR` via `log.exception` for the final, non-retried failure `AirflowFailException`, `AirflowSensorTimeout`, and `AirflowTaskTerminated` are untouched -- those are already documented in this file as never eligible for retry, so they keep unconditional `log.exception`. Closes: #32246 ## Test plan - [x] Updated `test_task_runner_not_fail_on_failed_callback`'s `retry` case (`task-sdk/tests/task_sdk/execution_time/test_task_runner.py`) to assert the new `log.warning` call instead of expecting `log.exception` for the retry-eligible path; the `failure` (non-retry) case still asserts `log.exception`, unchanged. - [x] Reviewed all other tests referencing `"Task failed with exception"` / `UP_FOR_RETRY` in this file and confirmed none assert on retry-path `log.exception` calls other than the one updated. - [x] `ruff check` and `ruff format --check` pass on both changed files. Note: I wasn't able to run the full suite in this environment (`task-sdk`'s tests require `apache-airflow-core` installed, which isn't published for this pre-release version, and building it locally needs Docker/breeze which wasn't available here). The change was verified via careful manual trace of the exact control flow instead -- happy to have CI/a maintainer double check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
