The GitHub Actions job "Scheduled verify release calendar" on airflow.git/main has failed. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: d4a4d853e9d6f32b212d85d514666bd5848927cb / Hemkumar Chheda <[email protected]> Fix duplicate heartbeat-timeout task callbacks (#68008) * Fix duplicate heartbeat-timeout task callbacks * Set task_callback_type on heartbeat-timeout callback based on retry eligibility Address review feedback: the heartbeat-timeout TaskCallbackRequest didn't set task_callback_type, so a timed-out task with retries left could dispatch on_failure_callback instead of on_retry_callback. Mirrors the existing externally-killed-task pattern. * Load task and send email in heartbeat-timeout purge path The purge path called handle_failure() without ever loading ti.task, so fail_fast (ti.task.dag.fail_fast) silently no-opped and the external-kill email path in process_executor_events never ran for these TIs since the TI already left RUNNING/RESTARTING by the time any executor event for it was processed. It also computed task_callback_type with a max_tries > 0 guard that could disagree with what handle_failure() actually persists for a RESTARTING TI with max_tries=0. Load ti.task before calling handle_failure(), decide the callback type with plain is_eligible_to_retry() (matching fetch_handle_failure_context exactly), and send an EmailRequest alongside the existing TaskCallbackRequest. closes: #42553 * Lock and revalidate TI state in heartbeat-timeout purge The heartbeat scan selected RUNNING/RESTARTING TIs unlocked, and the same session flowed into the purge. Between the scan and ti.handle_failure(), a worker could commit the task as SUCCESS; handle_failure() refreshes the TI but sets FAILED unconditionally, clobbering that terminal state after having already enqueued a failure callback. Lock the scan query with with_row_locks(of=TI, skip_locked=True) so a worker can't change the row between the scan and the handle_failure() that follows in the same transaction, matching how process_executor_events and the other TI-mutating paths in this file lock. As defense in depth, revalidate each row's committed state at the top of the purge loop and skip it (no callback, no email, no handle_failure) if it is no longer RUNNING/RESTARTING. closes: #42553 * Address review nits on heartbeat-timeout purge Use a real MockExecutor(do_update=False) instead of a bare MagicMock() in test_heartbeat_timeout_converges_ti_state_before_next_scan, matching the other heartbeat-timeout tests, and assert on callback_sink.send. Reword the _resolve_ti_callback_bundle_info docstring: it is used by the heartbeat-timeout purge path; process_executor_events inlines the same resolution rather than sharing this helper. Report URL: https://github.com/apache/airflow/actions/runs/29898102016 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
