ChickenBenny commented on issue #75865: URL: https://github.com/apache/airflow/issues/75865#issuecomment-6634911843
Found another one while testing rc2: #76190 Reproduced on 3.4.0rc2 (Breeze, Postgres, LocalExecutor) with the Dag from the issue. Setup: a `fail_fast=True` Dag where a task gets removed or renamed while a run is still going (e.g. a deploy lands in the middle of a long run). When the next task fails: - The API server returns 500 on the state update (`KeyError: 'victim'` in `_stop_remaining_tasks`). - The scheduler hits the same `KeyError` in `handle_failure` and exits. - After a restart it re-queues the orphaned TI, the task fails again, and the scheduler dies again. In my run the scheduler exited 6 times in about 5 minutes, and `boom` (`retries=0`) got to try 7. Not a regression though: `_stop_remaining_tasks` is the same in 3.3.2, so I don't think it should block 3.4.0. Fix is in #76191, would be nice to get it into 3.4.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
